A Decision is a Bet on the Future based on our Beliefs (part 1) - Resulting
Just completed Annie Duke's live online course on decision-making
“… there are only two things that determine how your life turns out: luck and the quality of your decisions. You have control over only one of those two things.”
“… your decision-making is the single most important thing you have control over that will help you achieve your goals.”
Annie Duke, How to Decide: Simple Tools for Making Better Choices. Penguin Publishing Group, 2020.
I have been fascinated with decision-making since I took my first course in decision analysis in medical school. My instructor was Milton Weinstein who wrote the classic “Clinical Decision Analysis.”1 My journey exposed me to a variety of quantitative and qualitative approaches.2 As a public health official I experimented applying these concepts in the real world. In public health our key challenge is making high stakes decisions3 in the face of uncertainty, trade-offs, and time constraints. The challenge with most methods is that they require good information, capable teams, and adequate time. This is rarely the case. And, when they are implemented, they exact an exhausting cognitive load making it difficult to sustain. It becomes much easier to default to our intuition, heuristics, or deferring decision-making to others.
This is where Annie Duke comes in. She has developed an effective, practical approach that is grounded in cognitive psychology, team science, and continuous improvement. Her approach can be deployed quickly as an individual or as a team.
She starts by helping us understand and mitigate the unconscious barriers that prevent us from becoming better decision makers. Her approach is foundational like the roots of a tree — the deeper and stronger the roots, the better our decisions. To this foundation one can add advanced methods when feasible.
I highly recommend Annie Duke’s:
Substack blog, or
live online course (Use PUBLICHEALTH25 to get 25% discount. I do not get any money for informing readers of this discount.)
In her book, How to Decide, Annie asks us to reflect on the following questions:
What was your best decision last year? Go with your gut and write it down.
What was your worst decision last year? Go with your gut and write it down.
Now, answer the next two questions.
Did your best decision have a good outcome?
Did your worst decision have a bad outcome?
On average, we answer yes, “the best decision had a good outcome,” and yes, “the worst decision had a bad outcome.” In general, this is flawed thinking: we judge the quality of a decision based on the outcome. Remember, a good decision can have a bad outcome, and a bad decision can have a good outcome. This cognitive bias is automatic (and often emotional) and is called “outcome bias” or “resulting” — the term Annie uses. Resulting hinders our ability to objectively and accurately judge the quality of our decisions (aka, “decision quality” or DQ). Once you are aware of this cognitive bias, you will see it everywhere including in yourself.
Resulting (outcome bias)
Think about the 2024 presidential election. Kamala Harris loss to Donald Trump. In the days to weeks following the loss, many articles were published criticizing the decisions of the Harris campaign (eg, not taking an interview with podcaster Joe Rogan). This is resulting!!!
Had Harris won the election, the opposite would have occurred. There would have been many articles praising her brilliant campaign decisions (eg, not taking an interview with podcaster Joe Rogan).
What about President Biden’s very late decision to drop out of the race? I am sure we all have very strong opinions. The strength of our opinions is what keeps us from becoming better decision makers because we jump to conclusions. The truth is we know nothing about the quality of the decision-making processes that led to his decisions.4
Resulting is a systematic bias; it is a biased retrospective analysis where we use a good or bad outcome to draw conclusions about the quality of the decision process. Also, bad outcomes are filled with negative emotions which prevent us from thinking straight.
How does this work? Because of the self-serving bias (SSB), we attribute good outcomes to internal factors (“I make good decisions.”), and we attribute bad outcomes to external factors (bad luck, blame others, etc.).
“The self-serving bias describes our tendency to attribute positive outcomes and successes to internal factors like our personal traits, skills, or actions but attribute negative results or failures to external factors, shifting the blame to situational factors beyond our control, such as bad luck or the actions of others.”5
Even worse, because the fundamental attribution error (FAE), for a given outcome, we judge ourselves using different criteria than we judge others. For example, for a bad outcome, if it was my decision, I attribute the outcome to a bad luck (situational factor); if it was someone else’s decision, I attribute the outcome to a bad decision (dispositional factor). Conversely, for a good outcome, if it was my decision, I attribute the outcome to a good decision (dispositional factor); if it was someone else’s decision, I attribute the outcome to dumb luck (situational factor).
“The fundamental attribution error describes how, when making judgments about people’s behavior, we often overemphasize dispositional factors and downplay situational ones. In other words, we believe that people’s personality traits have more influence on their actions, compared to the other factors outside of their control.”6
Resulting is a serious systematic bias that prevents us from evaluating the quality of our decisions, and it prevents us from learning from our experiences. Resulting operates through the self-serving bias and the fundamental attribution error.
An exception to the self-serving bias is when we blame ourselves (see upper right box in Figure 2). This may be well intentioned because we may be exercising self-accountability. The big problem is that none of these conclusions help us learn and improve our decision making!
Resulting is such a serious bias that Annie Duke dedicates a whole chapter to it. Resulting prevents us from evaluating and improving decision quality. To improve decision quality, first we need to understand what affects decision-making (Figure 3).
Decisions are affected by our beliefs and states of knowledge, including our skills in causal and probabilistic reasoning. Cognitive biases are created by prior beliefs and belief-consistent information processing.7 Cognitive biases affect how we seek and close information gaps and how we evaluate evidence. Mitigating cognitive biases is central to improving our decisions. In other words, decision processes should be designed to prevent or mitigate the worst cognitive biases. We are starting with the biases that prevent us from learning from our decision-making experiences.
In subsequent blogs, I will define decision quality more precisely and how to achieve it with well-designed team processes, meetings, and methods. If we are able to ensure decision quality and prospectively evaluate outcome quality (Figure 4), then we have the basis for learning and continuous improvement.
Here are some closing words from Annie Duke:8
“Making better decisions starts with learning from experience. Resulting interferes with that learning, causing you to repeat some low-quality decisions and stop making some high-quality decisions. It also keeps you from examining good-quality/good-outcome decisions (as well as bad-quality/bad-outcome decisions), which still offer valuable lessons for future decisions.”
“Resulting reduces compassion when it comes to how we treat others and ourselves.”
Annie Duke says there are three types of people: Those who …
do not know about DQ methods,
know about DQ methods but choose not to use them, and
know about DQ methods and choose to use them.
Which one are you? Which one will you be?
Appendix - Example from Super Bowl XLIX (49)
On February 1, 2015, the Seattle Seahawks had the football on the 1-yard line of New England Patriots. It was 2nd down with 26 seconds remaining. With one time out remaining, the Seahawks were down 24 to 28. Everyone expected quarterback Russell Wilson to handoff the ball to superstar running back Marshawn Lynch.
Instead, Russell Wilson attempted a quick pass which was intercepted by defensive back Malcolm Butler, and the Patriots won the Super Bowl!
Everyone was shocked!
Here are some of the responses after the game:
USA Today: “What on Earth was Seattle Thinking with Worst Play Call in NFL History?”
Washington Post: “ ‘Worst Play-Call in Super Bowl History’ Will Forever Alter Perception of Seahawks, Patriots”
FoxSports.com: “Dumbest Call in Super Bowl History Could be Beginning of the End for Seattle Seahawks”
Seattle Times: “Seahawks Lost Because of the Worst Call in Super Bowl History”
The New Yorker: “A Coach’s Terrible Super Bowl Mistake”
Coach Prime, @DeionSanders, on Twitter: “That was the worst play call in the history of the Superbowl!!! Worst QB decision Ever!!!!! Ever Ever! Naw I mean Ever!”
Clearly, it was a bad outcome for the Seahawks. But, was it a bad decision? The overwhelming consensus was YES, this was obviously as BAD decision! This is resulting!
It turns out that the probability of an interception was about 2% and the probability of catching a pass for a touchdown was about 37%. A quick pass would enable them to run two more plays, including up to two running plays with Marshawn Lynch.
It was a GOOD decision with a BAD outcome. Seattle Seahawks Coach Pete Carroll had a good process to ensure decision quality.
Footnotes
Milton C. Weinstein and Harvey V. Fineberg, Clinical Decision Analysis (Philadelphia: Saunders, 1980).
Tomás J. Aragón, “What Should I Read to Improve My Decision Making? Improving Human Decision Intelligence,” Team Public Health (blog), June 23, 2024, https://teampublichealth.substack.com/p/what-should-i-read-to-improve-my.
Tomás J. Aragón et al., “Crisis Decision-Making at the Speed of COVID-19: Field Report on Issuing the First Regional Shelter-in-Place Orders in the United States,” Journal of Public Health Management and Practice 27, no. Supplement 1 (January 2021): S19–28, https://doi.org/10.1097/PHH.0000000000001292.
I would bet that President Biden’s team did not use evidence-based methods for ensuring decision quality. How many of us do?
Aileen Oeberst and Roland Imhoff, “Toward Parsimony in Bias Research: A Proposed Common Framework of Belief-Consistent Information Processing for a Set of Biases,” Perspectives on Psychological Science 18, no. 6 (November 2023): 1464–87, https://doi.org/10.1177/17456916221148147.
Annie Duke, How to Decide: Simple Tools for Making Better Choices (Penguin Publishing Group, 2020), https://www.annieduke.com/books/.