Hosts – Marc Andreessen (@pmarca) and Sonal Chokshi (smc90)
Annie Duke (@AnnieDuke) is a professional poker player
  • She’s the author of Thinking in Bets: Making Smarter Decisions When You Don’t Have All the Facts
  • Annie has also founded the national decision education movement How I Decide
Once you know the outcome of a decision, it’s hard to look back retrospectively to determine its quality (this is known as resulting)
Analytics show that in the NFL, if you’re on your own one-yard line on 4th down, you should always go for it
  • Why? – If you punt, you’ll likely only punt to mid-field (so the other team is essentially guaranteed 3 points anyway)
  • BUT – this is very uncommon, most teams punt in this situation
Human beings very often choose the less controversial option (to avoid judgment) even though it might not be the most statistically optimal
Marc brings up the above 2×2 grid (right/wrong is related to the outcome)
  • “Consensus/right is fine, non-consensus/right is fine, consensus/wrong is fine, non-consensus/wrong is REALLY bad”
Some notes:
  • For consensus/wrong decisions, that consensus almost always provides an invisible clock against blame
    • For example – If someone dies in a car crash, we don’t say, “What a moron for getting in a car”
  • If an autonomous vehicle kills a pedestrian, it currently falls under the “non-consensus/wrong category” 
    • Now consider the reaction when a pedestrian dies because of a human driver – it falls under the “consensus/wrong category”
  • Similarly, for market crashes:
    • When a crash results from human beings selling stock, people say – “The market went down today”
    • But when an algorithm causes a crash – It’s a “flash crash”
      • This results in people pointing fingers and discussing whether algorithmic trading should even be allowed
It’s very hard to get the non-consensus/right results unless you’re willing to take risks and take on the non-consensus/wrong results
  • But people just emotionally can’t handle the non-consensus/wrong outcomes and will do nearly anything to avoid them (no matter how much we understand the above)
  • What can we do about this?
    • Revert from:
      • Giving non-consensus/right decisions too much praise
      • Giving non-consensus/wrong too much criticism
In the workplace, outcomes tend to dominate
  • If sales were 10% above expected, everything’s fine and everybody goes about their day (no one’s trying to figure out why this happened)
  • But if sales were 10% lower than expected – all hell breaks loose, resulting in a discussion about what went wrong
“You have to make it very clear to the people who work for you that you understand good outcomes will come from good processes”
Advice for the workplace:
  • Focus more on forecast quality (aka the model quality) rather than outcome quality
    • In addition, know in advance what would have to be true (or the circumstance that would have to present themselves) for you to rethink/re-evaluate your model
  • Always look up, down, and orthogonal – ask:
    • “Could we have lost less?”
    • “Should we have lost more?”
      • For example:
        • If sales were 10% lower than expected, don’t just try to figure out why sales weren’t 10% higher, also consider the fact that perhaps you should have lost more, and you actually got lucky
        • In the VC world, why does no one consider the question (related to a bad investment) – “Maybe we should have invested more money and we actually got lucky?”
    • “Should we have even been in this position at all?”
  • ALSO – In investing, when discussing a win, consider the fact that you may have “oversized the bet” and actually got bailed out by a fluke win
“Not making a decision is making a decision, we just don’t think about it that way”
  • This has to do with ALL the resources you invest – time, money, attention, and energy
REALIZE – There is a set of possible futures that exist for NOT making a decision, in addition to the possible futures that exist from choosing among a set of options
When considering options for a decision, zoom out and ask:
  • What does my life look like going with Option A/B/C a year from now?
Related to this: “Doing certain things today is like stealing from your future self” – Sonal
When soliciting feedback, don’t do it in a group setting (to avoid the pile-on effect)
  • Go to people individually
“Don’t confuse confidence and certainty”
  • You can express uncertainty in a decision and still convey confidence
    • For example:
      • Option A will work out 60% of the time (what you choose)
      • Option B will work out 30% of the time
      • Option C will work out 10% of the time
  • To better express uncertainty when talking to people, use percentages
    • For example – “I’ll get this to you by Friday 80% of the time and by Monday 99% of the time”
Instead of asking, “Are you sure?”, ask – “How sure are you?”
You don’t need to improve that much to get really big dividends
  • You make THOUSANDS of decisions per day
    • Getting even a little better at the art of decision-making by removing biases and thinking more probabilistically will prove very worthwhile