Fooled by Randomness

1. Randomness dominates our world, yet we struggle to perceive it

We are not naturally equipped to process probabilities; our brain is more receptive to the computation of frequencies.

Pattern-seeking machines. Humans are hardwired to seek patterns and causality, even where none exist. This tendency, while useful in many situations, often leads us astray when dealing with randomness. We tend to attribute skill to what is often just luck, and we struggle to recognize the role of chance in both success and failure.

Illusion of control. Our brains create narratives to explain events, giving us a false sense of understanding and control. This illusion can be dangerous, particularly in fields like finance, where randomness plays a significant role. Recognizing the limits of our knowledge and the prevalence of randomness is crucial for making better decisions and avoiding costly mistakes.

Areas highly susceptible to randomness:

• Financial markets
• Business success/failure
• Career trajectories
• Scientific discoveries
• Technological innovations

2. Success often results from luck, not skill

It is not how likely you are to succeed that matters; it is how much you stand to gain if you do and how much you stand to lose if you don't.

Asymmetry of outcomes. In many fields, particularly those with high uncertainty, the distribution of outcomes is highly skewed. A small number of individuals or events can have an outsized impact, while the majority cluster around the average. This asymmetry means that luck often plays a more significant role in success than we'd like to admit.

Survivorship bias. We tend to focus on successful individuals or strategies, ignoring the vast majority that failed. This creates a distorted view of what leads to success and can lead to dangerous generalizations. Understanding the role of randomness helps us avoid this trap and make more realistic assessments of skill and strategy.

Examples of luck-driven success:

• Bestselling authors
• Startup founders
• Investment managers
• Viral social media content
• Scientific breakthroughs

3. Our minds are not equipped to handle probabilities effectively

Our brain is not cut out for nonlinearities. People think that if, say, two variables are causally linked, then a steady input in one variable should always yield a result in the other one.

Cognitive limitations. Our brains evolved to handle immediate, concrete threats and opportunities, not abstract probabilities. This mismatch leads to systematic errors in judgment when dealing with uncertain situations. We tend to overestimate our ability to predict outcomes and underestimate the role of chance.

Heuristics and biases. To cope with complexity, our minds use mental shortcuts (heuristics) that can lead to predictable biases. These biases, such as the availability heuristic or confirmation bias, can severely distort our perception of probabilities and risk. Recognizing these biases is the first step in mitigating their effects on our decision-making.

Common probabilistic errors:

• Gambler's fallacy
• Base rate neglect
• Conjunction fallacy
• Overconfidence effect
• Recency bias

4. Survivorship bias skews our perception of success

I wondered how many historians use luck in their interpretation of success—or how many are conscious of the difference between process and result.

Hidden failures. We often only see and study the survivors, whether in business, investing, or any other field. This creates a distorted view of what leads to success, as we ignore the vast majority of attempts that failed. Understanding survivorship bias helps us avoid drawing false conclusions from limited data.

Process vs. outcome. Focusing solely on outcomes can lead to flawed reasoning. A good process can sometimes lead to bad outcomes due to randomness, while a poor process can occasionally yield good results. Evaluating the decision-making process, rather than just the outcome, provides a more accurate assessment of skill and strategy.

Areas prone to survivorship bias:

• Business case studies
• Investment strategies
• Self-help advice
• Historical narratives
• Career paths

5. Rare events (black swans) have outsized impacts

A Black Swan is an event with the following three attributes: rarity, extreme impact, and retrospective (though not prospective) predictability.

Power of the unexpected. Rare, high-impact events often shape history more than the steady progression of expected outcomes. These "black swans" can upend our understanding of the world and render many predictive models useless. Recognizing the potential for such events is crucial for robust decision-making.

Preparing for the unpredictable. While we can't predict specific black swan events, we can build systems and strategies that are resilient to their impacts. This involves maintaining flexibility, avoiding excessive leverage, and cultivating a mindset that embraces uncertainty rather than denying it.

Historical black swan events:

• 9/11 terrorist attacks
• 2008 financial crisis
• Internet revolution
• COVID-19 pandemic
• Major scientific discoveries

6. Emotions and cognitive biases cloud our judgment

We favor the visible, the embedded, the personal, the narrated, and the tangible; we scorn the abstract.

Emotional decision-making. Our emotions play a significant role in our decision-making, often overriding logical analysis. This can lead to poor choices, especially in high-stakes situations involving risk and uncertainty. Recognizing the influence of emotions on our judgment is crucial for making more rational decisions.

Cognitive biases. Our brains are subject to numerous cognitive biases that distort our perception of reality and probability. These biases, such as confirmation bias or the sunk cost fallacy, can lead us to make systematic errors in judgment. Understanding and actively countering these biases can significantly improve our decision-making abilities.

Key cognitive biases affecting probability assessment:

• Availability heuristic
• Anchoring bias
• Overconfidence effect
• Hindsight bias
• Narrative fallacy

7. Embrace uncertainty and practice intellectual humility

I consider myself as prone to foolishness as anyone I know, in spite of my profession and the time spent building my expertise on the subject.

Limits of knowledge. Recognizing the limits of our knowledge and the prevalence of uncertainty is a crucial step in making better decisions. This intellectual humility allows us to remain open to new information and adjust our beliefs in light of evidence.

Probabilistic thinking. Instead of seeking certainty, we should strive to think probabilistically. This means considering multiple possible outcomes, assigning probabilities to each, and making decisions based on expected values rather than single-point predictions.

Benefits of embracing uncertainty:

• Reduced overconfidence
• Increased adaptability
• Better risk management
• Improved decision-making
• Enhanced learning from mistakes

8. Focus on process, not outcomes

It is a fact that our brain tends to go for superficial clues when it comes to risk and probability, these clues being largely determined by what emotions they elicit or the ease with which they come to mind.

Long-term perspective. In fields dominated by randomness, focusing on short-term outcomes can be misleading. Instead, we should evaluate decisions based on the quality of the process used to make them. A good process will lead to better outcomes in the long run, even if individual results are subject to random variation.

Continuous improvement. By focusing on process, we create opportunities for continuous learning and improvement. Each decision becomes a chance to refine our approach, rather than a binary success or failure. This mindset is particularly valuable in fields with high uncertainty, where outcomes can be heavily influenced by chance.

Elements of a good decision-making process:

• Considering multiple perspectives
• Seeking disconfirming evidence
• Assessing probabilities of various outcomes
• Evaluating potential impacts (both positive and negative)
• Learning from past decisions and outcomes

9. Develop strategies to manage randomness in life and work

We need tricks to get us there but before that we need to accept the fact that we are mere animals in need of lower forms of tricks, not lectures.

Practical techniques. While we can't eliminate randomness, we can develop strategies to manage its effects. This includes diversification in investing, maintaining flexibility in planning, and building robust systems that can withstand unexpected shocks.

Mental models. Developing a toolkit of mental models helps us navigate complex, uncertain environments. These models, drawn from various disciplines, provide frameworks for thinking about probability and risk in more sophisticated ways.

Strategies for managing randomness:

• Diversification (don't put all eggs in one basket)
• Antifragility (systems that benefit from volatility)
• Scenario planning
• Regular reassessment of beliefs and strategies
• Maintaining optionality

Last updated: January 22, 2025