The Flaw of Averages

1. The Flaw of Averages: Why relying on average values leads to underestimating risk

Plans based on average assumptions are wrong on average.

Averages mislead. The Flaw of Averages occurs when single average numbers are used to represent uncertain values in plans or forecasts. This can lead to significant underestimation of risks and poor decision-making. For example:

• A statistician drowning in a river with an average depth of 3 feet
• An investment portfolio based on average returns, ignoring potential extreme losses
• A production plan using average demand, not accounting for fluctuations

The problem is exacerbated when multiple averages are combined, as the errors compound. To avoid the Flaw of Averages, it's crucial to consider the full range of possible outcomes and their probabilities, rather than relying on a single average value.

2. Probability Management: A new approach to modeling uncertainty

Probability Management may be viewed as a data management system in which the entities being managed are not numbers, but uncertainties, that is, probability distributions.

Managing distributions, not numbers. Probability Management is a framework for handling uncertainty in decision-making and risk assessment. Key components include:

• Scenario Libraries: Databases of potential future outcomes for uncertain variables
• SIPs (Stochastic Information Packets): Collections of scenarios for a single uncertainty
• SLURPs (Scenario Libraries Unit with Relationships Preserved): Collections of interrelated SIPs

This approach allows organizations to:

• Model complex systems with multiple uncertainties
• Share and consolidate risk information across departments
• Make more informed decisions by considering full ranges of possibilities

3. Interactive Simulation: Connecting intellect to intuition in decision-making

Interactive simulation does for uncertainty what the lightbulb does for darkness.

Instant feedback on uncertainties. Interactive simulation allows users to manipulate variables in real-time and immediately see the effects on outcomes. This powerful tool:

• Bridges the gap between analytical models and intuitive understanding
• Enables rapid exploration of "what-if" scenarios
• Helps decision-makers develop a gut feel for complex systems

For example, portfolio managers can instantly see how changing asset allocations affects risk and return profiles. This immediate feedback loop helps calibrate intuition and improve decision-making under uncertainty.

4. The Power of Diversification: Reducing risk through portfolio theory

Diversification is a Green Word for the CENTRAL LIMIT THEOREM.

Spread risk, increase stability. Diversification is a fundamental principle in risk management, based on the statistical concept of the Central Limit Theorem. Key points:

• Combining uncorrelated or negatively correlated assets reduces overall portfolio risk
• The effect is most dramatic when moving from one to two assets
• As more assets are added, the portfolio's behavior becomes more predictable

Examples:

• Investment portfolios with stocks from different sectors
• Movie studios producing a slate of films rather than betting on a single blockbuster
• Countries diversifying energy sources to reduce dependence on a single supplier

5. Real Options: Exploiting uncertainty for profit

The average value of an option is not the value of the option given the average future price of the stock.

Flexibility creates value. Real options are opportunities to make decisions in the future based on new information. They can significantly increase the value of projects or investments by:

• Providing the right, but not the obligation, to take certain actions
• Allowing managers to capitalize on upside potential while limiting downside risk

Examples of real options:

• The option to abandon a project if conditions become unfavorable
• The flexibility to expand production if demand increases
• The ability to switch between fuel sources in a power plant

Valuing real options requires considering the full distribution of possible outcomes, not just averages.

6. Visualizing Uncertainty: The importance of shapes and scatter plots

An uncertain number is a SHAPE known as its distribution.

See the whole picture. Visualizing uncertainty through shapes (distributions) and scatter plots provides crucial insights that averages alone cannot convey. Key benefits:

• Histograms show the full range of possible outcomes and their relative likelihoods
• Cumulative distribution functions display probabilities of exceeding specific values
• Scatter plots reveal relationships between variables that may not be linear

Tools like interactive dashboards and scenario explorers allow decision-makers to:

• Identify potential extreme events or "black swans"
• Understand complex relationships between variables
• Communicate risks and uncertainties more effectively to stakeholders

7. The Chief Probability Officer: Managing risk at the enterprise level

In the land of the averages, the man with the wrong distribution is king.

Coordinating uncertainty. The Chief Probability Officer (CPO) is a proposed role for managing uncertainty and risk across an organization. Responsibilities include:

• Developing and maintaining scenario libraries of key uncertainties
• Ensuring coherence and consistency in risk modeling across departments
• Challenging excessive certainty and promoting probabilistic thinking

The CPO differs from the traditional Chief Risk Officer by:

• Focusing on distributions rather than single-number risk metrics
• Promoting transparency and understanding of uncertainties
• Facilitating better decision-making throughout the organization

8. The False Positive Problem: Understanding probabilities in rare event detection

The chance is only one in 1,000 that a report will result in the FBI's nabbing a true terrorist, even with a 99 percent accurate detector.

Rarity skews results. The false positive problem arises when trying to detect rare events in large populations. Key points:

• Even highly accurate tests can produce mostly false positives when the event is rare
• The problem is counterintuitive and often underestimated by decision-makers
• It affects areas such as medical screening, terrorism detection, and fraud prevention

To address the false positive problem:

• Consider the base rate of the event in the population
• Use multiple, independent tests when possible
• Focus on improving the specificity of detection methods

9. Climate Change and Health Care: Applying probability management to complex issues

The Flaw of Averages creates trouble at organizational boundaries.

Crossing silos creates challenges. Complex issues like climate change and health care involve multiple uncertainties across different domains. Probability management can help by:

• Providing a common language for discussing and modeling uncertainties
• Enabling the integration of diverse data sources and models
• Highlighting potential extreme scenarios that averages might miss

Examples:

• Climate models incorporating uncertainties in emissions, feedback loops, and tipping points
• Health care decisions considering uncertainties in treatment efficacy, side effects, and costs

By explicitly modeling these uncertainties and their relationships, organizations can make more robust decisions and better prepare for potential outcomes.

10. The Distribution String (DIST): A new data type for uncertainty

The DIST data type was designed to be universally acceptable to spreadsheet and database software, and for parts of it to be human-readable as well, with the name and average prominently visible.

Standardizing uncertainty. The Distribution String (DIST) is a compact way to represent probability distributions in software systems. Benefits include:

• Efficient storage and transfer of uncertainty information
• Compatibility across different software platforms and tools
• Ability to perform calculations directly on distributions

The DIST format enables:

• Scenario libraries to be easily shared and integrated
• Interactive simulations to run more quickly
• Auditing and certification of risk models

By providing a standard format for representing uncertainty, DISTs facilitate the widespread adoption of probability management techniques across organizations and industries.

Last updated: August 9, 2024