The Tiger That Isn't

1. Numbers can deceive: Understanding the context is crucial

Numbers, pure and precise in abstract, lose precision in the real world.

Counting is complex. In the real world, numbers are rarely as straightforward as they appear. What seems like a simple count often involves numerous definitions, assumptions, and compromises. For example, counting cancer cases or unemployment figures requires decisions about what exactly constitutes a "case" or who qualifies as "unemployed."

Context is key. To interpret numbers accurately, we must understand their context:

• How were they defined and collected?
• What was included or excluded?
• What assumptions were made?

Be wary of sensational statistics. Headlines often use numbers to grab attention, but these figures may be misleading when taken out of context. For instance, a "40% increase" in a rare event might represent a small absolute change. Always ask: "Is this a big number?" and "Compared to what?"

2. Averages often misrepresent reality: Look for the distribution

Averages take the whole mess of human experience, some up, some down, some here, some there, some almost off the graph, and grind the data into a single number.

Averages can hide variety. An average combines diverse data points into a single figure, potentially obscuring important variations. For example, an "average income" might not represent anyone's actual earnings if there's a wide gap between high and low earners.

Consider the distribution. To get a more accurate picture:

• Look for the median (middle value) and mode (most common value)
• Examine the range and spread of data
• Ask about outliers and how they affect the average

Beware of the "average person" fallacy. Policies or products designed for the "average" may not suit anyone in reality. For instance, designing a cockpit for the average pilot's measurements led to poor fit for most actual pilots.

3. Chance plays a bigger role than we think: Beware of false patterns

Chance does not mean, in the ordinary meaning of these words, spread out, or shared, or messy.

Random clustering occurs naturally. Our brains are wired to see patterns, even where none exist. This can lead to false conclusions about causation. For example, cancer clusters in small communities might be due to chance rather than a specific environmental factor.

Understand regression to the mean. Extreme results tend to be followed by more average ones, not because of any intervention but simply due to natural variation. This effect can lead to false attribution of success or failure in various fields:

• Medical treatments
• Educational interventions
• Performance in sports or business

Be cautious with small samples. The smaller the sample, the more likely it is that chance variations will appear significant. Always consider the sample size when evaluating claims based on data.

4. Risk assessment requires personal context: Bring numbers home

Natural frequencies could easily be adopted more widely, but are not, so tempting the conclusion that there is a vested interest both for advocacy groups and journalists in obscurity.

Make risks relatable. Instead of using abstract percentages, express risks in terms of natural frequencies (e.g., "5 in 100 people" rather than "5%"). This makes the information more intuitive and easier to understand.

Consider baseline risks. A "50% increase" in risk means very different things depending on the starting point. Always ask:

• What was the original risk?
• What is the absolute change in risk?
• How does this compare to other risks in my life?

Personalize the numbers. When assessing risks, try to relate them to your own circumstances and experiences. For instance, compare a new risk to familiar ones, like the risk of a car accident.

5. Sampling can skew results: Consider how data is collected

Life as a fire hose, and samplers with tea cups and crooked fingers, is an unequal statistical fight.

Sampling is ubiquitous and necessary. It's often impossible or impractical to count or measure an entire population, so we rely on samples. However, the way samples are chosen can greatly affect the results.

Be aware of sampling biases. Common issues include:

• Selection bias: Some groups are over- or under-represented
• Voluntary response bias: Only those with strong opinions participate
• Survivorship bias: Only successful cases are considered

Question the sample's relevance. Ask:

• How was the sample selected?
• Is it representative of the population of interest?
• Are there important groups or factors not captured by the sample?

6. Targets and comparisons can be misleading: Question the metrics

Numbers, claiming with confidence to have counted something breathtaking, mug our attention on every corner.

Targets can distort behavior. When specific metrics become targets, people often find ways to game the system, potentially undermining the original goal. For example, hospitals might prioritize less urgent cases to meet waiting time targets.

Beware of inappropriate comparisons. League tables and rankings often compare entities that aren't truly comparable due to different contexts or definitions. Consider:

• Are the things being compared truly similar?
• Are the metrics appropriate for all entities being ranked?
• What important factors might be omitted from the comparison?

Look beyond the numbers. Quantitative metrics can't capture everything that matters. Always consider what qualitative aspects might be overlooked in purely numerical assessments.

7. Data quality matters: Scrutinize the source and methodology

The mechanics of counting are anything but mechanical. To understand numbers in life, start with flesh and blood.

Human factors affect data quality. The process of collecting and recording data involves many human decisions and potential errors. Consider:

• Who collected the data and why?
• How motivated were they to be accurate?
• What potential biases or errors might have been introduced?

Examine the methodology. Look for:

• Clear definitions of what was measured
• Transparency about data collection methods
• Acknowledgment of limitations and potential errors

Be skeptical of perfect data. Real-world data is often messy. If results look too clean or precise, it might indicate manipulation or oversimplification.

8. Statistical literacy empowers decision-making: Cultivate numerical skepticism

Numbers have amazing power to put life's anxieties into proportion: Will it be me? What happens if I do? What if I don't?

Develop critical thinking about numbers. Cultivate a healthy skepticism towards statistical claims by always asking questions:

• What's the source of this number?
• What does it actually measure?
• What might it be leaving out?

Use numbers as tools, not truths. Statistics can provide valuable insights, but they should inform decisions, not dictate them. Remember that numbers are approximations of reality, not reality itself.

Empower yourself with knowledge. Understanding basic statistical concepts can help you:

• Make more informed personal and professional decisions
• Critically evaluate media claims and policy proposals
• Communicate more effectively using data

By developing these skills, you can navigate a world increasingly driven by data and make better-informed choices in your personal and professional life.

Last updated: October 3, 2024