A Field Guide to Lies and Statistics

1. Critical thinking is essential in the age of information overload

We—each of us—need to think critically and carefully about the numbers and words we encounter if we want to be successful at work, at play, and in making the most of our lives.

Information explosion: We are living in an era of unprecedented access to information. However, this abundance comes with a significant challenge: distinguishing truth from falsehood. The internet, social media, and various news outlets bombard us with claims, statistics, and supposed facts daily.

Responsibility of the individual: While we might expect institutions like newspapers, government agencies, or educational platforms to verify information for us, they often fall short. Therefore, the onus falls on each individual to develop and apply critical thinking skills. This involves:

• Questioning the source of information
• Examining the methodology behind statistics
• Looking for potential biases or conflicts of interest
• Seeking corroborating evidence from multiple sources
• Being willing to change our minds when presented with new, credible information

2. Evaluate numbers and statistics with healthy skepticism

Statistics are not facts. They are interpretations. And your interpretation may be just as good as, or better than, that of the person reporting them to you.

Beyond face value: Numbers often appear authoritative and objective, but they can be manipulated or misinterpreted. When encountering statistics, it's crucial to:

• Check for plausibility: Does the claim make logical sense given what you know about the world?
• Understand the context: What was actually measured, and how?
• Look for comparisons: Is the statistic meaningful on its own, or does it need a point of reference?

Common pitfalls:

• Cherry-picking data to support a predetermined conclusion
• Confusing correlation with causation
• Using small or unrepresentative samples to make broad claims
• Failing to account for margins of error or statistical significance

Always ask: "What's the source of this number?" and "What might be missing from this picture?"

3. Understand the power and pitfalls of averages and probabilities

There are three ways of calculating an average, and they often yield different numbers, so people with statistical acumen usually avoid the word average in favor of the more precise terms mean, median, and mode.

Types of averages: Each type of average (mean, median, mode) can tell a different story about a dataset. Understanding which one is being used and why is crucial for accurate interpretation.

• Mean: Sum of all values divided by the number of values
• Median: Middle value when data is arranged in order
• Mode: Most frequently occurring value

Probability misconceptions: Many people struggle with understanding and interpreting probabilities, leading to poor decision-making.

Common errors:

• Gambler's fallacy: Believing past events influence independent future outcomes
• Base rate neglect: Ignoring the overall prevalence of an event when assessing its likelihood
• Confusing conditional probabilities: Misunderstanding the difference between P(A|B) and P(B|A)

Developing a better grasp of probability can improve risk assessment and decision-making in various aspects of life, from medical choices to financial planning.

4. Recognize how graphs and visual representations can mislead

If you want to lie with statistics, start your y-axis somewhere near the lowest value you're plotting—this will emphasize the difference you're trying to highlight, because the eye is drawn to the size of the difference as shown on the graph, and the actual size of the difference is obscured.

Visual manipulation: Graphs and charts are powerful tools for conveying information quickly, but they can also be used to distort reality. Common tactics include:

• Truncated axes: Starting the y-axis above zero to exaggerate differences
• Misleading scales: Using non-linear or inconsistent scales
• Cherry-picked time frames: Selecting data ranges that support a specific narrative
• 3D effects: Adding unnecessary depth that can distort proportions

Critical viewing: When examining visual data representations:

• Always check the axes and scales
• Look for complete context (full range of data)
• Be wary of visually striking but potentially meaningless comparisons
• Consider alternative ways the same data could be presented

Remember, a well-designed graph should clarify information, not obscure it. If a visual seems to be trying too hard to make a point, it's worth scrutinizing more closely.

5. Assess the credibility of sources and expertise

Expertise is typically narrow. An economist in the highest echelons of the government may not have any special insight into what social programs will be effective for curbing crime.

Source hierarchy: Not all information sources are equally reliable. Generally, peer-reviewed academic journals are more trustworthy than books, which are more reliable than self-published websites. However, even reputable sources can make mistakes or have biases.

Evaluating expertise:

• Check credentials: What are the author's qualifications in the specific field?
• Look for peer review: Has the work been scrutinized by other experts?
• Consider conflicts of interest: Does the source have financial or ideological motivations?
• Assess methodology: How was the information gathered and analyzed?

Beware of false authority: Expertise in one area doesn't necessarily translate to others. A Nobel Prize-winning physicist isn't automatically an authority on climate change or economics.

Cross-referencing: Always seek multiple sources, especially for important decisions. Look for consensus among experts in the field, but also be open to well-supported dissenting views.

6. Be aware of common logical fallacies and cognitive biases

Our brains are built to make stories as they take in the vastness of the world with billions of events happening every second. There are apt to be some coincidences that don't really mean anything.

Logical fallacies: These are errors in reasoning that can lead to flawed conclusions. Common examples include:

• Ad hominem: Attacking the person rather than their argument
• False dichotomy: Presenting only two options when more exist
• Appeal to nature: Assuming what's natural is inherently good or right
• Slippery slope: Arguing that one small step will inevitably lead to extreme consequences

Cognitive biases: These are systematic errors in thinking that affect our judgments and decisions. Key biases to watch for:

• Confirmation bias: Seeking information that confirms existing beliefs
• Availability heuristic: Overestimating the likelihood of events we can easily recall
• Dunning-Kruger effect: Overestimating one's own knowledge or ability
• Anchoring: Relying too heavily on the first piece of information encountered

Mitigation strategies:

• Actively seek out information that challenges your views
• Consider alternative explanations for events or data
• Be willing to say "I don't know" or "I'm not sure"
• Practice metacognition: Think about your own thinking processes

7. Apply scientific thinking to evaluate claims and evidence

Science progresses by a combination of deduction and induction. Without induction, we'd have no hypotheses about the world.

Scientific method: While not all claims can be subjected to rigorous scientific testing, applying scientific principles can help evaluate information more objectively:

1. Formulate a clear question or hypothesis
2. Gather relevant data and evidence
3. Analyze the information critically
4. Draw tentative conclusions
5. Be open to revising those conclusions with new evidence

Key scientific principles:

• Falsifiability: A claim should be testable and potentially disprovable
• Replicability: Results should be reproducible by others
• Peer review: Findings should withstand scrutiny from experts
• Occam's Razor: Simpler explanations are generally preferable to complex ones

Applying to everyday life: Even outside formal scientific contexts, these principles can guide critical thinking:

• Seek evidence for claims, not just opinions
• Consider alternative explanations
• Be wary of unfalsifiable or overly complex theories
• Look for consensus among experts, but remain open to new discoveries

8. Embrace uncertainty and update beliefs with new information

Science doesn't present us with certainty, only probabilities. We don't know for 100 percent sure that the sun will come up tomorrow, or that the magnet we pick up will attract steel, or that nothing travels faster than the speed of light. We think these things very likely, but science yields only the best Bayesian conclusions we can have, given what we know so far.

Bayesian thinking: This approach involves updating our beliefs as we encounter new evidence. It acknowledges that our current understanding is always provisional and subject to change.

Key aspects of Bayesian reasoning:

• Start with prior probabilities based on existing knowledge
• Incorporate new evidence to adjust those probabilities
• Recognize that strongly held beliefs require proportionally strong evidence to change

Embracing uncertainty:

• Acknowledge the limits of our knowledge
• Be comfortable saying "I don't know" or "I'm not sure"
• Distinguish between different levels of certainty
• Remain open to changing your mind with new evidence

Practical application:

• Regularly reassess your beliefs and opinions
• Seek out information that challenges your views
• Be willing to admit mistakes and update your understanding
• Cultivate intellectual humility

Remember, the goal is not to achieve perfect certainty, but to make the best decisions possible with the information available, while remaining open to new discoveries and insights.

Last updated: August 17, 2024