Searching...
English
EnglishEnglish
EspañolSpanish
简体中文Chinese
FrançaisFrench
DeutschGerman
日本語Japanese
PortuguêsPortuguese
ItalianoItalian
한국어Korean
РусскийRussian
NederlandsDutch
العربيةArabic
PolskiPolish
हिन्दीHindi
Tiếng ViệtVietnamese
SvenskaSwedish
ΕλληνικάGreek
TürkçeTurkish
ไทยThai
ČeštinaCzech
RomânăRomanian
MagyarHungarian
УкраїнськаUkrainian
Bahasa IndonesiaIndonesian
DanskDanish
SuomiFinnish
БългарскиBulgarian
עבריתHebrew
NorskNorwegian
HrvatskiCroatian
CatalàCatalan
SlovenčinaSlovak
LietuviųLithuanian
SlovenščinaSlovenian
СрпскиSerbian
EestiEstonian
LatviešuLatvian
فارسیPersian
മലയാളംMalayalam
தமிழ்Tamil
اردوUrdu
How to Read Numbers

How to Read Numbers

A Guide to Stats in the News
by Tom Chivers 2021 208 pages
4.34
100+ ratings
Listen
Listen to Summary

Key Takeaways

1. Numbers can mislead: Context is crucial for interpretation

Numbers do not feel. Do not bleed or weep or hope. They do not know bravery or sacrifice. Love and allegiance. At the very apex of callousness, you will find only ones and zeros.

Numbers lack context. Without proper context, numbers can be misleading or meaningless. For example, reporting that 361 cyclists were killed on London roads between 1993 and 2017 sounds alarming. However, this number becomes less significant when you consider that during this period, there were approximately 437,000 daily cycling journeys in London. This context reveals that the risk of death per journey is actually very low.

Denominators matter. To properly interpret numbers, it's crucial to understand the denominator - the total population or base number from which a statistic is derived. For instance, if someone claims that 300 people are murdered by undocumented immigrants in the USA every year, it's essential to know:

  • The total number of murders in the USA
  • The total population of undocumented immigrants
  • The murder rate for the general population

Without this context, it's impossible to determine whether this number is high or low relative to expectations.

2. Sample size matters: Larger samples yield more reliable results

Assuming that the null hypothesis is true and the study is repeated an infinite number times by drawing random samples from the same population(s), less than 5% of these results will be more extreme than the current result.

Bigger is better. Larger sample sizes generally provide more reliable and representative results. Small samples are more susceptible to random variations and outliers, which can lead to misleading conclusions. For example, a study claiming that swearing makes people stronger based on only 29 participants should be viewed with skepticism.

Statistical power. The ability to detect a true effect increases with sample size. This is particularly important when studying subtle effects. As a rule of thumb, be wary of studies with fewer than 100 participants, especially if they make surprising claims. However, keep in mind that:

  • Some smaller studies can be robust if well-designed
  • Even large studies can be flawed if poorly conducted or biased
  • The appropriate sample size depends on the size of the effect being studied and the variability in the population

3. Correlation is not causation: Beware of confounding variables

Let's imagine that you're conducting a study looking at how fast people can run. You notice something: on average, the more grey hairs a person has, the slower their time for the mile.

Hidden factors. Correlation between two variables doesn't necessarily mean one causes the other. There may be hidden factors (confounding variables) influencing both. For example, the correlation between grey hair and slower running speed is likely due to age affecting both variables, rather than grey hair directly causing slower running.

Types of relationships:

  • Causal: A directly causes B
  • Reverse causality: B actually causes A
  • Common cause: C causes both A and B
  • Coincidence: No real relationship, just random chance

To establish causation, consider:

  • Temporal sequence: The cause must precede the effect
  • Strength of association: Stronger correlations are more suggestive of causation
  • Dose-response relationship: Changes in the cause lead to proportional changes in the effect
  • Consistency: The relationship is observed across different studies and populations
  • Plausibility: There's a logical mechanism for the causal relationship

Randomized controlled trials (RCTs) are the gold standard for establishing causation, but they're not always possible or ethical.

4. Statistical significance doesn't equal practical importance

Statistical significance is confusing, even for scientists. A 2002 study found that 100 per cent of psychology undergraduates misunderstand significance – as, even more shockingly, did 90 per cent of their lecturers.

P-value limitations. Statistical significance (typically p < 0.05) only tells us that a result is unlikely to have occurred by chance. It doesn't indicate the size or practical importance of an effect. A tiny, inconsequential difference can be statistically significant with a large enough sample size.

Effect size matters. To understand the practical importance of a finding, we need to consider the effect size:

  • How large is the difference between groups?
  • What's the magnitude of the correlation?
  • Is the effect meaningful in real-world terms?

For example, a study might find a statistically significant link between reading on screens before bed and reduced sleep. However, if the actual effect is only 10 minutes of lost sleep after 4 hours of reading, the practical importance may be minimal for most people.

5. Absolute vs. relative risk: Understand the true impact

If you tell me that eating burnt toast will raise my risk of a hernia by 50 per cent, that sounds worrying. But unless you tell me how common hernias are, it's meaningless.

Relative risk can mislead. Reporting only relative risk changes can exaggerate the importance of findings. A 50% increase in a very rare event is still a very rare event. Always look for the absolute risk to understand the true impact.

Interpreting risk:

  • Baseline risk: How common is the outcome initially?
  • Relative risk change: The percentage increase or decrease in risk
  • Absolute risk change: The actual difference in risk

Example:

  • Headline: "Eating bacon daily increases bowel cancer risk by 20%"
  • Baseline risk: 6% lifetime risk for women
  • Relative risk increase: 20%
  • Absolute risk change: 6% to 7.2% (1.2 percentage point increase)
  • Interpretation: The risk goes from about 1 in 17 to 1 in 14 - a real increase, but perhaps less dramatic than the headline suggests

6. Survivorship bias: Don't overlook hidden failures

The Navy was looking at a particular subset of planes – those planes which had returned to the carrier. The planes which had been hit a lot on the fuselage and wings tended to have made it back to base successfully. Those that had been hit on the engines, meanwhile, had predominantly fallen into the sea and not been counted in the statistics.

Hidden failures. Survivorship bias occurs when we focus only on successful examples, overlooking those that failed. This can lead to false conclusions about what contributes to success. For example, studying only successful businesses to determine factors for success ignores all the failed businesses that may have had similar characteristics.

Examples of survivorship bias:

  • Self-help books by successful entrepreneurs
  • Investment strategies based on past performance
  • Medical studies that don't account for patients who dropped out
  • Historical artifacts that survived due to durability, not representativeness

To avoid survivorship bias:

  • Look for the "silent evidence" - what's missing from the data?
  • Consider the full population, not just successful examples
  • Be skeptical of success stories and "secrets to success"
  • Look for studies that account for attrition and non-responders

7. Forecasting limitations: The future is inherently uncertain

As a reader, you need to be aware of how forecasts are made, and you need to know that they are not mystical insights into fate – but nor are they random guesses. They're the outputs of statistical models, which can be more or less accurate; and the very precise numbers (1.2 per cent, 50,000 deaths, whatever) are central estimates inside a much bigger range of uncertainty.

Models have limits. Forecasts are based on models, which are simplified representations of reality. While they can be useful, they're inherently uncertain and based on assumptions that may not hold true. Economic forecasts, election predictions, and climate models all come with significant uncertainty.

Key points about forecasts:

  • They're based on past data and current assumptions
  • Unexpected events can dramatically alter outcomes
  • Longer-term forecasts are generally less reliable
  • Point estimates (single numbers) can give a false sense of precision
  • Always look for confidence intervals or ranges of possible outcomes
  • Consider multiple models and scenarios for a more complete picture

When reporting or interpreting forecasts, it's crucial to communicate the underlying uncertainty and the range of possible outcomes, not just a single point estimate.

8. Goodhart's Law: When a measure becomes a target, it ceases to be a good measure

There's an old saying in economics, Goodhart's law, named for Charles Goodhart, a former economic adviser to the Bank of England: 'When a measure becomes a target, it ceases to be a good measure.'

Perverse incentives. When a metric is used as a target, people will find ways to optimize for that metric, often at the expense of the underlying goal. This can lead to unintended consequences and distorted behavior.

Examples of Goodhart's Law in action:

  • Educational targets leading to teaching to the test rather than fostering genuine learning
  • Healthcare metrics causing hospitals to refuse high-risk patients to maintain good statistics
  • Business KPIs resulting in short-term thinking at the expense of long-term value
  • Scientific publishing incentives leading to p-hacking and publication bias

To mitigate Goodhart's Law:

  • Use multiple, diverse metrics to assess performance
  • Regularly review and update metrics to prevent gaming
  • Focus on the underlying goals, not just the numbers
  • Be aware of potential unintended consequences
  • Use qualitative assessments alongside quantitative metrics

9. Publication bias: Negative results often go unreported

There's a clever way of checking whether there is publication bias in a field, known as a funnel plot. A funnel plot plots the results of all the studies on a topic, with smaller, weaker studies towards the bottom of the chart and larger, better studies towards the top.

Missing evidence. Publication bias occurs when studies with positive or novel results are more likely to be published than those with negative or null results. This can lead to a skewed understanding of the evidence, potentially overestimating the effectiveness of treatments or the strength of relationships.

Consequences of publication bias:

  • Overestimation of effect sizes in meta-analyses
  • Waste of resources on research that has already been done but not published
  • Potential harm to patients if ineffective treatments appear effective
  • Difficulty in assessing the true state of knowledge in a field

Methods to detect and mitigate publication bias:

  • Funnel plots: Visual tool to detect asymmetry in published results
  • Pre-registration of studies: Committing to publish results regardless of outcome
  • Registered Reports: Journals agreeing to publish based on methodology, not results
  • Databases of unpublished studies
  • Encouragement of publishing null results

As a reader or researcher, always consider the possibility of unpublished negative results when evaluating the evidence on a topic.

10. Critical thinking: Develop statistical literacy to navigate a data-driven world

This book can be read as just such a style guide: a sort of AP Style Book for statistical good practice. We hope that media outlets start to follow it, or (equally validly) see the need for one and then write their own. This is not just a book, in fact, but the start of a campaign for statistical literacy and responsibility in the media.

Statistical literacy. In a world increasingly driven by data and statistics, it's crucial to develop critical thinking skills and statistical literacy. This allows us to better understand and evaluate the information we encounter in the media, research, and everyday life.

Key skills for statistical literacy:

  • Understanding basic statistical concepts (e.g., averages, variability, probability)
  • Recognizing common statistical pitfalls and biases
  • Evaluating the quality of data sources and methodologies
  • Interpreting graphs and visualizations accurately
  • Asking critical questions about claims based on statistics

Actions to promote statistical literacy:

  • Encourage media outlets to adopt better practices for reporting statistics
  • Support education initiatives that teach statistical thinking
  • Be skeptical of sensational headlines based on numbers
  • Look for original sources and context when encountering statistics
  • Engage in discussions about the proper use and interpretation of data

By developing these skills, we can become more informed citizens, make better decisions, and contribute to a more statistically literate society.

Last updated:

FAQ

What's "How to Read Numbers" about?

  • Understanding statistics in media: "How to Read Numbers" by Tom Chivers and David Chivers is a guide to understanding how statistics are used and often misused in the media.
  • Focus on critical thinking: The book emphasizes the importance of critical thinking when interpreting numbers and statistics presented in news stories.
  • Real-world examples: It uses real-world examples to illustrate how numbers can be misleading and how to spot these issues.
  • Educational approach: The book aims to educate readers on the basics of statistical literacy, making complex concepts accessible to a general audience.

Why should I read "How to Read Numbers"?

  • Improve media literacy: Reading this book will enhance your ability to critically evaluate statistical claims in the media.
  • Avoid being misled: It provides tools to help you avoid being misled by exaggerated or cherry-picked statistics.
  • Practical knowledge: The book offers practical knowledge that can be applied to everyday news consumption and decision-making.
  • Engaging and informative: Written in an engaging style, it makes learning about statistics interesting and relevant.

What are the key takeaways of "How to Read Numbers"?

  • Numbers can mislead: Statistics can be used to mislead, and it's crucial to understand the context and methodology behind them.
  • Importance of sample size: Small sample sizes can lead to unreliable results, and understanding this is key to interpreting studies.
  • Absolute vs. relative risk: Knowing the difference between absolute and relative risk is essential for understanding the true impact of statistics.
  • Demand for novelty: The book highlights how the demand for novel findings can lead to publication bias and misleading headlines.

What are the best quotes from "How to Read Numbers" and what do they mean?

  • "Numbers do not feel. Do not bleed or weep or hope." This quote emphasizes the cold, objective nature of numbers, contrasting them with human emotions and experiences.
  • "While it is easy to lie with statistics, it is even easier to lie without them." This highlights the idea that statistics, when used correctly, can provide clarity and truth, but they can also be manipulated.
  • "The trouble is that, in Simpson’s-paradox situations, you can use the same data to tell diametrically opposed stories." This quote illustrates how data can be interpreted in multiple ways, leading to different conclusions.
  • "When a measure becomes a target, it ceases to be a good measure." Known as Goodhart's Law, this quote warns about the pitfalls of using metrics as targets, which can lead to manipulation and loss of true value.

How can numbers mislead according to "How to Read Numbers"?

  • Cherry-picking data: Selecting specific data points to support a narrative while ignoring others can create a misleading picture.
  • Misleading averages: Using mean averages without considering the distribution can distort the reality of the data.
  • Confounding variables: Failing to account for confounding variables can lead to incorrect conclusions about causation.
  • Statistical significance: Misunderstanding or misrepresenting statistical significance can lead to overestimating the importance of findings.

What is the importance of sample size in "How to Read Numbers"?

  • Reliability of results: Larger sample sizes generally lead to more reliable and accurate results, reducing the impact of random variation.
  • Small sample pitfalls: Small samples can produce misleading results due to greater susceptibility to outliers and random chance.
  • Statistical power: The book explains that the power of a study to detect an effect depends on the sample size, especially for subtle effects.
  • Practical advice: Readers are advised to be cautious of studies with small sample sizes, especially when they make surprising claims.

How does "How to Read Numbers" explain absolute vs. relative risk?

  • Absolute risk: This refers to the actual likelihood of an event occurring, providing a clearer picture of risk.
  • Relative risk: This is the ratio of the probability of an event occurring in an exposed group versus a non-exposed group, often used to exaggerate findings.
  • Importance of context: Understanding both absolute and relative risk is crucial for interpreting the true significance of statistical claims.
  • Practical examples: The book uses examples like health risks to illustrate how relative risk can be misleading without absolute risk context.

What is the "demand for novelty" in scientific research according to "How to Read Numbers"?

  • Publication bias: The demand for novel findings can lead to publication bias, where only exciting results are published.
  • Impact on science: This bias can skew the scientific literature, making it seem like certain effects are more significant than they are.
  • Media influence: The media's focus on novel findings can further distort public understanding of scientific research.
  • Call for change: The authors advocate for more balanced reporting and consideration of the broader scientific context.

How does "How to Read Numbers" address the issue of cherry-picking data?

  • Selective reporting: Cherry-picking involves selecting data that supports a specific narrative while ignoring contradictory data.
  • Misleading trends: This practice can create the illusion of trends or effects that don't exist in the broader dataset.
  • Critical evaluation: The book encourages readers to critically evaluate the data presented and consider what might be left out.
  • Real-world examples: It provides examples from media and research to illustrate how cherry-picking can mislead.

What is Goodhart's Law as explained in "How to Read Numbers"?

  • Definition: Goodhart's Law states that when a measure becomes a target, it ceases to be a good measure.
  • Implications: This can lead to gaming the system, where people manipulate metrics to meet targets rather than achieve genuine improvements.
  • Examples: The book discusses examples in education, healthcare, and business where metrics have been manipulated.
  • Understanding limitations: Recognizing the limitations of metrics is crucial for making informed decisions based on data.

How does "How to Read Numbers" explain the Texas Sharpshooter Fallacy?

  • Definition: The Texas Sharpshooter Fallacy involves focusing on random patterns and assigning them significance after the fact.
  • Misleading conclusions: This can lead to false conclusions about causation or correlation based on random data clusters.
  • Real-world application: The book uses examples like cancer clusters and economic predictions to illustrate this fallacy.
  • Critical thinking: It emphasizes the importance of critical thinking and skepticism when interpreting patterns in data.

What is the role of assumptions in models according to "How to Read Numbers"?

  • Model accuracy: Assumptions are crucial for the accuracy and reliability of models, influencing their predictions.
  • Complexity vs. simplicity: The book discusses the trade-off between model complexity and simplicity, and how assumptions impact this balance.
  • Real-world examples: It uses examples like economic and disease models to show how assumptions can lead to different outcomes.
  • Understanding assumptions: Readers are encouraged to understand the assumptions behind models to better interpret their results.

Review Summary

4.34 out of 5
Average of 100+ ratings from Goodreads and Amazon.

How to Read Numbers is highly praised for its accessible explanation of statistics and data interpretation. Readers appreciate its clarity, real-world examples, and practical advice for critically examining numbers in media. The book covers common statistical pitfalls, biases, and misrepresentations, empowering readers to better understand and question data-driven claims. Many reviewers recommend it as essential reading for journalists and the general public alike, noting its potential to improve statistical literacy and critical thinking skills.

Your rating:

About the Author

Tom Chivers is a British science writer and journalist known for his ability to explain complex scientific concepts in an accessible manner. He has written for various publications, including The Telegraph and BuzzFeed UK. Chivers has a background in philosophy and is particularly interested in statistics, rationality, and the intersection of science and society. His work often focuses on debunking misconceptions and promoting critical thinking. In addition to "How to Read Numbers," Chivers has authored other books on scientific topics, demonstrating his commitment to improving public understanding of science and data.

Download PDF

To save this How to Read Numbers summary for later, download the free PDF. You can print it out, or read offline at your convenience.
Download PDF
File size: 0.23 MB     Pages: 16

Download EPUB

To read this How to Read Numbers summary on your e-reader device or app, download the free EPUB. The .epub digital book format is ideal for reading ebooks on phones, tablets, and e-readers.
Download EPUB
File size: 2.93 MB     Pages: 13
0:00
-0:00
1x
Dan
Andrew
Michelle
Lauren
Select Speed
1.0×
+
200 words per minute
Home
Library
Get App
Create a free account to unlock:
Requests: Request new book summaries
Bookmarks: Save your favorite books
History: Revisit books later
Recommendations: Get personalized suggestions
Ratings: Rate books & see your ratings
Try Full Access for 7 Days
Listen, bookmark, and more
Compare Features Free Pro
📖 Read Summaries
All summaries are free to read in 40 languages
🎧 Listen to Summaries
Listen to unlimited summaries in 40 languages
❤️ Unlimited Bookmarks
Free users are limited to 10
📜 Unlimited History
Free users are limited to 10
Risk-Free Timeline
Today: Get Instant Access
Listen to full summaries of 73,530 books. That's 12,000+ hours of audio!
Day 4: Trial Reminder
We'll send you a notification that your trial is ending soon.
Day 7: Your subscription begins
You'll be charged on May 3,
cancel anytime before.
Consume 2.8x More Books
2.8x more books Listening Reading
Our users love us
100,000+ readers
"...I can 10x the number of books I can read..."
"...exceptionally accurate, engaging, and beautifully presented..."
"...better than any amazon review when I'm making a book-buying decision..."
Save 62%
Yearly
$119.88 $44.99/year
$3.75/mo
Monthly
$9.99/mo
Try Free & Unlock
7 days free, then $44.99/year. Cancel anytime.
Scanner
Find a barcode to scan

Settings
General
Widget
Appearance
Loading...
Black Friday Sale 🎉
$20 off Lifetime Access
$79.99 $59.99
Upgrade Now →