Everybody Lies

1. Google searches reveal hidden truths about human behavior

People lie to friends, lovers, doctors, surveys, and themselves. But on Google they might share embarrassing information.

Digital truth serum. Google searches provide a unique window into people's true thoughts, desires, and behaviors. Unlike surveys or social media posts, where people often present an idealized version of themselves, Google searches are typically honest and unfiltered. This "digital truth serum" allows researchers to uncover hidden truths about human nature that were previously difficult or impossible to study.

Surprising insights. Analysis of Google search data has revealed unexpected patterns in human behavior. For example:

• Searches for jokes peak when people are happy, not sad as previously thought
• There are more searches for "Is my son gifted?" than "Is my daughter gifted?", revealing gender biases in parenting
• Searches related to anxiety are higher in rural areas with lower education levels, contrary to the stereotype of the urban neurotic

By examining aggregate search data, researchers can gain insights into sensitive topics like racism, sexuality, and mental health that people are often reluctant to discuss openly.

2. Big Data offers new insights into racism and prejudice

The single most telling fact I have found regarding hate on the internet is the popularity of the word "nigger."

Uncovering hidden prejudice. Traditional surveys often fail to capture the true extent of racism and prejudice in society, as people are reluctant to admit to socially unacceptable views. However, analysis of Google search data reveals a disturbing prevalence of racist attitudes:

• Searches for "nigger jokes" are 17 times more common than searches for other ethnic slurs combined
• Racist searches spike after major events involving African Americans, such as Obama's election
• Areas with high levels of racist searches show lower support for black political candidates

Measuring impact. This data allows researchers to quantify the real-world effects of racism:

• Obama lost an estimated 4 percentage points nationwide due to explicit racism
• Racist Google searches correlate with lower wages for black workers in an area
• Support for Donald Trump in the 2016 Republican primary correlated strongly with racist search rates

By providing hard data on the prevalence and impact of racism, Big Data analysis offers a powerful tool for understanding and addressing prejudice in society.

3. Internet behavior exposes our true sexual desires and preferences

Fully 25 percent of female searches for straight porn emphasize the pain and/or humiliation of the woman.

Hidden desires revealed. Pornography searches and consumption patterns offer unprecedented insights into human sexuality:

• About 5% of male porn searches are for gay content, suggesting higher rates of same-sex attraction than self-reported
• Heterosexual women often search for lesbian porn, even if they don't identify as bisexual
• There is significant interest in taboo topics like incest-themed videos

Challenging assumptions. This data often contradicts conventional wisdom about sexuality:

• Men's top concern is penis size, while women worry most about vaginal odor
• There are more searches complaining about boyfriends not wanting sex than girlfriends
• Interest in violent or non-consensual content is higher among women than men

While this data must be interpreted cautiously, it provides valuable insights into the complexity and diversity of human sexual desires that are often hidden from view in everyday life.

4. Social media data can be misleading compared to search data

Facebook is digital brag-to-my-friends-about-how-good-my-life-is serum.

Curated personas. Social media posts often present an idealized version of people's lives, showcasing only positive experiences and socially desirable traits. This can create a misleading picture of reality:

• People are more likely to share highbrow content (e.g., Atlantic articles) than lowbrow content (e.g., National Enquirer) on social media, even if they consume both equally
• Relationship statuses on Facebook may not reflect the true state of a couple's happiness or struggles

Search data vs. social media. Google searches reveal a very different picture of people's lives:

• While social media posts describe husbands as "the best" and "amazing," common searches include "My husband is a jerk" and "My husband won't have sex"
• People may post about attending cool parties on weekends, but searches reveal many are home alone binge-watching Netflix

To understand human behavior accurately, researchers must look beyond the curated images presented on social media and examine the unfiltered desires and concerns revealed in private searches.

5. Childhood experiences shape lifelong preferences and beliefs

The single most important year in a man's life, for the purposes of cementing his favorite baseball team as an adult, is when he is more or less eight years old.

Critical periods. Big Data analysis reveals that many of our adult preferences and beliefs are strongly influenced by experiences during specific periods of childhood:

• Men typically become lifelong fans of sports teams that were successful when they were 8-12 years old
• Political views are heavily shaped by the popularity of presidents when people are 14-24 years old
• Sexual preferences often relate to experiences or exposures during adolescence

Long-term impact. These childhood imprints can have significant effects throughout life:

• Adults born in areas with successful sports teams during their youth are more likely to be sports fans
• People's political leanings often reflect the dominant views of their formative years, rather than changing with age as previously thought
• Early exposure to diverse ideas and experiences in college towns correlates with higher rates of adult success and innovation

Understanding these critical periods offers insights into how preferences and beliefs form, and may provide opportunities for positive interventions during key developmental stages.

6. A/B testing revolutionizes business and social science research

Facebook now runs a thousand A/B tests per day, which means that a small number of engineers at Facebook start more randomized, controlled experiments in a given day than the entire pharmaceutical industry starts in a year.

Rapid experimentation. The internet has made it possible to conduct large-scale randomized experiments quickly and cheaply:

• Companies like Google and Facebook run thousands of A/B tests daily to optimize their products
• Political campaigns use A/B testing to refine messaging and increase donations
• Researchers can test social science theories on massive online populations

Surprising results. A/B testing often reveals that our intuitions about human behavior are wrong:

• Minor changes in wording or design can have major impacts on user behavior
• Violent movies decrease crime rates by keeping potential criminals occupied
• Seemingly trivial factors (like arrow icons) can significantly increase ad click-through rates

By allowing rapid, data-driven decision-making, A/B testing is transforming how businesses operate and how social scientists conduct research. However, the ethics of constantly experimenting on users without their knowledge raises important questions.

7. Big Data has limitations and ethical concerns that must be addressed

We can't blindly trust government data. The government may tell us that child abuse or abortion has fallen and politicians may celebrate this achievement. But the results we think we're seeing may be an artifact of flaws in the methods of data collection.

Limitations of Big Data. While powerful, Big Data analysis has important limitations:

• The curse of dimensionality: With enough variables, spurious correlations are inevitable
• Overemphasis on measurable factors at the expense of intangible qualities
• Potential for privacy violations and unethical use of personal data

Ethical concerns. The use of Big Data raises significant ethical questions:

• Should companies be allowed to use online behavior to determine creditworthiness or employability?
• How can we protect individual privacy while benefiting from aggregate data analysis?
• What are the implications of constant A/B testing on unsuspecting users?

To harness the full potential of Big Data responsibly, researchers and policymakers must grapple with these limitations and ethical concerns. Combining Big Data insights with traditional research methods and ethical frameworks will be crucial for achieving meaningful progress while protecting individual rights.

Last updated: August 8, 2024