The Seven Deadly Sins of Psychology

1. Bias: The Unseen Force Shaping Psychological Science

The human understanding when it has once adopted an opinion … draws all things else to support and agree with it.

Confirmation bias is pervasive. Psychology, the very field that studies bias, is ironically highly susceptible to it. Researchers, like all humans, tend to seek out and favor evidence that confirms their existing beliefs, while ignoring or devaluing contradictory evidence. This bias manifests in various forms, from the preference for positive results to the dismissal of replication attempts.

Neophilia and positivity. The academic culture in psychology often prioritizes novelty and positive results, leading to a publication bias where studies that fail to show statistically significant effects are less likely to be published. This creates a distorted view of the scientific landscape, where negative or null findings are often hidden from view. This bias is further exacerbated by the tendency to favor conceptual replications over direct replications, allowing researchers to confirm previous findings without ever truly testing their robustness.

Hindsight bias and HARKing. Hindsight bias, the tendency to believe that an observation was expected even when it was a surprise, further distorts the scientific record. This leads to Hypothesizing After Results are Known (HARKing), where researchers alter their predictions after analyzing data to fit unexpected outcomes, presenting them as if they were a priori. This practice undermines the hypothetico-deductive model of science and reinforces confirmation bias.

2. Hidden Flexibility: The Silent Saboteur of Research

Torture numbers and they will confess to anything.

Researcher degrees of freedom. Psychologists often have numerous defensible options when analyzing data, such as excluding outliers, choosing which variables to include, or deciding when to stop data collection. This "hidden flexibility" allows researchers to manipulate their analyses to achieve statistically significant results, a practice known as p-hacking.

P-hacking and its consequences. P-hacking inflates the rate of false positives, leading to a scientific literature filled with unreliable findings. The practice of peeking at data and adding participants until statistical significance is reached further undermines the validity of null hypothesis significance testing (NHST). This is because NHST requires a prespecified stopping rule, which is often violated in practice.

Unconscious analytic tuning. Even when researchers believe they are acting honestly, subtle forms of p-hacking and HARKing can occur unconsciously. Vague hypotheses, post hoc explanations of unexpected results, and biased debugging can all contribute to the publication of false discoveries. This highlights the insidious nature of hidden flexibility and the need for more transparent research practices.

3. Unreliability: The Fragile Foundation of Psychological Findings

And it’s this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science.

Replication is essential. Direct replication, the process of repeating a previous experiment as exactly as possible, is the immune system of science. It is crucial for identifying false discoveries and ensuring the reliability of research findings. However, psychology has largely abandoned direct replication in favor of conceptual replication, which tests related ideas using different methods.

Low statistical power. Many studies in psychology suffer from low statistical power, meaning they have a low probability of detecting a true effect. This increases the rate of false negatives (missing true discoveries) and also increases the rate of false positives. This is because low-powered studies require larger effect sizes to achieve statistical significance, which can lead to an overestimation of the true effect.

Methodological shortcomings. A lack of methodological disclosure, statistical fallacies, and the failure to retract irreproducible findings further contribute to the unreliability of psychological research. Many studies fail to provide sufficient detail to allow for replication, and researchers often misunderstand the meaning of p-values, leading to erroneous conclusions. The failure to retract flawed studies from the literature further perpetuates the problem.

4. Data Hoarding: The Barrier to Scientific Progress

Code and data or it didn’t happen.

Data sharing is vital. Data sharing is essential for transparency, reproducibility, and the detection of fraud. It allows independent scientists to verify analyses, repeat experiments, and conduct new analyses that the original authors may not have considered. It also prevents the permanent loss of valuable information.

Culture of secrecy. Despite the clear benefits of data sharing, it is not the norm in psychology. Many researchers treat their data as personal property, withholding it from others unless motivated by self-interest. This culture of secrecy hinders scientific progress and prevents the detection of questionable research practices.

Consequences of data hoarding. The lack of data transparency makes it difficult to detect p-hacking, HARKing, and other forms of research misconduct. It also prevents the detection of data fabrication, as demonstrated by the cases of Stapel and Smeesters. By making data sharing the norm, we can create a more open and trustworthy scientific community.

5. Corruptibility: The Dark Side of Academic Ambition

I was doing fine, but then I became impatient, overambitious, reckless.

Fraud is a spectrum. Academic fraud is not a black-and-white issue. It exists on a continuum, ranging from questionable research practices to outright data fabrication. The pressure to publish positive results in prestigious journals can lead researchers to engage in increasingly dishonest behaviors.

The anatomy of fraud. Cases like Diederik Stapel highlight the dangers of unchecked ambition and the ease with which data can be manipulated or fabricated. Stapel's story reveals how a desire for recognition and success can lead to a complete abandonment of scientific integrity. The case of Smeesters shows how even the selective reporting of results can be considered misconduct.

Junior scientists at risk. Junior scientists are particularly vulnerable to the temptation of fraud, as they face intense pressure to publish and secure their careers. The cases of Braet and Savine demonstrate how the desire for success can lead to the falsification of data. Whistle-blowers, who expose fraud, often face retaliation and career damage, highlighting the need for better protection and support.

6. Internment: The Paywall That Silences Science

Publish means “make public.”

Barrier-based publishing. The traditional system of academic publishing, where articles are hidden behind paywalls, restricts access to scientific knowledge. This system prevents the public, policymakers, and even some researchers from accessing the results of publicly funded research.

Open access publishing. Open access (OA) publishing provides a solution by making research freely available to all. Full OA journals charge authors a fee to publish, while hybrid OA journals offer a mix of subscription-based and open access articles. However, many psychologists still support barrier-based publishing, prioritizing prestige over accessibility.

The need for change. The current system of barrier-based publishing is unsustainable and unethical. It hinders the dissemination of knowledge and perpetuates a system where publishers profit from publicly funded research. A shift towards open access is essential for democratizing science and making it accessible to all.

7. Bean Counting: The Misguided Quest for Quantifiable Quality

Not everything that counts can be counted, and not everything that can be counted counts.

The tyranny of metrics. Psychology, like many other sciences, has become obsessed with quantifiable metrics, such as journal impact factors (JIFs), grant income, and authorship order. These metrics are often used as proxies for quality, but they are easily gamed and fail to capture the true value of scientific contributions.

The flaws of JIF. The JIF is a flawed metric that is easily manipulated and does not accurately reflect the quality of individual articles. It is based on the average number of citations to a journal, not the actual citations of individual articles. It also fails to account for the skewness of citation distributions, where a minority of articles attract the majority of citations.

Beyond bean counting. The focus on metrics has led to a culture where quantity is valued over quality, and where the pursuit of grants and publications has become an end in itself. To improve the quality of science, we must move beyond bean counting and focus on the true goals of research: the advancement of knowledge and the betterment of society.

8. Registered Reports: A Path to Transparency and Reproducibility

Registered Reports: A Vaccine against Bias

Preregistration is key. Registered Reports offer a solution to many of the problems plaguing psychological science. By requiring authors to prespecify their hypotheses, methods, and analysis plans before data collection, Registered Reports prevent p-hacking, HARKing, and publication bias.

Two-stage review process. Registered Reports involve a two-stage review process. At Stage 1, the study protocol is reviewed and, if deemed scientifically sound, is given in-principle acceptance (IPA). At Stage 2, the completed study is reviewed to ensure that the authors adhered to their preregistered protocol.

Focus on rigor, not results. Registered Reports prioritize methodological rigor and theoretical importance over the novelty or statistical significance of results. This approach ensures that all well-designed studies, regardless of their outcomes, are published, reducing publication bias and promoting transparency.

9. Open Science: A Call for Collaboration and Transparency

The method of science, as stodgy and grumpy as it may seem, is far more important than the findings of science.

Open science principles. Open science promotes transparency, collaboration, and accessibility in research. It involves sharing data, code, and materials, as well as making research publications freely available. Open science practices are essential for improving the reliability and credibility of psychological science.

Grassroots and top-down reforms. The open science movement is driven by both grassroots initiatives, such as the Peer Reviewers’ Openness (PRO) initiative, and top-down reforms, such as the Transparency and Openness Promotion (TOP) guidelines. These initiatives aim to change the culture of science by incentivizing transparency and reproducibility.

A new era of collaboration. Open science fosters a more collaborative and inclusive research environment. By sharing data and methods, researchers can build on each other's work, accelerate discovery, and create a more robust and reliable body of knowledge.

10. A New Path: Rebuilding Psychology on Solid Ground

One should be able to see that things are hopeless and yet be determined to make them otherwise.

Embrace imperfection. We must accept that bias is an inherent part of human nature and that we can never completely eliminate it. Instead, we should focus on implementing measures that protect the outcome of science from our inherent flaws.

Prioritize transparency and reproducibility. To rebuild psychology on solid ground, we must prioritize transparency and reproducibility over novelty and positive results. This requires a fundamental shift in the way we conduct, evaluate, and publish research.

A call to action. The future of psychological science depends on our willingness to embrace change and challenge the status quo. By adopting open science practices, promoting direct replication, and prioritizing methodological rigor, we can create a more robust, reliable, and trustworthy scientific discipline.

Last updated: February 2, 2025