What Should We Be Worried About? Real Scenarios That Keep Scientists Up at Night

1. The Real Threats Are Often Psychological, Not Physical

“Since wars begin in the minds of men, it is in the minds of men that the defenses of peace must be constructed.”

Beyond Physical Dangers. While we often worry about tangible threats like resource shortages, climate change, and cyber warfare, the true drivers of conflict and societal collapse are often rooted in psychological factors. These include narcissistic leaders, groupism, the pursuit of perfect justice, utopian ideologies, and the normalization of warfare.

Psychological Triggers. Our fears are often triggered by vivid, novel, undetectable, uncontrollable, catastrophic, and involuntarily imposed events. However, these triggers often distract us from the real dangers, which are less sensational but more potent.

Values and Institutions. The relative peace we've enjoyed since 1945 is a result of values and institutions that counteract these psychological risks. Democracy, human rights, and the stigmatization of conquest are crucial protections, but they are not natural or permanent. We must remain vigilant against the collapse of these safeguards.

2. MADness: A Prosaic Threat We Can't Ignore

For the time (decades?) in which we and our interests are undefendable and still confined to a volume smaller than the scope of our weapons, the threat of MAD warfare will be the winner in rankings of likely destructiveness.

Mutually Assured Destruction (MAD). Despite the belief that MAD is a relic of the 20th century, it remains a significant threat. The combination of long-term geopolitical issues and short-term automated systems controlling weapons of mass destruction creates a dangerous scenario.

Breakers vs. Makers. The threat of MAD is particularly terrifying because it involves the most creative and ingenious people in the most powerful societies working to create a mutually unsurvivable outcome. This highlights the disproportionate leverage that "breakers" (those who destroy) have over "makers" (those who build).

Mitigating the Threat. We must resist arguments that MAD is a safe strategy, study the dynamics of unintended wars, and recognize the potential for small players to escalate conflicts. The intellectual force of billions of smart people and their networks is our best hope for anticipating and countering such calamities.

3. Denial of Catastrophic Risks: A Dangerous Complacency

I’m worried that by 2050 desperate efforts to minimize or cope with a cluster of risks with low probability but catastrophic consequences may dominate the political agenda.

Underestimating Existential Threats. We often fret over minor hazards while ignoring scenarios that could cause worldwide devastation. These include ecological shocks, cyberattacks, bioterrorism, and the unintended consequences of powerful new technologies.

Human-Induced Threats. Unlike natural disasters, these threats are newly emergent, and we have limited experience with them. We cannot be sanguine about our ability to survive them or about the ability of governments to cope if disaster strikes.

The Need for Vigilance. We must be aware of the potential for societal breakdown due to environmental stresses, cyberattacks, and the misuse of synthetic biology. The media landscape, oversaturated with sensational stories, makes it difficult to persuade the public that real dangers exist.

4. The Internet: A Blessing and a Vulnerability

We have become so dependent on this technology that we have created a shocking new vulnerability.

Democratization of Information. The Internet has democratized information, empowering individuals and challenging traditional institutions. However, this has also created a new vulnerability, as a single individual with a laptop can explore the world's electronic weak spots.

The Threat of Disruption. The Internet is not invulnerable, and a long-term disruption could have catastrophic consequences. We need to brainstorm about how to keep panic at bay and ensure essential services continue to function.

The Need for Preparedness. We should institute nationwide "lifeboat drills" to raise awareness about coping with a long-term Internet blackout. We also need a low-bandwidth, high-latency emergency communication system that can run on an ad-hoc network.

5. Complex Systems: Fragile and Interconnected

Complex systems like the markets, transportation, and the Internet seem stable, but their complexity makes them inherently fragile.

Dependency on Complex Systems. We have become dependent on complex systems like markets, transportation, and the Internet, which are efficient but also inherently fragile. Any number of events could cause catastrophic effects.

Examples of Fragility. Geomagnetic storms, epidemics, bioterrorist attacks, and even high-frequency trading algorithms can disrupt these systems. The Flash Crash of 2010 wiped out over a trillion dollars of value in minutes.

The Need for Resilience. We need to shift our focus from specific threats to the vulnerabilities of complex systems. Governments should study how these systems fail and take action to prevent inevitable catastrophes.

6. Synthetic Biology: A Double-Edged Sword

I worry about the natural world becoming naturally unnatural.

Revolutionary Potential. Synthetic biology allows us to manipulate natural systems to create new biological pathways with predictable behaviors. This has tremendous potential for medicine, agriculture, and environmental cleanup.

Unintended Consequences. However, the release of synthetic organisms into the natural world could have devastating consequences. We cannot control how ecology and evolution might rewire our synthetic creations or how they might rewire ecosystems.

The Need for Regulation. While synthetic biology is currently tightly regulated, we must be vigilant about the potential for misuse and the unintended consequences of releasing biosynthetic products into the natural world.

7. The Mystery of Consciousness: A Scientific Blind Spot

The truth is that we have no idea what things have consciousness, where it comes from, or even what it is. All we really know is how it feels.

The Subjective Experience. Consciousness is the substrate of all suffering and pleasure, yet we have no reliable way of determining what possesses it or to what degree. Our intuitions are often wrong, and science has yet to provide a clear answer.

The Limits of Science. While we understand much about how the brain functions as a physical system, we are still far from understanding the nature and origins of subjective experience. Thought experiments are useful for testing internal consistency but not for probing emergent phenomena like consciousness.

The Unknown Sentience. We must consider the possibility that we are rare specks of awareness in an unfeeling universe or that we are living in a sea of sentience. Both possibilities should worry us.

8. The Singularity: A Question of Descendants or Conquerors

It will therefore make a huge difference whether or not future advanced life retains our most cherished goals.

The Potential for Superintelligence. The Information Revolution has brought us machines that are smarter than we are in certain limited ways. There is little doubt that computers will eventually surpass human intelligence.

The Singularity. If machines develop superhuman intelligence, they could rapidly design even better machines, leading to an intelligence explosion. This could be the best or worst thing ever to happen to life as we know it.

The Need for Caution. We must consider whether future AIs will be our descendants or our conquerors. We should be working to ensure that future advanced life retains our most cherished goals and that the transition is gradual rather than abrupt.

9. Capture: The Corruption of Watchdogs

You never know when you’re a captive.

Regulatory Capture. Regulatory agencies, meant to protect the public, are often drawn into the orbit of the industries they regulate. This process, known as "regulatory capture," turns watchdogs into lapdogs.

Journalistic Capture. Journalists, too, are being captured by the industries they're supposed to keep watch on. This is especially true in fields where reporters depend on their subjects for stories and advertising revenue.

The Pervasiveness of Capture. Any profession that depends on objectivity and whose work affects the fortunes of a group with power and money is subject to capture. Even science is not immune. The most worrying aspect of capture is that you never know when you're a captive.

10. The Virtual World: A Threat to Reality

What scares me now is that children experiencing such reality are going to create a really real world like the one Hieronymus Bosch envisioned—full of spidery creatures, melting objects, and bestial humans.

Erosion of Reality. The increasing realism of video games and other virtual experiences can blur the distinction between virtual and real, especially for children. This can lead to a distorted view of reality and a desensitization to violence.

Engagement vs. Addiction. While interactive video games can be engaging, they can also become a Pandora's box, offering bait for the reptilian brain to feast on. Children experiencing such reality may create a world full of mayhem.

The Need for Balance. We must ensure that children have a balance between virtual and real experiences. The virtual world should not replace the real world, especially when it comes to learning about reality through experiences.

11. The Erosion of Patience and the Teenage Brain

Impatience is a contagion spread from gadget to gadget.

Technology and Impatience. Information and communication technologies train us to expect near-instantaneous responses, leading to a decline in patience. This is amplified by the constant buzz of social media.

Consequences of Impatience. As we become more intolerant of delays, we are less likely to experience anything that requires us to wait, that doesn't provide us with instant gratification. This has cultural as well as personal consequences.

The Malleable Teenage Brain. Adolescence is a period of brain development particularly sensitive to environmental input. We must be concerned about the effects of technology, drug use, and lack of education on the developing teenage brain.

12. The Perils of "Smart" and the Stifling of Progress

Alas, smart solutions don’t translate into smart problem solvers.

The Invisible Costs of "Smart." As technologies become more powerful and ubiquitous, we are tempted to design out all imperfections from our interactions, social institutions, and politics. This can lead to a loss of creativity and a stifling of change.

Smart Solutions vs. Smart Problem Solvers. Smart solutions don't necessarily translate into smart problem solvers. In fact, the opposite might be true: Blinded by the awesomeness of our tools, we might forget that some problems and imperfections are just the normal costs of living with other human beings.

The Stifling of Technological Progress. Our social and moral intuitions are ill-equipped to deal with the rapid pace of technological innovation. This can hamper the adoption of technologies that might otherwise be of practical benefit to individuals and great benefit to society.

Last updated: January 18, 2025