The Coming Wave

1. The Coming Wave: AI and Synthetic Biology Reshape Our World

"Technology is not just a tool or platform but a transformative meta-technology, the technology behind technology and everything else, itself a maker of tools and platforms, not just a system but a generator of systems of any and all kinds."

Paradigm shift. The coming wave of technology, centered on artificial intelligence (AI) and synthetic biology, represents a fundamental transformation in human capabilities. These technologies are not mere tools, but meta-technologies that will reshape every aspect of our lives, from how we think and communicate to how we manipulate the very building blocks of life.

Unprecedented power. AI is rapidly advancing, with large language models like GPT-4 demonstrating human-level performance across a wide range of tasks. Synthetic biology allows us to read, edit, and write the code of life itself, opening up possibilities from designer organisms to radical life extension. This convergence of technologies promises to unlock incredible potential for solving global challenges, but also introduces risks on an unprecedented scale.

Societal impact. The coming wave will transform:

• Healthcare (personalized medicine, disease eradication)
• Energy (fusion, advanced renewables)
• Food production (lab-grown meat, engineered crops)
• Manufacturing (atomically precise production)
• Education (AI tutors, brain-computer interfaces)
• Governance (AI-assisted decision making)
• Warfare (autonomous weapons, bioweapons)

2. Unstoppable Proliferation: Technology's Historical Pattern

"Proliferation is the default."

Historical precedent. Throughout history, new technologies have consistently followed a pattern of widespread diffusion, regardless of attempts to contain them. From the printing press to nuclear weapons, once a technology is invented, it tends to spread and become more accessible over time.

Exponential growth. The rate of technological proliferation is accelerating. While it took centuries for innovations like the wheel to spread globally, modern technologies like smartphones achieved near-universal adoption in just a decade. This acceleration is driven by:

• Improved communication and information sharing
• Global interconnectedness
• Falling costs and increasing accessibility
• Network effects and positive feedback loops

Inevitability. Attempts to completely halt technological progress or proliferation have historically failed. Even when specific applications are banned or restricted (e.g., certain weapons), the underlying knowledge and capabilities continue to advance and spread. This pattern suggests that the coming wave of AI and biotech will likely follow a similar trajectory of unstoppable proliferation.

3. The Containment Problem: Unintended Consequences and Control

"Technology's unavoidable challenge is that its makers quickly lose control over the path their inventions take once introduced to the world."

Unintended consequences. Even well-intentioned technologies often lead to unforeseen and potentially harmful outcomes. Examples include:

• Social media's impact on mental health and political polarization
• Fossil fuels driving climate change
• Antibiotics leading to resistant superbugs

Control challenges. As technologies become more powerful and complex, maintaining meaningful control becomes increasingly difficult. This is especially true for AI systems that can learn and evolve independently, and for synthetic biology where small changes can have far-reaching consequences.

Systemic risks. The interconnected nature of modern technologies means that failures or misuse in one area can quickly cascade to affect entire systems. This amplifies the potential for catastrophic outcomes and makes effective containment even more crucial.

4. Four Features of the Coming Wave: Asymmetry, Hyper-Evolution, Omni-Use, and Autonomy

"The coming wave is going to change the world. Ultimately, human beings may no longer be the primary planetary drivers, as we have become accustomed to being."

Asymmetry. The coming wave technologies enable individuals or small groups to wield disproportionate power. A single programmer with an AI system could potentially impact millions of lives, or a lone biohacker could create a dangerous pathogen.

Hyper-evolution. These technologies are developing at an unprecedented pace, constantly improving and branching into new areas. This rapid evolution makes it difficult for regulatory frameworks and social norms to keep up.

Omni-use. AI and synthetic biology are general-purpose technologies with an incredibly wide range of applications. This versatility makes them powerful but also challenging to regulate or restrict to specific domains.

Autonomy. Increasingly, these technologies can operate independently of human oversight. This raises profound questions about control, accountability, and the role of human decision-making in critical systems.

5. Unstoppable Incentives: Why We Can't Say No to Technology

"Science has to be converted into useful and desirable products for it to truly spread far and wide. Put simply: most technology is made to earn money."

Economic drivers. The potential for enormous profits drives relentless technological development. Companies and investors pour billions into AI and biotech research, seeking competitive advantages and new markets.

Geopolitical competition. Nations view technological supremacy as crucial for economic and military power. This fuels an arms race mentality, with countries like the US and China competing intensely in areas like AI development.

Scientific curiosity. The innate human drive to explore and understand pushes researchers to continually push boundaries, even when potential risks are acknowledged.

Solving global challenges. Many see advanced technologies as essential for addressing existential threats like climate change, pandemics, and resource scarcity.

6. The Grand Bargain: Nation-States Under Pressure

"The nation-state will be subject to massive centrifugal and centripetal forces, centralization and fragmentation."

Eroding foundations. The traditional social contract between citizens and nation-states is under strain. Technologies of the coming wave challenge core state functions like:

• Maintaining monopoly on violence (autonomous weapons)
• Economic management (AI-driven markets, cryptocurrencies)
• Border control (digital realms, bioengineered identities)
• Information control (decentralized networks, synthetic media)

Power shifts. New centers of power are emerging, including:

• Tech giants with resources rivaling nations
• Decentralized autonomous organizations (DAOs)
• Empowered individuals and small groups

Adaptation imperative. Nation-states must evolve rapidly to remain relevant and maintain legitimacy in the face of these technological disruptions. This may require reimagining core governance structures and the very nature of citizenship.

7. Fragility Amplifiers: New Threats to Stability

"Eventually, as smaller-scale technology failures mount, calls for control increase. As control increases, checks and balances get whittled down, the ground shifts and makes way for further interventions, and a steady downward spiral to techno-dystopia begins."

Cyber vulnerabilities. As critical infrastructure becomes increasingly connected and AI-dependent, the potential for devastating cyberattacks grows. A single breach could potentially cripple entire sectors of the economy or government functions.

Information warfare. Advanced AI-generated disinformation and deepfakes threaten to undermine trust in institutions and the very concept of shared reality. This poses existential risks to democratic societies.

Bioterror potential. Synthetic biology lowers the barriers for creating designer pathogens, potentially putting pandemic-level threats in the hands of non-state actors or lone individuals.

Economic disruption. AI and automation may lead to massive job displacement, exacerbating inequality and social unrest if not carefully managed.

8. The Dilemma: Navigating Between Catastrophe and Dystopia

"On the one hand, total openness to all experimentation and development is a straightforward recipe for catastrophe. If everyone in the world can play with nuclear bombs, at some stage you have a nuclear war. [...] On the other side of the ledger, though, as should also be clear, complete surveillance and complete closure are inconceivable, wrong, and disastrous."

Catastrophic risks. Uncontrolled development of powerful technologies could lead to existential threats:

• Artificial General Intelligence (AGI) beyond human control
• Engineered pandemics
• Nanotechnology "grey goo" scenarios
• Environmental collapse from unforeseen consequences

Dystopian control. Attempts to completely contain technological risks could result in authoritarian nightmares:

• Pervasive surveillance states
• Severe restrictions on individual freedom and innovation
• Technological stagnation leading to inability to solve global challenges

Narrow path. The challenge is to find a balance that allows for beneficial technological progress while mitigating the most severe risks. This requires unprecedented global cooperation and governance innovation.

9. Ten Steps Toward Containment: A Framework for Action

"Containment is not a resting place. It's a narrow and never-ending path."

Multilayered approach. Effective containment of the coming wave requires action at multiple levels:

1. Technical safety: Building robust safeguards directly into AI and biotech systems.
2. Audits: Developing rigorous, independent verification processes.
3. Choke points: Identifying and managing key bottlenecks in technological development.
4. Makers: Fostering a culture of responsibility among technologists.
5. Businesses: Aligning corporate incentives with long-term societal benefit.
6. Governments: Strengthening state capacity to understand and regulate advanced technologies.
7. Alliances: Forging international agreements and collaborative frameworks.
8. Culture: Promoting societal values that balance innovation with caution.
9. Movements: Building popular support for responsible technological development.
10. The Narrow Path: Maintaining constant vigilance and adaptability.

10. The Narrow Path: Balancing Innovation and Safety

"Safe, contained technology is, like liberal democracy, not a final end state; rather, it is an ongoing process, a delicate equilibrium that must be actively maintained, constantly fought for and protected."

Continuous effort. There is no permanent solution to the challenges posed by the coming wave. Instead, society must commit to an ongoing process of assessment, adaptation, and careful governance.

Embracing complexity. Effective containment requires nuanced approaches that:

• Allow beneficial technological progress
• Mitigate the most severe risks
• Preserve core values like individual liberty and human rights
• Remain flexible enough to address unforeseen developments

Global cooperation. The transnational nature of these technologies demands unprecedented levels of international collaboration. This includes:

• Harmonizing regulatory frameworks
• Sharing best practices and early warning systems
• Joint research initiatives on safety and ethics
• Mechanisms for collective response to global technological crises

Last updated: July 21, 2024