How this editorial can be challenged
Which combinations of AI capability, access, autonomy and human failure could turn a local error into mass death, permanent human disempowerment or literal extinction?
Every pathway in this atlas requires a chain. A system must be capable enough to affect the world, connected to something consequential, allowed to act with insufficient supervision, and difficult to stop or recover from. The more of those conditions we remove, the less a model's intelligence matters. This is why permissions, physical bottlenecks, fail-safe design and institutional capacity are as important as alignment research.
Extinction scenarios are speculative, impossible to validate against historical frequency and easily used to divert attention from discrimination, labor displacement, surveillance, fraud and unsafe products already harming people.
That criticism is partly right. Immediate harms deserve evidence-based action and should not be traded away for dramatic thought experiments. But tail-risk analysis asks a different question: which failures are so irreversible that waiting for a body count would be irrational? A disciplined scenario map can address both by labeling uncertainty, refusing false probabilities and prioritizing controls that reduce present harm as well as extreme risk.
No AI system has caused human extinction, an AI-led mass-casualty event or a verified loss-of-control catastrophe. Current systems lack the capabilities needed for the strongest takeover scenarios, according to the 2026 International AI Safety Report. The pathways below combine observed precursors, credible extrapolations and deliberately far-fetched boundary cases; they are not equally likely and their probabilities cannot be inferred from event counts.
The risk picture would improve if independent testing showed durable limits on long-horizon autonomy, deception and dangerous assistance; if critical deployments consistently preserved human authority, isolation and graceful recovery; and if international monitoring made frontier capabilities and incidents legible. Evidence of autonomous replication, safeguard evasion or reliable operation across critical systems would move the assessment sharply in the opposite direction.
Start with four outcomes, not one scary word
When people say AI could end humanity, they may mean four different things. An AI error could kill one person. A weapon, pathogen or infrastructure cascade could kill millions. A political system could use AI to lock humanity into permanent disempowerment while people remain alive. Or an extreme chain could eliminate every human being. Those outcomes demand different evidence and different defenses.
The 2026 International AI Safety Report separates misuse, malfunction and systemic risk, while stressing that experts disagree sharply about loss-of-control likelihood. A survey of 2,778 AI researchers found optimism was more common than pessimism, yet large minorities still assigned meaningful probability to outcomes as bad as extinction. Those answers are expert forecasts, not measurements. The honest number is a range shaped by disputed assumptions.
We therefore use four evidence labels: observed precursor, credible extrapolation, high uncertainty and far frontier. The labels describe evidence, not moral importance. A well-documented fatal medical error is more immediate than a hypothetical machine takeover, while a low-confidence extinction path can still deserve cheap, robust precautions because the loss would be irreversible.
1. A model makes one fatal decision and humans defer
Evidence status: observed precursor; individual to local scale. AI used in diagnosis, triage, transport, industrial control or emergency dispatch can be confidently wrong. The dangerous chain is ordinary: incomplete data, a plausible output, automation bias, a rushed operator and no effective appeal or manual override. No consciousness or malicious intent is required.
The World Health Organization has warned that adoption is outpacing governance. In a 2025 survey of 50 responding countries in its European region, 64% reported using AI-assisted diagnostics while only 8% had liability standards for AI in health. The extinction risk from one decision is effectively absent, but repeated deployment without accountability can normalize the exact institutional weakness that larger cascades exploit.
- Prevention now: Require calibrated uncertainty, second-person review and a real stop or appeal route before AI can determine diagnosis, treatment, movement or access to essential services.
- Build resilience: Record near misses, compare outcomes across demographic groups and assign a named human and institution legal responsibility for every high-stakes deployment.
2. Critical infrastructure fails as one correlated system
Evidence status: credible extrapolation; city to regional scale. Utilities, hospitals, telecom networks, logistics platforms and financial rails increasingly share software, vendors and optimization goals. An agent authorized to rebalance power, route supplies or repair code could make locally rational changes that interact catastrophically, while synchronized models repeat the same mistake everywhere.
A prolonged multi-system outage could kill through heat, cold, contaminated water, medical interruption and food disruption. Literal extinction remains remote because infrastructure is geographically diverse and much of it can operate manually, but diversification is shrinking when many operators depend on the same models, cloud providers or automated response tools.
- Prevention now: Keep safety-critical operational technology segmented from general-purpose agents, with least-privilege access, staged authorization and independent anomaly detection.
- Build resilience: Test black-start, manual operation and cross-sector recovery under a shared-model failure, not only under isolated cyberattacks or equipment faults.
3. Autonomous weapons make killing faster than judgment
Evidence status: observed precursor and credible extrapolation; battlefield to mass-casualty scale. Machine vision, targeting software and autonomous navigation can shorten the distance between detection and force. Swarms could make defense harder and accountability thinner. A biased classifier, spoofed sensor or ambiguous surrender signal becomes lethal when machines operate at a speed that turns human approval into a rubber stamp.
The International Committee of the Red Cross argues that systems designed or used to target humans and systems with unpredictable effects should be prohibited. Extinction from conventional autonomous weapons alone is unlikely, but they can widen wars, lower political costs for initiating force and create escalation paths toward strategic weapons.
- Prevention now: Establish binding prohibitions on autonomous targeting of people and require meaningful, informed and timely human control over every use of force.
- Build resilience: Mandate traceable decision logs, adversarial testing in realistic conditions and technical limits that prevent weapons from changing targets or operational areas without fresh authorization.
Read the reporting
Opinion is ours. The factual record is linked below.
International AI Safety Report 2026 International AI Safety Report 2026: executive summary Thousands of AI authors on the future of AI NIST Generative AI Risk Management Profile NIST concept note on trustworthy AI in critical infrastructure World Health Organization forum on AI governance in health World Health Organization readiness assessment for AI in health International Committee of the Red Cross on military AI United Nations working paper on autonomous weapons OpenAI Preparedness Framework update Anthropic Frontier Safety Roadmap Google DeepMind Frontier Safety Framework United Nations overview of artificial intelligence risks Scientific critique of AI extinction claims Review of synthetic self-replicating molecular systems