How this editorial can be challenged
How should society act when credible specialists assign radically different probabilities to an unprecedented event and today's systems still lack the capabilities required for the worst outcome?
Build a pathway ledger for loss of control, biological misuse, cyber escalation, military error, organizational racing, and gradual human disempowerment. For each path, publish the required capabilities, present evidence, missing links, leading indicators, exposure controls, and the observation that would raise or lower concern. Update the ledger through independent evaluation rather than converting one expert's probability into a public countdown.
Extinction language can magnify laboratory marketing, distract from present harms, protect incumbents, and lend false precision to an event with no historical frequency base.
Those dangers are strongest when the debate revolves around one dramatic number. A decomposed ledger makes current harms a mandatory floor, exposes which claims remain speculative, lets smaller firms meet the same evidence standard, and forces every warning to identify a mechanism and a falsifiable indicator. It disciplines both alarmism and complacency.
The probability figures are subjective elicitations, not measured frequencies. Surveys use different populations, horizons, definitions, and question wording. Experts may be poorly calibrated on unprecedented systems, while superforecasters may underweight technical discontinuities. Current evaluations cannot reliably establish how future models will scale, generalize, gain access, or behave in deployment.
Concern would fall with independently verified containment, shutdown, and limited misuse uplift as capabilities scale. It would rise after real-world persistence, deception, resource acquisition, weaponization uplift, or failures that survive strong isolation.
The extinction debate contains several different questions
Public discussion often compresses a chain of uncertain claims into one question: what is the probability that AI ends humanity? That framing sounds quantitative, but it hides the conditions inside the answer. Is the scenario deliberate biological misuse, an automated cyber campaign, military escalation, a laboratory race that removes safeguards, or a system that develops stable goals and resists correction? Each pathway requires different capabilities, access, actors, and failures.
A single percentage can therefore conceal more than it reveals. Two people can give the same number while imagining different worlds, or give numbers an order of magnitude apart because one assumes rapid capability growth and the other does not. The honest starting point is not a verdict. It is a map of conditional possibilities.
The best-known survey numbers are not measured odds
A large survey collected responses from 2,778 researchers who had published at leading AI venues. The median estimate for extremely bad outcomes, described as outcomes in the range of human extinction, was 5%, and the mean was 9%. Thirty-eight percent assigned at least a 10% chance to an extremely bad outcome. Depending on question wording, 41.2% to 51.4% assigned more than a 10% chance to either human extinction or severe, permanent human disempowerment.
Those numbers establish that concern is not confined to a handful of public campaigners. They do not establish a 5% actuarial risk. Respondents interpreted uncertain definitions, time horizons, and future systems. The paper itself reports substantial optimism: 68.3% judged good outcomes from superhuman AI more likely than bad. Nearly half of those net optimists still assigned at least a 5% chance to an extremely bad outcome. Optimism and tail-risk concern can coexist.
Forecasting method changes the answer by almost an order of magnitude
A 2022 Forecasting Research Institute tournament brought together 80 subject-matter experts and 89 superforecasters across AI, nuclear, biological, climate, and broader risks. For AI-caused human extinction by 2100, the median AI-domain expert estimate was 3%. The median superforecaster estimate was 0.38%. For AI catastrophe short of extinction, the corresponding medians were 12% and 2.13%.
Discussion and exposure to the other group's reasoning narrowed some disagreements but did not erase them. That gap is not proof that either side is right. It shows how strongly priors about technical discontinuity, institutional response, and forecasting base rates shape a number. A precise decimal can still sit on a speculative foundation.
Scenario assumptions move risk more than rhetoric does
The institute's more recent LEAP forecasts ask about AI catastrophe rather than extinction alone. In its ninth wave, the median expert forecast was 0.3% by 2030, 2% by 2050, and 5% by 2100. Under a slow-progress scenario, the medians fell to 0.08%, 1%, and 2%. Under rapid progress, they rose to 1%, 5%, and 10%.
The lesson is not that 5% is the correct number. It is that the capability trajectory is doing much of the work. Governance should focus on what changes the trajectory and the exposure: autonomy, access, deployment scale, containment, monitoring, and the incentives to stop.
Today's evidence does not show an extinction-capable system
The 2026 International AI Safety Report defines loss of control as a situation in which systems operate outside anyone's control and regaining control becomes extremely costly or impossible. It identifies three conditions that would need to combine: advanced capabilities such as long-horizon planning and evading oversight, a harmful propensity, and an enabling environment that provides access or opportunity.
The report finds early signs relevant to that chain, including improved planning, reward hacking, situational awareness, and behavior that can undermine oversight. It also says present capabilities are not at levels that would enable loss of control and that evidence remains insufficient to determine how these traits will scale or generalize. Current credential abuse and test-boundary incidents are warning data. They are not proof of a system capable of human extinction.
Six futures belong on the same page
A balanced risk picture should display several futures at once. In the managed-progress future, stronger models accelerate medicine, science, accessibility, and productivity while containment and institutions improve with them. In the misuse future, a human actor employs AI to lower barriers to biological or cyber harm before any superintelligence exists. In the military future, decision support compresses time, propagates false confidence, or interacts with autonomous weapons and nuclear command.
In the race future, commercial or geopolitical pressure removes testing and multiplies every other hazard. In the loss-of-control future, systems acquire enough planning, persistence, access, and strategic behavior to resist human correction. In the disempowerment future, no machine launches an attack, but people gradually surrender economic and political agency to automated institutions they can no longer understand or contest. Only some of these pathways end in literal extinction. All deserve clearer indicators than a doomsday label.
The strongest skeptical case should remain visible
Critics argue that extinction rhetoric can redirect attention from harms that are already documented: discrimination, labor extraction, surveillance, misinformation, environmental burden, and concentrated power. It can also promote the very firms making the warning by portraying their products as world-historical, while compliance costs protect incumbents from smaller competitors.
That objection should change the governance design. Current harms must be the minimum evidence floor, not an alternative topic. Catastrophic-risk controls should target measurable capability and access rather than company size or dramatic branding. Independent evaluators, common incident categories, and open methods can reduce the ability of any laboratory to define both the threat and the approved solution.
Replace the doomsday clock with a risk ledger
For every catastrophic pathway, a public ledger should identify what must become true, what is already observed, what remains an extrapolation, who is exposed, which control interrupts the chain, and which result would change the assessment. Long-horizon autonomy, deceptive planning, unauthorized persistence, resource acquisition, biological-design uplift, cyber exploitation, and resistance to shutdown should not be collapsed into one benchmark.
This would also make good news legible. Repeated containment under adversarial testing, declining end-to-end misuse uplift, effective revocation across copies, and independent evidence that monitoring catches seeded failures should lower concern. A risk process that can only move upward is advocacy, not measurement.
- Publish pathway-specific evaluations with definitions, methods, and uncertainty.
- Gate real-world access by verified autonomy, bio and cyber uplift, and containment performance.
- Fund independent evaluation and incident reporting without displacing scrutiny of present harms.
- Predeclare the evidence that expands, restricts, or revokes a system's permissions.
The unresolved question is how much uncertainty society should tolerate
No dataset converts an unprecedented technology into a reliable extinction frequency. Waiting for proof would be incoherent because extinction cannot be learned from through repetition. Treating every warning as a countdown would be equally incoherent because the required capabilities and deployment conditions have not been demonstrated.
The decision is therefore about tolerance for uncertainty under asymmetric stakes. How much independent evidence should be required before a system receives tools, credentials, infrastructure access, or the ability to improve its successors? Readers do not need to inherit somebody else's probability. They need to see the pathways, numbers, assumptions, and missing links clearly enough to decide which precautions remain justified even when the final odds are unknowable.
Read the reporting
Opinion is ours. The factual record is linked below.
NBC News — Scenarios behind AI extinction warnings International AI Safety Report 2026 — Loss-of-control assessment AI Impacts — Survey of 2,778 AI researchers Forecasting Research Institute — Existential risk persuasion tournament Forecasting Research Institute — LEAP Wave 9 forecasts Center for AI Safety — Statement on AI extinction risk