The market cannot grade its own safety exam

The laboratory has offered to mark its own exam

Meta's chief executive argues that frontier laboratories already have enough incentive to build safely. Competition should reward trustworthy products, liability should punish damaging ones, and users should prefer companies that align systems with their interests. Reuters reports that Meta delayed its Muse model for months to strengthen security and uses independent evaluators, presenting voluntary caution as evidence that the incentive structure can work.

That is a serious argument, not a cartoon. A company can lose customers, talent, capital, and legal protection after a major failure. But the argument quietly substitutes motivation for measurement. Wanting to avoid a disaster does not establish that a laboratory can detect one, that its tests cover the right behavior, that decision-makers see unfiltered results, or that executives will stop when safety and growth point in opposite directions.

A safety incentive is not safety evidence

Markets reward qualities that buyers can observe or credibly infer. A car's crash performance can be compared through standardized tests. A bank's capital can be examined against defined requirements. Frontier-AI safety is harder: important capabilities emerge unpredictably, external evaluators see only part of the system, incident records remain private, and the laboratory controls the moment of release.

If trust itself becomes a product, the incentive to appear safe rises alongside the incentive to be safe. Those goals overlap until a finding threatens a launch, valuation, government relationship, or strategic lead. At that point, the public needs an institution capable of distinguishing a repaired control from a polished explanation.

Liability looks backward

Liability matters because it places a price on preventable harm. It is also an instrument that usually activates after someone can identify an injury, a responsible actor, a legal duty, and evidence connecting the failure to the damage. Frontier risk can be diffuse, delayed, cross-border, or hidden inside systems whose logs and training decisions belong to the defendant.

A lawsuit may compensate a victim or discipline a company. It cannot recover an exposed security method, reverse a manipulated election after certification, or reconstruct a capability jump that the laboratory never documented for outsiders. Liability should remain a backstop. It cannot be the sensor at the front of the system.

Europe is asking the more useful question

Reuters reports that the European Commission president plans talks with leading frontier laboratories and supports cooperation on model evaluation, verification, early warning, and AI security. The EU AI Act already gives the Commission oversight responsibilities for risk mitigation in advanced models. The proposal is preliminary, and a meeting is not an enforcement system. Still, its vocabulary is better.

Evaluation asks what the model can do. Verification asks whether a claim survives inspection by someone who does not benefit from the answer. Early warning asks how evidence travels before the harm is undeniable. Security asks whether the test environment, model, and resulting knowledge are protected. Together these questions move the debate from whether executives feel responsible to whether institutions can observe responsibility operating.

• Give independent evaluators continuing access rather than a curated release-day demonstration.
• Use comparable tests while allowing them to change as capabilities and attack methods change.
• Disclose material incidents, blocked evaluations, exceptions, and the decision owner.
• Predeclare the findings that trigger containment, delayed release, restricted tools, or regulator review.
• Protect sensitive technical detail without converting confidentiality into immunity from oversight.

The catastrophe debate shows why one score will fail

The Guardian's examination of six AI-catastrophe claims reveals disagreement over internet takeover scenarios, probability-of-doom estimates, regulatory capture, nuclear analogies, slowdown proposals, and competition with China. Some experts see credible pathways to loss of control. Others argue that exact probabilities are unfalsifiable, current systems remain brittle, and regulation may entrench the largest companies.

Independent verification will not manufacture consensus about extinction. It can force sharper claims. Which capability was demonstrated? Under what access? Against which defense? What evidence would lower the estimate? What control failed? A governance system is most valuable when it allows disagreement to remain while making the underlying evidence harder to manipulate.

Weak governance is already being financed by workers

Deloitte estimates that British workers spend about £958 million a year of their own money on generative-AI tools for work. In its weighted survey of 25,000 workers, 17 percent of users said they paid for at least one work tool themselves, 31 percent used generative AI without employer knowledge, and about half said they had received no formal training. Respondents reported saving time, much of which they used to do more work for the same employer.

This is the incentive story at ordinary scale. Employers can receive output and time savings while workers absorb subscription costs, uncertainty, stigma, and potential data exposure. The market is adopting AI, but the party enjoying the gain is not necessarily the party carrying the control burden. Verification must therefore reach deployment: approved tools, data boundaries, training, accountability, and a way to report failure without punishing the person who found it.

Politics is making the evidence test harder

The BBC describes a White House that treats AI acceleration as economic strategy, geopolitical competition, and political identity. Investment in chips and data centers is tied to growth, market value, and retirement savings, while the race with China supplies a ready answer to every request for delay. Skeptics exist across party lines, but opposing the president can carry a political cost for members of his coalition.

That environment strengthens the case for institutions that do not require leaders to resolve the entire philosophy of AI before acting. A laboratory can believe markets are powerful. A regulator can believe catastrophic risk deserves attention. A voter can believe both sides exaggerate. They should still be able to demand test access, incident evidence, and a named threshold for intervention.

Here is the challenge: publish the failed test

Every leading laboratory says it cares about safety. The useful distinction is what happens when the evidence becomes expensive. Will the company disclose the evaluation that delayed a launch? Will an independent reviewer be allowed to describe a blocked test? Will a regulator see the incident before a reporter does? Will a board identify the capability that it is unwilling to deploy even when a competitor moves first?

Do not ask the public to choose between faith in business and faith in bureaucracy. Run the harder experiment. Give outsiders continuing access, establish common evidence, define intervention before the pressure arrives, and publish enough failure to prove the system can learn. If the incentives are truly sufficient, independent verification will confirm it. If the laboratory refuses the test, the refusal is the result.