How this editorial can be challenged
What does it mean to trust a powerful system when the people affected cannot inspect the evidence?
AI vendors control model access, logs, benchmarks and commercial terms. The people exposed to the results often see only a polished answer or a product promise. That gap lets both genuine progress and genuine problems remain difficult to independently verify.
Full public access to weights, attack details and customer logs could enable abuse, expose private data and make hard-won intellectual property easy to copy.
That is a serious constraint, not an excuse for zero scrutiny. Confidential third-party evaluations, scoped regulator access, aggregate incident reporting and independently auditable usage records can test consequential claims without publishing exploit details or personal records.
Anthropic's replication and distillation results are company-run evaluations, not a census of models in the wild. Reuters reports supervisors' questions but not a new binding banking rule. SPUR's telemetry standard has been published, but industry-wide adoption and payment outcomes remain unknown.
If independently selected evaluators could routinely reproduce consequential model claims, regulators could review agent controls before incidents, and publishers could reconcile usage and compensation without relying on vendor-only records, the evidence divide would be materially smaller.
The black box is not only technical
A bank customer, a publisher and a person using an AI assistant do not have the same problem. Yet each may depend on facts held almost entirely by the company selling the tool: what the system did, what it used and what it was permitted to do. A model can be genuinely useful while the evidence needed to judge it remains inaccessible.
That is an institutional black box. Its walls are contracts, logs, evaluation access and bargaining power as much as neural-network complexity.
A laboratory result needs its boundaries
Anthropic's self-replication research describes contained tests in which models completed pieces of a replication task under simplified conditions. The work is a reason to study this capability carefully; it is not proof that a model escaped into ordinary networks. The paper's explicit constraints are part of the evidence, not fine print to omit.
The question for the next lab claim is whether outsiders can examine the setup, repeat a meaningful portion and see failures as well as successes. If only the vendor can inspect the test, public debate oscillates between overreaction and dismissal.
The bank's version is a stop button
Reuters reports that banking supervisors are asking institutions how AI agents are approved, monitored, limited and shut down. Those are reported questions, not a newly enacted rule. They still reveal the practical form of accountability: a supervisor needs to know who authorized an agent, what records it left and whether a human can intervene before losses cascade.
Banks already understand that a system can be profitable and fragile at once. AI should not receive a waiver from that discipline because its output looks conversational.
The publisher's version is a meter
The SPUR coalition has published a content telemetry standard that separates retrieval, grounding, citation, display and engagement. That granularity matters. A link in an answer is not the same thing as a system using reporting to construct the answer, and neither tells a publisher what was licensed or paid for.
A standard is only an opening move. Its economic force depends on adoption, independent reconciliation and enforceable agreements. But a shared record gives a negotiation something more concrete than competing stories about how often content was used.
Secrecy has a legitimate case
The strongest objection to inspection is not that companies dislike criticism. Some details should stay private. Attack pathways can help attackers; customer logs can expose personal data; model outputs and methods can be copied. Anthropic's new anti-distillation defenses are an example of why a lab may limit what it reveals and to whom.
The answer cannot be indiscriminate transparency. It is graded access: confidential evaluators for sensitive tests, regulators with defined authority, public summaries that disclose limits, and records that can be challenged without dumping raw data onto the open web.
A test for the next year
The European Commission says its AI Office has received serious-incident reports and is asking companies questions. We do not know the underlying incidents from that notice. We should resist filling that gap with speculation. We should also ask whether the process eventually produces findings that affected people can understand and contest.
The same test applies across the economy: can someone other than the vendor check a claim that matters? I would look for reproducible evaluation methods, agent activity logs accessible to supervisors, documented incident follow-up and publisher usage records that can actually be reconciled.
Trust should be earned in daylight
My forecast is that the next durable AI institutions will not be those that demand the most trust. They will be those that make narrow, consequential claims easier to check while protecting legitimate secrets. That may mean slower announcements and more uncomfortable findings in public. It could also mean fewer surprises after deployment.
If independent inspection remains exceptional, the costs of error will continue to land on people least able to see the evidence. A customer will be told a decision was automated; a publisher will be told its work was merely cited; a community will be told a model was tested. The right response is not to accept or reject AI as a whole. It is to insist that the evidence behind each consequential claim can be examined by someone who does not answer to the seller.
Read the reporting
Opinion is ours. The factual record is linked below.
Anthropic — self-replicating AI evaluation Reuters — banking watchdog scrutiny SPUR — content telemetry standard Anthropic — distillation defenses European Commission — AI Scientific Panel meeting