Publish the decision. Keep the dangerous test secret.

A letter that counts precisely

The part of the federal response I keep coming back to is the count. Thirty records. Sixty pages. An index describes emails about access for trusted partners. An ordinary reader learns something quite specific about the machinery of government and almost nothing about the judgment that machinery produced. That isn’t proof of a bad review. It is a good illustration of why it is so hard to tell a good review from a bad one from the outside.

What I would leave out

Suppose a lab demonstrates a dangerous capability to a government evaluator. I don’t want the prompt, the exploit or a recipe for recreating it in a public file. Even a numerical score can be revealing if an outsider already knows how the test works. A review process that makes it dangerous to share a finding with the reviewer could leave us less informed, not more. Secrecy has a legitimate job here.

What I would put in

Now imagine a second document, short enough to read on a phone. It says which office made the decision, what it had authority to decide, whether the recommendation was accepted and when it is due for reconsideration. If the lab and the reviewer disagreed, it records the existence of the disagreement without describing the exploit. In a voluntary scheme, it says so. The distinction between “we advised waiting” and “we could prevent release” should not require a security clearance. This is a proposal, not an account of what those sixty pages contain.

Give the clock a place in the record

Anthropic’s filing makes the timing problem concrete. Its warning about accelerated research rests on expectations for future models; it hasn’t designated a present one above the threshold. A reviewer could reasonably reach one conclusion today and a different one next month. Without a date and a reason to reopen the decision, yesterday’s careful judgment is liable to become tomorrow’s blanket assurance. A dated decision lets someone ask whether the evidence changed before the permission did.

Show me a decision someone changed

There is a modest procurement example worth borrowing from Canada. A federal prototype challenge asks for a way to accept, reject or annotate governance findings and retain their history. It is a request to bidders, not a product that has proved itself or a Canadian rule for American labs. The valuable detail is the verb “reject.” When someone challenges an AI finding, can the institution preserve what changed without depending on the vendor to reconstruct the dispute later?

The empty chair at the next review

Picture the next reviewer taking over after a colleague leaves. A thick technical appendix may tell her what the system did on a particular test. It may not tell her who accepted the remaining risk or promised to look again after an update. If she has to hunt down the author to find out, the institution hasn’t really kept the decision. It has kept an attachment.

I would be satisfied with a public record that leaves several boxes deliberately blank, provided a cleared, independent reviewer can inspect what lies behind them. And if even a delayed summary would expose a dangerous capability, say what kind of information was withheld and arrange the independent check in private. The person sitting in that chair next month should inherit more than a page count.