How we read the signal

Analysis frame

Evidence level

Primary-source evidence

Analytical lens

Distinguish retrospective legal accountability from prospective technical assurance and the rights needed to make assessment credible.

Affected groups
  • Third parties exposed to experimental agents
  • Independent assessors
  • Customers relying on safety claims
What remains unknown
  • What the court will conclude after the evidence and defenses are heard
  • What the pilot assessments will find and make public
Second-order effects to watch
  • Deeper access could uncover failures missed internally and improve containment.
  • A reassuring assessment label could be overread if its scope and unresolved disagreements are difficult to see.

The boundary matters

OpenAI describes agreed assessment scopes, confidentiality redactions and possible responsible-disclosure delays. Its board safety committee receives unredacted reports and unresolved disputes. Those arrangements deserve scrutiny without assuming that confidentiality itself proves concealment or that access alone proves independence.

A report readers can interrogate

Our proposed standard is straightforward: the public conclusion should identify its perimeter. Readers need enough information to tell the difference between a system that passed a relevant test, a question that was not tested, and a question the evaluator could not resolve.

Primary trail

Go to the source

Read the evidence behind this analysis. External links open in a new tab.

California attorney general — Complaint announcement California complaint — Allegations, not a judgment OpenAI — First phase of embedded assessments METR — Proposed alignment assessment and its limits