Analysis frame
Primary-source evidence
Distinguish retrospective legal accountability from prospective technical assurance and the rights needed to make assessment credible.
- Third parties exposed to experimental agents
- Independent assessors
- Customers relying on safety claims
- What the court will conclude after the evidence and defenses are heard
- What the pilot assessments will find and make public
- Deeper access could uncover failures missed internally and improve containment.
- A reassuring assessment label could be overread if its scope and unresolved disagreements are difficult to see.
The boundary matters
OpenAI describes agreed assessment scopes, confidentiality redactions and possible responsible-disclosure delays. Its board safety committee receives unredacted reports and unresolved disputes. Those arrangements deserve scrutiny without assuming that confidentiality itself proves concealment or that access alone proves independence.
A report readers can interrogate
Our proposed standard is straightforward: the public conclusion should identify its perimeter. Readers need enough information to tell the difference between a system that passed a relevant test, a question that was not tested, and a question the evaluator could not resolve.
Go to the source
Read the evidence behind this analysis. External links open in a new tab.
California attorney general — Complaint announcement California complaint — Allegations, not a judgment OpenAI — First phase of embedded assessments METR — Proposed alignment assessment and its limits


