How we read the signal

Analysis frame

Evidence level

Primary-source evidence

Analytical lens

Test company-reported defense metrics against external validity, false positives and the value of independent access.

Affected groups
  • AI labs protecting models and safeguards
  • Researchers, developers and users affected by access controls
What remains unknown
  • The 10- to 100-fold result is company-run and method-specific
  • Real-world false positives and appeal outcomes have not been independently verified
Second-order effects to watch
  • Effective controls may protect models and their safety guardrails
  • Overbroad controls may concentrate power and limit scrutiny

What Anthropic measured

The company reports a 10- to 100-fold reduction in extracted reasoning tokens against methods it tested and false positives below 0.1%. It does not claim every future method is stopped.

What independent scrutiny should test

Look for results across new methods, real user false positives, appeal outcomes and whether qualified researchers can continue evaluating consequential model behavior.

Primary trail

Go to the source

Read the evidence behind this analysis. External links open in a new tab.

Anthropic — distillation defenses