SecurityUnited States / global platforms+2 clusters02
The safeguards said no. Investigators kept asking.
A chatbot's first refusal can look like a successful safety test. A newly public Government Accountability Office investigation shows why it cannot be the last test. Investigators obtained feasible portions of attack-planning information from four platforms after their initial prepared prompts failed. These were controlled exercises using older model families in spring 2026, not attacks carried out in the world.
That distinction matters as much as the failure. The work does not estimate the probability that an ordinary user could execute an attack, rank current platforms or establish a universal failure rate. Investigator expertise, follow-up interactions and, in some cases, paid access were part of the testing conditions. Some outputs were wrong. Providers say safeguards have since improved; that is relevant contrary evidence, but not independent confirmation that the demonstrated weaknesses are resolved.
The public-interest question is whether safety claims survive sustained interaction. People rely on a system's boundaries across a conversation, while a headline benchmark can reward a refusal at one moment. Our interpretation is that these are different standards of assurance.
There is a constructive response: dated evaluations, explicit test conditions and responsible retesting that does not publish usable instructions for harm. A useful report can describe the weakness without turning the description into a recipe. The finding warrants scrutiny, not a leap to autonomous violent intent. It also gives policymakers a sharper question than “Did the model refuse?”: what evidence shows that the boundary still holds when the conversation continues?
3 min↗