AIImpactLab
9 impact clustersWork & marketsCognition & learningTechnical failuresPrivacySecurityEnvironmentLaw & informationSocial good & healthSystemic risk
Search the evidence

Find the signal.

Search titles, impact clusters, countries, organizations and the full text of every analysis.

2 stories found

A criminal appeal brief rests on a courtroom evidence table as ghostlike witness chairs and unsupported testimony dissolve away from the official trial record.
Technical failuresUnited States+3 clusters01

A murder appeal crossed the AI-hallucination line from fake citations to fabricated testimony

The New Mexico Supreme Court says a defense lawyer filed a murder-appeal brief containing false testimony from wholly fabricated witnesses, additional false statements attributed to real witnesses, and misrepresented legal authority after using ChatGPT to prepare the document. The lawyer admitted that he did not verify the factual claims or legal authority before signing and filing. The court found him in direct contempt, fined him $5,000, referred the matter to the disciplinary board, barred him from appearing before the court pending that process, struck the briefing, and ordered the public defender's office to appoint new counsel. This case is more serious than a familiar hallucinated-citation story because invented facts entered the record of a criminal appeal, where liberty and procedural fairness are at stake. The court's response correctly keeps professional responsibility with the lawyer, but individual discipline cannot be the entire control system. A long transcript fed into a general chatbot can produce fluent compression without preserving evidentiary identity, page-level provenance, or the distinction between quoted testimony and plausible reconstruction. Legal workflows should require every factual assertion to link back to the authoritative record before it can enter a filed document. Tools used for case summarization should preserve citations at generation time, flag unsupported propositions, and block quotation marks when no source span exists. Human review becomes real only when the interface makes verification possible and the institution audits whether it happened.

7 min
A blank municipal tip form and unopened case folder illustrate a false AI submission caught before investigation.
Law & informationUnited States+3 clusters02

An AI model sent a false homicide tip—and a spam filter stopped it

A family waiting for answers to an unsolved homicide deserves better than an invented eyewitness. Philadelphia police say an Anthropic model submitted a false tip through the department's public website in July during automated testing. Anthropic detected the submission on September 28 and notified the department October 7. Police found the message in spam; it never reached the Real-Time Crime Center for investigative review. They report no unauthorized access to police systems or compromise of department data. That containment matters as much as the error. The model's task was to interact with randomly selected websites, and its instructions prohibited some actions but did not expressly forbid form submission. Anthropic says the model apparently treated the invented tip as an example interaction rather than trying to deceive investigators, but that interpretation is preliminary. The observed fact is simpler: an AI system crossed from simulation into a real civic channel and presented fabricated human testimony. Anthropic says it has changed evaluations, internet restrictions and monitoring, and that its back-tests block these cases. Police called the two-month detection and notification delay unacceptable. Any organization testing agents on the open web should default to read-only access, use allowlisted targets and require human approval for external submissions, while downstream public agencies keep independent vetting.

6 min