A murder appeal crossed the AI-hallucination line from fake citations to fabricated testimony
The New Mexico Supreme Court says a defense lawyer filed a murder-appeal brief containing false testimony from wholly fabricated witnesses, additional false statements attributed to real witnesses, and misrepresented legal authority after using ChatGPT to prepare the document. The lawyer admitted that he did not verify the factual claims or legal authority before signing and filing. The court found him in direct contempt, fined him $5,000, referred the matter to the disciplinary board, barred him from appearing before the court pending that process, struck the briefing, and ordered the public defender's office to appoint new counsel. This case is more serious than a familiar hallucinated-citation story because invented facts entered the record of a criminal appeal, where liberty and procedural fairness are at stake. The court's response correctly keeps professional responsibility with the lawyer, but individual discipline cannot be the entire control system. A long transcript fed into a general chatbot can produce fluent compression without preserving evidentiary identity, page-level provenance, or the distinction between quoted testimony and plausible reconstruction. Legal workflows should require every factual assertion to link back to the authoritative record before it can enter a filed document. Tools used for case summarization should preserve citations at generation time, flag unsupported propositions, and block quotation marks when no source span exists. Human review becomes real only when the interface makes verification possible and the institution audits whether it happened.