AI-generated software proofs are impressive because they make a limited promise

This is a kind of AI progress worth taking seriously: an output that can be checked rather than merely admired. Anthropic and Theorem describe using Claude to produce formal proofs about defined software properties. The important difference from a reassuring explanation is that a proof checker can verify whether the formal conclusion follows. That does not remove every source of trust, but it can make the remaining assumptions more visible. The public OSS Verify repository concerns compiled RISC-V binaries derived from selected open-source fuzz targets. Its proofs operate within stated limits and a formal model of a Linux process. The repository explicitly describes that model as incomplete and containing assumptions. The result is not a certificate that every version of a library, every execution environment or an entire deployed product is free of bugs. Anthropic also describes a wider research effort; the public proof subset should not be mistaken for the whole claimed corpus. What interests me is the division of labor. AI can attempt a difficult construction, while a separate checker assesses a precise proposition. The human responsibility shifts toward deciding whether that proposition captures the property that matters and whether the model resembles the deployment. If the wrong property is proved, the mathematics can be sound while the practical assurance is insufficient. The next useful evidence would be independent reproduction against exact versions, clear treatment of changes and examples of maintainers using the proofs in their release process. This is company-led research with public artifacts, not independent confirmation of universal security. Even with those limits, it points toward a more productive standard for AI claims: show what was proved, what must still be trusted, and where the guarantee ends.