Test, don't (just) verify

kept by eddie dec 24, 25

AI-assisted formal verification shows promise but faces limitations — autoformalization trust, proof assistant speed, and model complexity — making random testing a necessary complement.

Human mathematicians are being outcounterexampled
AI tools have rapidly generated and formalized counterexamples to longstanding conjectures such as Erdős’ unit distance, Grothendieck’s group scheme problem, and the Jacobian conjecture, showing machines now out‑counterexample humans.
also on Lean, #open-source, #developer-tools
All Lean Books and Where to Find Them
A personal guide lists and reviews nine Lean 4 books, offering subjective opinions and suggested learning paths.
also on Lean, #open-source, #developer-tools
A New Era of Software Quality Starts Today
Momentic launches a rebuilt autonomous testing platform with memory, knowledge base, and AI agents that understand product behavior to close the QA gap caused by AI-accelerated code shipping.
also on #testing, #ai-integration, #developer-tools
How to really stop your agents from making the same mistakes (via @garrytan)
LangChain's LangSmith testing platform uses trajectory evals and trace-to-dataset pipelines to prevent AI agents from repeating mistakes, despite its high valuation and funding.
also on #testing, #ai-integration, #developer-tools