Reading up on llm-evaluation

Claude Fable 5 demonstrably solves difficult coding problems—like compiling CPython to WASM—that stumped earlier models, though evaluation remains subjective and vibe-based.

A study measuring five frontier LLMs on 1,000 recent fact-check claims found 67% disagreement, indicating limited inter-model agreement.

Iterating a prompt 200 times to improve codebase quality with Claude produces exponentially worse code, introducing bloat and bugs.