Demystifying evals for AI agents

kept by eddie jan 12

Anthropic's guide to building evaluations for AI agents emphasizes structured tasks, multiple grader types, and iterative refinement to enable confident shipping at scale.

[2602.02361] SWE-Universe: Scale Real-World Verifiable Environments to Millions
SWE-Universe automatically constructs 807,693 real-world software engineering environments from GitHub PRs via iterative self-verification and in-loop hacking detection.
also on SWE-bench Verified, #software-engineering
GPT-5.3-Codex | Hacker News
OpenAI releases GPT-5.3-Codex, a faster agentic coding model that achieves state-of-the-art results on SWE-Bench Pro and Terminal-Bench 2.0.
also on Terminal-Bench, #benchmarks, Anthropic, #developer-tools
Introducing Composer 2
Cursor released Composer 2, a frontier-level coding model achieving strong benchmarks (CursorBench 61.3, Terminal-Bench 61.7) at competitive pricing.
also on Terminal-Bench, #benchmarks, #developer-tools