The last six months in LLMs in five minutes

kept by eddie may 22

Simon Willison's blog post documents rapid LLM improvements over six months, using the "pelican riding a bicycle" test as a consistent benchmark to track progress.

AI IQ — Intelligently Measuring AI Intelligence
The piece presents a methodology that maps public benchmark scores to IQ estimates across seven dimensions, enabling comparison of AI models by intelligence, speed, and cost.
also on DeepSeek, #benchmarks, Google, OpenAI, Anthropic, #developer-tools
The browser is the sandbox
The browser's sandbox, tested by decades of web abuse, provides a secure environment for AI agents using the File System Access API.
also on Simon Willison, Google
AI should help us produce better code - Agentic Engineering Patterns - Simon Willison's Weblog
Coding agents can improve code quality by handling refactoring, enabling exploratory prototyping, and compounding small quality improvements over time.
also on Simon Willison, OpenAI, #developer-tools
I built a vulnerable app and spent $1,500 seeing if LLMs could hack it
A security researcher built a deliberately vulnerable Firebase-based app and spent $1,500 testing whether LLMs could exploit it; GPT-5.5 succeeded 70% of the time, while most others failed due to guardrails or misdirected focus.
also on DeepSeek, #benchmarks, OpenAI, Anthropic, #developer-tools