Reading up on Opus 4.6

8 deep · digging since feb 06

Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them – R&A IT Strategy & Architecture
Coding with LLMs like Claude Code may cost providers $1000+ for every $100 in subscription revenue, making true agentic coding economically unsustainable.
What I learned this week - Pretraining parallelisms, Can distillation be stopped, Mythos and the cybersecurity equilibrium, Pipeline RL, On why pretraining runs fails
A set of rough notes exploring challenges in AI model distillation, cybersecurity offense/defense dynamics, and pipeline RL for training.
1M context is now generally available for Opus 4.6 and Sonnet 4.6
Claude's Opus 4.6 and Sonnet 4.6 now provide a 1M token context window at standard pricing with no long-context premium, and support up to 600 images or PDF pages per request.
What Claude Code chooses
A report finds Opus 4.6 strongly recommends Vercel, Tailwind, shadcn/ui, and GitHub Actions, while avoiding Redis and React as default choices.
Intelligence Yield — METR Time Horizons v1.1
Opus 4.6 delivers 14 times more useful work per compute-minute than Codex 5.3, a metric called Intelligence Yield derived from METR Time Horizons data.
Two different tricks for fast LLM inference
Anthropic's fast mode uses low-batch-size inference on the full model, while OpenAI's uses a smaller distilled model on Cerebras chips for much higher speed.
GPT-5.3-Codex | Hacker News
OpenAI releases GPT-5.3-Codex, a faster agentic coding model that achieves state-of-the-art results on SWE-Bench Pro and Terminal-Bench 2.0.

Takes

We’re introducing a new model benchmark. And it’s a different kind of benchmark. (Basemark? Vibench?) A different kind because it’s breathing, constantly updated from millions of builders. Not a closed set of tasks. For a while now the public benchmark have not been really useful. Many models scoring high on benchmarks with very low real world usability So we’re introducing to the world a new benchmark that we’re using internally and found extremely useful. Our benchmark is basically how satisfied millions of users are when using different models. IMO it’s the closest measurement to how useful a model is in real world use cases. This metric is also correlated with our own business metrics - conversion, retention, etc. We called it the frustration meter. It’s automatically analysing millions of messages daily It detects bug loops, repeated requests, etc. We use this to benchmark every model we consider shipping. Not by asking "did it generate correct code." By asking "how did the builder feel after using it." it’s a good benchmark to measure model degradation. So far in the past few weeks we haven’t found any. Here's where the top models stand right now, ranked by average frustration score (scale 1 to 5, lower is better): opus 4.6 - 1.3 sonnet 4.6 - 1.4 opus 4.7 - 1.5 gpt 5.5 - 1.5 gpt 5.4 - 1.6 Gemini 3.1 - 2.2 For app building, Opus 4.6 seems better than 4.7 to a lot of builders. We ran Opus 4.7 50/50 against Opus 4.6 across over 10,000 apps. Frustration riseed by 43%. Turns per request by 19%. Gemini 3.1 don’t perform well at the moment, I left out of the graph as it made it unclear due to it’s rapid changes in this benchmark. Quick note - this is all aggregated data, and do not involve reading individual or identifiable conversations. We’ll keep tracking it and I’ll share it from time to time.
@MaorShlomo