Designing AI resistant technical evaluations

kept by eddie jan 22

Anthropic redesigned its performance engineering take-home test three times because Claude models kept solving it, eventually adopting unconventional puzzles to stay ahead.

How To Scale Your Model
This online book teaches LLM scaling through roofline analysis, TPU/GPU hardware, Transformer math, and parallelism techniques to estimate training and inference costs.
also on TPU, #software-engineering, #developer-tools
Amazon’s Durability – Stratechery by Ben Thompson
Amazon's decade-long investments in infrastructure, custom silicon, and physical-world operations position it to dominate the inference era of AI.
also on Trainium, Anthropic, #ai
An exclusive tour of Amazon's Trainium lab, the chip that's won over Anthropic, OpenAI, even Apple
Amazon's Trainium chip is winning customers like Anthropic, OpenAI, and Apple as a cheaper Nvidia alternative, with 1.4 million chips deployed.
also on Trainium, Anthropic, #ai
Changing Devtools Is Cheap. Owning Them Isn’t. - Lalit Maganti
Personalizing devtools with AI lowers upfront costs but creates ongoing attention demands, making shared, well-designed tools with extension points preferable to widespread forking.
also on Perfetto, #ai-coding, #software-engineering, #developer-tools