Reading up on helm
2 deep · digging since feb 27
- AI evals are becoming the new compute bottleneck
AI evaluation costs now rival or exceed training costs, with agent benchmarks reaching $40,000 per sweep and training-in-the-loop evals resisting effective compression.
- Introducing helm - Ben Gubler
Helm is a typed TypeScript framework that replaces dozens of AI agent tools with two (search and execute) and sandboxes LLM-generated code with granular permissions.