Reading up on ai-evaluation

The piece presents a methodology that maps public benchmark scores to IQ estimates across seven dimensions, enabling comparison of AI models by intelligence, speed, and cost.

AI evaluation costs now rival or exceed training costs, with agent benchmarks reaching $40,000 per sweep and training-in-the-loop evals resisting effective compression.

LMArena's human-voted leaderboard rewards superficiality and verbosity over factual accuracy, making it an unreliable metric for AI model quality.