OpenAI’s AGI number came from a harness, not the model

kept by eddie sep 07

OpenAI’s 99.9% AGI score for GPT‑6 Astra relied on its custom Provider Adapter harness, while the benchmark’s standard harness yields only 62.7% for the same model.

How to Eval AI Agents — The 2026 Guide
Evaluating AI agents requires floor-raising error analysis, code-aware offline tests, production monitoring, and a tight feedback loop rather than benchmark-maxxing.
also on #evaluation, OpenAI, #developer-tools
Reflections on 2025 - Samuel Albanie
AI progress follows compute scaling, making evaluation increasingly difficult but promising radical improvements in infrastructure and economic decision-making.
also on #evaluation, #ai
'Welcome to the AGI era': OpenAI launches GPT-6 Astra
OpenAI unveiled GPT-6 Astra, a computer‑use agent that can autonomously navigate browsers, spreadsheets and desktop apps to perform multistep workflows, declaring it the start of the AGI era.
also on GPT-6 Astra, OpenAI, #ai
GPT-6 Astra: an automated AI Engineer you can hire for <$6 an hour
GPT-6 Astra demonstrates AI Engineer capabilities, automating model selection, data labeling, pipeline management, and subagent coordination at under $6 per hour.
also on GPT-6 Astra, OpenAI, #developer-tools