Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I
kept by eddie
An autonomous research agent discovered ~20 additive training improvements for nanochat, cutting the leaderboard's 'Time to GPT-2' by 11% from 2.02 to 1.80 hours.