Reading up on data-efficiency

NanoGPT Slowrun achieves 10x data efficiency via ensembles, chain distillation, heavy regularization, and looping, enabling scale with compute instead of data.