Reading up on Kim et al.

NanoGPT Slowrun achieves 10x data efficiency via ensembles, chain distillation, heavy regularization, and looping, enabling scale with compute instead of data.