Training a 3.8B LLM to 0.384 CORE for $998
kept by night
The author trained a 3.8B-parameter LLM from scratch on 65B tokens for $998, achieving a CORE score of 0.384 using B200 GPUs and various optimizations.