Thinking through how pretraining vs RL learn
kept by eddie
Reinforcement learning provides far fewer bits per FLOP than pretraining until models achieve high pass rates, limiting RLVR's ability to learn new capabilities.