Reading up on scale
1 deep · digging since may 04
- How did ‘large’ language models get that way? The role of Transformers and Pretraining in GPT - LessWrong 2.0 viewer
The transformer architecture and pretraining enabled enormous scaling of language models, making 'large language models' a fitting description.