Reading up on FlexAttention
1 deep · digging since jun 22
- LLMs are complicated now – Ian’s Blog
Modern LLMs have grown complex with many attention variants and mixture-of-experts, echoing the messy evolution of recommendation systems.
One topic. Every takeSeek and you shall find