Reading up on LLaMA

Modern LLMs have grown complex with many attention variants and mixture-of-experts, echoing the messy evolution of recommendation systems.

AGI capable of most cognitive work could arrive by 2028–2034, but deployment will lag capability due to verification bottlenecks, uneven automation, and institutional friction.

LLM inference works by tokenizing input, computing embeddings through transformer layers, then generating tokens autoregressively with KV caching and quantization optimizations.