Reading up on AWQ

Billion-parameter LLMs now run in real time on phones due to advances in model compression, quantization, and efficient architectures, not just faster chips.

LLM inference works by tokenizing input, computing embeddings through transformer layers, then generating tokens autoregressively with KV caching and quantization optimizations.