On-Device LLMs: State of the Union, 2026 – Vikas Chandra – AI Research @ Meta

kept by eddie jan 29

Billion-parameter LLMs now run in real time on phones due to advances in model compression, quantization, and efficient architectures, not just faster chips.

How LLM Inference Works
LLM inference works by tokenizing input, computing embeddings through transformer layers, then generating tokens autoregressively with KV caching and quantization optimizations.
also on AWQ, #machine-learning, #ai, #developer-tools
As Rocks May Think
Coding agents combined with reasoning LLMs have become automated scientists, enabling a golden age where all computer science problems appear tractable through vast inference compute.
also on DeepSeek-R1, #ai-research
The Future is for Everyone
Mark Zuckerberg argues that distributing superintelligence broadly to individuals, rather than centralizing it, will empower people, drive invention, and ensure a balanced, prosperous AI future through personal agency and widespread access.
also on HuggingFace, Meta, #ai, #developer-tools
Kimi-K3 on HuggingFace
The article announces that Kimi-K3, a recently released language model, has been uploaded to the HuggingFace model hub for developers to download and use.
also on HuggingFace, #machine-learning, #ai
datalab-to/surya-ocr-2
Surya OCR 2 is a 650M-parameter vision-language model achieving 83.3% on olmOCR-bench and 87.2% on a 91-language benchmark, offering layout analysis, table recognition, and OCR in a single model.
also on Llama.cpp, #machine-learning, #ai, #developer-tools