Reading up on DeepSeek-V3

LLM inference splits into compute-bound prefill and memory-bound decode, driving optimization techniques like batching, quantization, speculative decoding, and disaggregation.

Open-weight LLMs now uniformly use Mixture-of-Experts transformers, with key differences in attention mechanisms, expert count, post-training via RL, and permissible licenses.

Contra Tim Dettmers, Dan Fu argues current AI systems significantly underutilize hardware, models lag hardware buildout, and multiple paths exist to useful AGI.