Reading up on gpu-computing

LLM inference splits into compute-bound prefill and memory-bound decode, driving optimization techniques like batching, quantization, speculative decoding, and disaggregation.

nCPU implements a complete CPU entirely with neural networks on a GPU, achieving 100% accuracy on integer arithmetic with multiplication 12x faster than addition.