Engram: How DeepSeek Added a Second Brain to Their LLM | rewire.it

kept by eddie jan 14

DeepSeek's Engram architecture adds conditional memory via N-gram lookup tables to LLMs, improving knowledge and reasoning benchmarks by offloading static pattern reconstruction from neural computation.

The Limit in the Loop
As AI agents scale, stateless interactions fail, making memory a systems problem requiring active maintenance—write control, deduplication, and purposeful forgetting—rather than simple storage.
also on Engram, #software-architecture
Thinking through how pretraining vs RL learn
Reinforcement learning provides far fewer bits per FLOP than pretraining until models achieve high pass rates, limiting RLVR's ability to learn new capabilities.
also on #scaling-laws, DeepSeek, #machine-learning
AI Versus Microservices - michaelnygard.com
Microservices were originally a technical fix for organizational scaling, but the rise of AI coding agents now demands larger code ownership and new architectural boundaries, creating significant tension.
also on #scaling-laws, #software-architecture
Synthetic Pretraining
Synthetic pretraining is reshaping AI by integrating designed data generation early in training, enhancing memorization, logical hardwiring, and system simulations beyond web-curated data.
also on DeepSeek, #machine-learning, #ai-research
2025 LLM Year in Review
Karpathy's 2025 review identifies RLVR, jagged intelligence, Cursor, Claude Code, vibe coding, and Gemini Nano Banana as paradigm-changing developments in LLMs.
also on DeepSeek, #machine-learning, #ai-research