Synthetic Pretraining

kept by eddie feb 02

Synthetic pretraining is reshaping AI by integrating designed data generation early in training, enhancing memorization, logical hardwiring, and system simulations beyond web-curated data.

2025 LLM Year in Review
Karpathy's 2025 review identifies RLVR, jagged intelligence, Cursor, Claude Code, vibe coding, and Gemini Nano Banana as paradigm-changing developments in LLMs.
also on DeepSeek, #machine-learning, #ai-research, #generative-ai
Engram: How DeepSeek Added a Second Brain to Their LLM | rewire.it
DeepSeek's Engram architecture adds conditional memory via N-gram lookup tables to LLMs, improving knowledge and reasoning benchmarks by offloading static pattern reconstruction from neural computation.
also on DeepSeek, #machine-learning, #ai-research