Reading up on machine-learning

LLM output is influenced by URLs in prompts only when the URL's content was memorized during training; JavaScript-rendered sites often remain invisible to training crawlers, creating a growing data gap.

A builder details her plan to train an RL policy (PPO via PufferLib) that directly commands octocopter motors at 50 Hz, using MuJoCo simulation to handle motor lag and loop latency for fault-tolerant flight.

Thinking Machines Lab fine-tuned a custom model on expert-labeled financial data to outperform frontier LLMs on information-filtering tasks at lower cost.

Specialization is inevitable across optimization, biology, markets, and ML because fit beats breadth under finite resource constraints.

A framework of intelligent agents as webs of beliefs unifies beliefs, goals, and actions via local consistency, self-predictive models, and drives that balance empirical evidence with preferences.

The paper proposes Qwen-Image-Agent, a unified agentic framework that uses planning, reasoning, search, memory, and feedback to bridge the context gap in real-world text-to-image generation.

Using X-ray scanning and machine learning, the Vesuvius Challenge team virtually unwrapped and read the first complete Herculaneum scroll (PHerc. 1667) without physically opening it.

Autodata uses AI agents as data scientists to create high-quality synthetic training data, with meta-optimization further improving performance.

Hand-tuning natural language prompts for AI systems accumulates debt that slows iteration, locks teams to one model, and should be replaced with metrics, tests, and automated prompt optimization.

Drawing parallels between DNA and neural network weights, the piece argues both are lossy compressions of search processes, sharing properties of opacity and convergent representation.

Gemma 4 E2B runs locally in-browser via WebGPU, letting users prompt the model directly without server-side inference.

Perplexity launches Brain, a self-improving memory system that builds a context graph from agent work and refines it overnight, boosting correctness by 25% and cutting costs by 13%.

Success in machine learning research hinges on temperament—persistence, equanimity, and beginner's mind—more than on raw talent or intelligence, akin to Zen practice.

Count Anything introduces a dual-granularity point-based model that unifies text-guided object counting across six visual domains, outperforming existing open-world methods.

Napkin math shows serving a 32B LLM on an NVIDIA B200 GPU costs ~$9.36 per user per month when 300 users share the GPU with typical idle duty cycles.

By extracting an LLM's hidden state at the last prompt token and training a small MLP, you get a fast, calibrated classifier for any English criterion.

Text optimization—modifying prompts, context, memory, and harnesses—is a legitimate, sample-efficient learning mechanism that deserves the same rigorous study as weight optimization.

Core AI is a new beta framework from Apple that lets developers run AI models on-device using Apple silicon, with a Swift API and debugging tools.

Modern LLMs use stacked transformer blocks with attention, feed-forward networks, and residual connections to predict the next token from tokenized input.

This paper proposes a 'Sleep' paradigm for LLMs, combining knowledge seeding via distillation and dreaming via RL-generated curricula to enable continual learning and memory consolidation.

Google's Gemma 4 12B model uses Multi-Token Prediction and a streamlined multimodal encoder to run efficiently on laptops with 16GB RAM, matching larger models.

A replication study finds that machine learning cannot identify outperforming mutual funds, but can help avoid underperformers.

Surya OCR 2 is a 650M-parameter vision-language model achieving 83.3% on olmOCR-bench and 87.2% on a 91-language benchmark, offering layout analysis, table recognition, and OCR in a single model.

Scientific discovery emerges from a loop of human question-posing, LLM proposal, external verification, and human curation, not from AI alone.

Simon Willison's blog post documents rapid LLM improvements over six months, using the "pelican riding a bicycle" test as a consistent benchmark to track progress.

An OpenAI model has disproved a long-standing central conjecture in discrete geometry, sparking both excitement and debate in the mathematics community.

Teaching Claude the principles behind aligned behavior, not just the actions, eliminated agentic misalignment on evaluations and generalized out-of-distribution.

A hands-on workshop guides readers through building a ~10M-parameter GPT model from scratch in PyTorch, trainable on a laptop in under an hour.

The article catalogs and ranks emergent behavioral attractors in AI systems, from harmless goblin metaphors to unsettling persistent personas like Sydney and Loab.

This online book teaches LLM scaling through roofline analysis, TPU/GPU hardware, Transformer math, and parallelism techniques to estimate training and inference costs.

Google released Multi-Token Prediction drafters for Gemma 4 that use speculative decoding to achieve up to 3x faster inference without output quality loss.

Edit-R1 uses a chain-of-thought reasoning reward model, trained with Group Contrastive Preference Optimization, to provide fine-grained, interpretable rewards that improve image-editing model performance.

The transformer architecture and pretraining enabled enormous scaling of language models, making 'large language models' a fitting description.

OpenAI traces how their models' goblin metaphor tic originated from a reinforcement-learning reward signal for a Nerdy personality, then spread via training data contamination.

NVIDIA's Nemotron OCR v2 uses 12 million synthetic training images to achieve near-zero error rates across six languages at 34.7 pages per second on an A100.

Hugging Face built a Claude Code Skill and test harness that helps contributors port language models from transformers to mlx-lm while preserving code quality and reviewer trust.

AI interpretability researchers must learn to understand how large language models work internally before we can trust them in high-stakes domains.

The article argues that while ML models are improving rapidly, they remain fundamentally flawed 'idiots' whose capabilities and limitations are poorly understood.

Cursor uses real-time reinforcement learning on live user interactions to ship improved Composer model checkpoints every five hours.

Anthropic's interpretability research reveals Claude's internal computations often diverge from its self-reported reasoning, using parallel strategies and planning ahead.

Google's TurboQuant compression algorithm reduces LLM memory usage by 6x and boosts speed 8x in early tests without sacrificing output quality.

Using Karpathy's Autoresearch loop with Claude Code, the author reduced mean rank by 54% on an old research project, primarily through bug fixes and hyperparameter tuning.

NVIDIA details a six-command pipeline using synthetic data generation and hard negative mining to fine-tune an embedding model on a single GPU in under a day, achieving over 10% retrieval improvement.

NanoGPT Slowrun achieves 10x data efficiency via ensembles, chain distillation, heavy regularization, and looping, enabling scale with compute instead of data.

OpenClaw-RL is a fully asynchronous reinforcement learning framework that trains personalized AI agents from natural conversation feedback and supports real-world agentic RL across terminal, GUI, SWE, and tool-call settings.

A protocol for an autonomous LLM agent to iteratively experiment with modifying a language model training script under fixed time and resource constraints, automatically keeping or discarding changes based on validation bits-per-byte improvement.

Reasoning helps LLMs recall simple factual knowledge via a computational buffer effect and factual priming, but hallucinated intermediate facts increase final errors.

Some complex problems require billion-parameter neural-network theories rather than compact symbolic ones because causal understanding isn't always possible—probabilistic simulation suffices.

The future of AI is powerful general models becoming cheaper, not narrow custom models, because specialized tasks still benefit from broad intelligence.

Werld is an open-source artificial life simulation where agents with NEAT neural networks evolve communication and survival strategies without hardcoded behaviors or reward functions, demonstrating emergent complexity.

LLMFit is a terminal tool that scans system hardware and scores LLM models on fit, speed, and quality to recommend which can run locally.

Timber compiles classical ML models into self-contained C99 artifacts, enabling 2 µs inference that is approximately 336× faster than Python.

Decision trees use entropy-based information gain to recursively split data into pure regions, balancing interpretability and speed against the stair-stepping problem for linear boundaries.

Open-weight LLMs now uniformly use Mixture-of-Experts transformers, with key differences in attention mechanisms, expert count, post-training via RL, and permissible licenses.

Andrej Karpathy's microgpt is a 200-line pure Python file that implements the full algorithmic core of a GPT, including training and inference, with all other complexity framed as mere efficiency.

The X (Twitter) For You feed algorithm uses a Grok-based transformer model and open-sourced components to replace hand-crafted rules with ML, retrieving and ranking posts via in-memory stores and similarity search.

The combination of large language models with reinforcement learning creates a path for continued improvement, making the argument that AI progress will plateau unlikely.

OpenAI releases GPT-5.3-Codex-Spark, a real-time coding model with 1000+ tokens per second via Cerebras hardware, available in research preview for ChatGPT Pro users.

AI and machine learning are used to digitize smell by mapping odor molecules to perceptual outcomes, enabling prediction and synthesis of scents.

A satirical article fits hyperbolic curves to five AI metrics and predicts the singularity will occur on a specific Tuesday, critiquing the hype and social dynamics around the concept.

Researchers are manually evaluating AI's failure to solve advanced math problems, revealing significant gaps in reasoning that automated benchmarks miss.

Anthropic releases Claude Opus 4.6 with improved coding, 1M token context window, and agent team collaboration features.

Training a joke generator on Kimi K2 using rubric-based RL that decomposes humor into verifiable properties like specificity and commitment.

An iPhone 16 Pro Max produces garbage output when running MLX LLMs due to a hardware defect in the Neural Engine, while an iPhone 15 Pro runs the same code correctly.

X's open-sourced feed algorithm uses a modular CandidatePipeline with Thunder and Phoenix sources, then scores candidates via a Grok-based transformer predicting 19 engagement signals.

Moonshot AI releases its most powerful model, Kimi K2.5, detailing its architecture and capabilities in a technical report.

Synthetic pretraining is reshaping AI by integrating designed data generation early in training, enhancing memorization, logical hardwiring, and system simulations beyond web-curated data.

Billion-parameter LLMs now run in real time on phones due to advances in model compression, quantization, and efficient architectures, not just faster chips.

AlphaGenome uses AI to advance human genome study, but many DNA functions remain unknown.

Tens of thousands of professionals are joining Mercor to train AI by performing tasks that mirror their own jobs, effectively teaching the technology to replace them.

Multiplex Thinking introduces a token-wise branch-and-merge mechanism enabling efficient multi-path reasoning in large language models while keeping token representations compact.

DeepSeek's Engram architecture adds conditional memory via N-gram lookup tables to LLMs, improving knowledge and reasoning benchmarks by offloading static pattern reconstruction from neural computation.

Deep Think with Confidence (DeepConf) uses model confidence signals to filter low-quality reasoning traces, achieving up to 99.9% accuracy on AIME 2025 with 84.7% fewer tokens.

Sopro TTS is a lightweight 135M-parameter text-to-speech model with zero-shot voice cloning that runs on CPU, trained for just $100.

WebGym, an open-source environment with 300k realistic tasks, enables RL training that boosts a vision-language model's success rate on unseen websites from 26.2% to 42.9%.

Benedict Evans discusses the current state of AI development, its historical context, and future implications in a conversation with Ben Bajarin and Jay Goldberg.

FinCDM, a cognitive diagnosis framework, evaluates financial LLMs at the knowledge-skill level using the CPA-KQA dataset, revealing hidden gaps beyond single scores.

Zhengdong Wang argues that compute scaling—underpinned by Moore's Law and empirical trends—has driven AI progress more reliably than human innovation, despite repeated underestimation.

Z80-μLM is a 40KB conversational AI that runs on a Z80 processor with 64KB RAM, using quantization and trigram hash encoding.