First make it fast, then make it smart

kept by eddie nov 27, 25

Faster AI coding models, even if less intelligent, are more productive for quick mechanical code edits than slower, smarter models, especially for users with short attention spans.

Software engineering at a proprietary trading company: Optiver
Optiver, a proprietary trading firm, has evolved from latency-focused trading to AI-driven models, building custom hardware and full-stack systems while maintaining high engineering ownership and risk-averse speed.
also on Groq, Cerebras, #ai-coding, #software-engineering, #developer-tools
The emerging role of SRAM-centric chips in AI inference
SRAM-centric chips outperform GPUs on memory-bound AI inference workloads, particularly decode, by placing memory near compute for faster bandwidth.
also on Groq, Cerebras
Training Composer for longer horizons
Cursor trains Composer to generate its own compact summaries mid-task, reducing context errors by 50% while using 80% fewer tokens than traditional prompting.
also on Composer, Cursor, #ai-coding, #software-engineering, #developer-tools
I think Anthropic and OpenAI have found product-market fit
Anthropic and OpenAI may have reached product-market fit based on enterprise willingness to pay $200/month for tokens, though valuation and cost sustainability remain contested.
also on Cerebras, Cursor
Two different tricks for fast LLM inference
Anthropic's fast mode uses low-batch-size inference on the full model, while OpenAI's uses a smaller distilled model on Cerebras chips for much higher speed.
also on Cerebras, #ai-coding, #developer-tools