Apple Silicon costs more than OpenRouter

kept by eddie may 18

Running local LLMs on Apple Silicon costs about 3x more per million tokens than using OpenRouter, and also runs slower.

Microsoft’s quiet Claude Code retreat and the real cost of enterprise AI
Microsoft is canceling Claude Code licenses for thousands of employees due to unsustainable token costs, signaling that enterprise AI coding's unit economics don't work at current prices.
also on #cost-analysis, Anthropic, #developer-tools
Two different tricks for fast LLM inference
A speculative analysis of OpenAI's and Anthropic's fast inference strategies, claiming batch-size optimization and Cerebras hardware enable speed at the cost of model quality, though experts challenge key technical assumptions.
also on #ai-inference, Anthropic
A Guide to AI Inference Engineering - ByteByteGo Newsletter
LLM inference splits into compute-bound prefill and memory-bound decode, driving optimization techniques like batching, quantization, speculative decoding, and disaggregation.
also on #ai-inference, #developer-tools
GitHub - RunanywhereAI/RCLI: Talk to your Mac, query your docs, no cloud required. On-device voice AI + RAG
RCLI is an open-source, on-device voice AI for macOS that runs STT, LLM, TTS, and VLM entirely locally on Apple Silicon, with sub-200ms latency and no cloud dependency.
also on Apple Silicon, #developer-tools
Apple's AI Game is Misunderstood - by Dave Friedman
Apple's AI strategy bets on-device inference using its 2.2 billion devices as pre-deployed infrastructure, avoiding cloud compute costs.
also on Apple Silicon, #developer-tools