Reading up on cost-optimization
9 deep · digging since feb 10
- The Cache Is the Price - Julien Simon
The cost of handing off a job from a cheap model to a frontier model depends on prompt caching rates, which can make routing more expensive than using the frontier alone.
- The Pulse: a new trend, smart model routing - The Pragmatic Engineer
Several vendors now offer intelligent model routers that automatically select the cheapest sufficient LLM per task, promising 20-30% cost savings, a trend expected to become table stakes.
- Most AI Work Can Wait
Routing layer design, not model choice, drives cost efficiency by sending 70-80% of AI traffic to cheap local or async models.
- never waste a token
Putting a durable buffer between AI agents and LLM providers prevents token waste and re-billing when a process crashes mid-stream, with resumable streaming and crash recovery using the same mechanism.
- I run multiple $10K MRR companies on a $20/month tech stack
Running multiple $10K MRR businesses on a $20/month stack with SQLite, a VPS, and minimal services is possible, but the approach is controversial among developers.
Takes
Bye bye Beehiiv 👋 I vibe-coded my own newsletter platform: 📧 Campaign management w/ editor 🖥️ Web archives (https://t.co/pypYdNhdlZ) 🔗 API support 🫧 List cleanup, deliverability reports It took a few prompts with Opus 5.5. I left Beehiiv because they raised prices for the 3rd time in 2 years. - My first yearly bill was $1,000. - My future bill would have been $4,000. Now I'm using AWS SES + MongoDB, and I will pay less than $300/year. Newsletter platforms charge per subscriber, when in fact the real cost is the number of emails sent. Subscribers will naturally become inactive, so you end up paying more for something that hurts your deliverability (sending to dormant subscribers = spam trap). So I've set a cleanup cron job for my newsletter alternative. It marks inactive subs as dormant, sends one last re-engaging email, and stops emailing them. Money saved. Deliverability increased. I also gain a priceless feature: Freedom. I can change the UI, add features, connect all my apps - anything is possible with just a prompt. I'm replacing Resend next. I love it, but it's very expensive ($400/mo)
@marclou
Configured Grafana for all my projects Managed APMs are fine for typical SaaS where most visitors are paying you. But they get expensive quick if you have high traffic sites like job boards, etc. Now self-hosting, own the data, and can give my AI agents direct access for troubleshooting 👌
@marckohlbrugge
P.S. this Mapbox cost is another cost detected by my new Situation Monitor dashboard with AI insights, it scouts all my projects insights on what to improve A few weeks ago it detected the Cloudflare bill was too high and we found they made a mistake which they quickly fixed and refunded Really nice!
@levelsio
https://t.co/ex8cdBWSQr
@Shpigford