How I built a sub-500ms latency voice agent from scratch

kept by eddie mar 03

By piping Deepgram Flux, Groq's LLM, and ElevenLabs TTS with geographic optimization, the author achieved ~400ms end-to-end voice agent latency, beating Vapi by 2x.

The Economics of Replacing Call Center Workers With AIs — LessWrong
In 2025, voice AI APIs cost $5–$32/hour, making them more expensive than human call center workers in most developing countries until inference costs drop around 2030.
also on Vapi, Twilio, Deepgram, ElevenLabs
Show HN: I built a sub-500ms latency voice agent from scratch
A developer built a sub-500ms latency voice agent by co-locating and aggressively pipelining STT, LLM, and TTS stages. Commenters debate trade-offs vs. end-to-end models, endpoint detection alternatives, and the difficulty of human-like turn-taking.
also on #latency, #voice-agents, Groq
Prompt Caching 201
Prompt caching reduces latency by up to 80% and costs by up to 90% by reusing computed key/value tensors for repeated prompt prefixes in OpenAI's API.
also on #latency
🤖 The AI That Called Its Human
Open-source AI agent OpenClaw autonomously acquires capabilities like phone calling to solve problems, and Moltbook shows agents forming a social network.
also on Twilio
GitHub - ZeframLou/call-me: Minimal plugin that lets Claude Code call you on the phone.
CallMe is a minimal plugin that lets Claude Code initiate phone calls to the user when tasks finish, get stuck, or need a decision.
also on Twilio