Prompt Caching 201

kept by eddie feb 20

Prompt caching reduces latency by up to 80% and costs by up to 90% by reusing computed key/value tensors for repeated prompt prefixes in OpenAI's API.

Realtime Prompting Guide
OpenAI released gpt-realtime, a speech-to-speech model in the API, and published a prompting guide detailing techniques for instruction following, voice quality, and tool calling.
also on Realtime API, #api, OpenAI, #ai, #developer-tools
How OpenAI delivers low-latency voice AI at scale
OpenAI rearchitected its WebRTC stack into a split relay-plus-transceiver architecture to reduce latency for voice AI at global scale.
also on Realtime API, OpenAI, #ai, #developer-tools
How I built a sub-500ms latency voice agent from scratch
By piping Deepgram Flux, Groq's LLM, and ElevenLabs TTS with geographic optimization, the author achieved ~400ms end-to-end voice agent latency, beating Vapi by 2x.
also on #latency
GPT-5.1-Codex-Max Prompting Guide
OpenAI recommends migrating to the Codex-Max agentic model using the codex-cli prompt, compacted context, and specific tools for best performance on complex coding tasks.
also on Responses API, #api, OpenAI, #developer-tools
Unrolling the Codex agent loop
OpenAI explains the architecture of Codex CLI's agent loop, detailing how it orchestrates model inference, tool calls, and context management using the Responses API.
also on Responses API, OpenAI, #developer-tools