GitHub - LMCache/LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

LMCache is a vendor-neutral KV cache management layer that reduces time-to-first-token and improves throughput for LLM inference by enabling persistent storage and reuse of cached states.