Production RAG: The Chunking, Retrieval, and Evaluation Strategies That Actually Work

kept by kotsakis oct 28

Production RAG systems fail with naive fixed-size chunking and top-k retrieval, so the author recommends semantic chunking, hybrid search with BM25, reranking, context expansion, and a rigorous evaluation framework to improve accuracy and reliability.

Can agents replace the search stack?
Using a basic BM25 or e5 retriever with an LLM agent can achieve 0.289→0.453 NDCG on Amazon ESCI by reasoning over queries, but this approach fails on passage retrieval where the embedding model already knows best.
also on #retrieval-augmented-generation, BM25
How AI Agent Memory Works
An interactive essay explains how agent memory systems work through context windows, embeddings, and retrieval architectures to overcome LLM statelessness.
also on #retrieval-augmented-generation
The first search engine with fully vectorized query execution
Turbopuffer v3 will be the first search engine to adopt fully vectorized query execution, enabling strong performance on complex queries.
also on BM25
People that worked on the same idea for decades — Nitya's Notes
The article showcases long-term dedication, illustrating how Robertson's IR research over decades produced BM25 and Hokusai's forty-year journey to his iconic Great Wave.
also on BM25
Reflections on 2025 - Samuel Albanie
AI progress follows compute scaling, making evaluation increasingly difficult but promising radical improvements in infrastructure and economic decision-making.
also on #evaluation, #software-engineering, #ai