Schedule

Most RAG demos fall over the moment real users show up. We rebuilt our retrieval stack three times; this talk is the version that stuck, including the chunking decisions we regret and the reranker that finally paid for itself.

A tour of the batching, caching, and routing changes that took our monthly bill from eye-watering to unremarkable — with the measurements that proved quality held.

Everyone writes an eval suite. Almost nobody maintains one. A practical pattern for evals that stay green, stay fast, and catch the regressions that matter.