▲ 0 r/Rag
I built SPANCHOR regression testing for RAG retrieval pipelines
I've been working on a problem I don't see discussed enough in RAG: retrieval regressions.
You change chunking, embeddings, the retriever, indexing, or configuration, and your tests can still pass while the evidence being retrieved has changed.
I built SPANCHOR, an open-source Python package for testing this.
The core workflow is:
retrieved chunk → source span → evidence evaluation → baseline vs candidate → regression result
The goal is to verify that a RAG pipeline continues retrieving the expected source evidence after a change.
I'm looking for technical feedback from people actually building RAG systems:
What retrieval regressions have you encountered that your current evaluation setup doesn't catch?
u/mukeshrams — 11 days ago