Multi-tenant RAG with hybrid search and an eval harness

Most SaaS products that add a 'chat with your docs' feature face the same two risks: one customer's content surfacing in another customer's answers, and retrieval quality silently degrading when someone tweaks chunking or prompts. This design treats both as engineering problems with hard guarantees instead of things to check by hand.

• Tenant filtering done in application code is one forgotten WHERE clause away from a data leak.
• Pure vector search misses exact terms — product codes, error strings, names — that keyword search finds trivially.
• Without a repeatable evaluation, every change to chunking, embeddings or prompts is a guess.

Documents are parsed, chunked by heading structure and stored in a single PostgreSQL table with a pgvector embedding column and a tsvector column. Every row carries a tenant ID, and a Row Level Security policy restricts all reads and writes to the tenant set on the current database session — the API sets it once per request, so no query can reach another tenant's rows. At query time the question runs through two retrievers in parallel: full-text search and vector similarity. Their rankings are merged with reciprocal rank fusion and the top candidates go through a reranker before the LLM composes an answer with citations back to the source chunks. A separate eval harness holds a golden set of questions with expected source passages; it reports recall@k and answer faithfulness, and runs in CI so a change that lowers either score fails the pipeline.

The part I would pressure-test first is the shared index: RLS filters after the vector index returns candidates, so a small tenant inside a large table can get fewer results than requested. The next step is measuring that on a skewed dataset and comparing partial indexes per tenant against partitioning.