Hybrid retrieval
BM25-style lexical search and semantic concept matching recover different kinds of evidence instead of depending on one signal.
Open source · Evaluation-first · Live
A RAG system that shows its work.
RAG Studio makes retrieval-augmented generation inspectable. It exposes the evidence, scores, latency, citations, confidence gate, and evaluation behind an answer—because a fluent answer is not the same as a trustworthy one.
LIVE WORKBENCH ↗
12golden-set queries
0.833mean reciprocal rank
0.958context recall @ 4
100%citation coverage
100%abstention accuracy
01 / SYSTEM DESIGN
Retrieval is a pipeline,
Retrieval is a pipeline,
not a prompt trick.
The system combines complementary retrieval signals, ranks evidence deliberately, then makes “do not answer” a valid outcome when grounding is weak.
INGESTSourcesPDF · Markdown · Notion design
PREPAREParse + chunk420-token target · 64 overlap
Lexical+Semantic
RETRIEVERRF + rerankRelevance · coverage · affinity
DECIDEConfidence gateAnswer or abstain
Reciprocal-rank fusion
RRF combines ranked lists without pretending incomparable raw scores share the same scale.
Evidence-aware reranking
Candidates are reordered using relevance, query coverage, title affinity, and freshness before context assembly.
Grounded citations
Claims stay connected to source excerpts, making support visible to the reader and measurable by the system.
Confidence-gated abstention
Weak evidence produces a transparent refusal instead of a confident answer assembled from irrelevant context.
Full retrieval trace
Query rewrite, candidates, component scores, context tokens, and per-stage latency turn debugging into specific action.
02 / WALKTHROUGH
See the evidence path,
See the evidence path,
not just the final sentence.
The complete project walkthrough is hosted with the portfolio. Press play to follow query rewrite, hybrid retrieval, reranking, citations, traces, and evaluation.
No autoplay. Playback begins only when you choose it.
Download MP4 · 3.8 MB ↓
03 / EVALUATION
The metrics are executable,
The metrics are executable,
and the weakness is visible.
The latest checked-in report runs a deterministic 12-query golden set. It includes in-domain retrieval questions and out-of-domain prompts that should abstain.
Open the raw evaluation report ↗deterministic-demo-v12026-07-28
- Mean reciprocal rank
- 0.833
- Context precision @ 4
- 0.521
- Context recall @ 4
- 0.958
- Abstention accuracy
- 1.000
- Citation coverage
- 1.000
Honest read: recall, citation coverage, and abstention are strong in the current set. Precision@4 at 0.521 exposes useful headroom in candidate filtering and reranking rather than hiding it behind a polished demo.
04 / ENGINEERING BOUNDARY
Reproducible today.
Reproducible today.
Adapter-ready for production.
The hosted demo deliberately avoids paid model keys. That keeps behavior deterministic, repeatable, and open to inspection while preserving clear seams for production infrastructure.
Transparent by design
- Deterministic semantic concept matching
- Public, reproducible corpus and evaluation
- No paid model key required
- Visible ranking and latency traces
- Executable health checks and tests
Replace through interfaces
- Embedding provider and vector database
- Cross-encoder or hosted reranker
- Grounded generation model
- Tenant filters, authorization, and audit
- OpenTelemetry, Langfuse, or Phoenix traces
Reranking must earn latency.
Measure lift against added delay; architecture should serve retrieval quality, not decoration.
Chunk boundaries are product behavior.
Heading preservation and overlap affect whether evidence is understandable after retrieval.
Abstention is a feature.
A bounded “I do not have enough evidence” is safer and more useful than unsupported confidence.
05 / INSPECT THE WORK
Open every layer.
The implementation, architecture, API contract, evaluation design, executable report, live workbench, and walkthrough are public.
BUILT TO BE QUESTIONED