Benchmarks
Measured, not asserted.
Every claim on this site traces back to the offline eval harness — labeled queries, human relevance, and per-mode scores you can reproduce.
Results
Offline scores by retrieval mode
Higher is better across all three metrics. Hybrid wins on every column.
| Mode | NDCG@5 | Recall@10 | MRR |
|---|---|---|---|
| Keyword (BM25) | .58 | .66 | .61 |
| Semantic (pgvector) | .71 | .79 | .74 |
| Hybrid (RRF)Best | .78 | .84 | .79 |
Headline numbers
At a glance
The signals that summarize the run.
+34%
NDCG@5 lift, hybrid vs keyword
3
retrieval modes, one query
384-d
bge-small embeddings
<40ms
query-time embed (cached vectors)
Methodology
How the numbers are made
No cherry-picked screenshots — a fixed catalog, labeled queries, and a deterministic scorer.
Labeled query set
A curated set of shopper queries with human relevance judgments over the electronics + shoes catalog.
Offline harness
NDCG@5, Recall@10, and MRR computed per mode — the same numbers that drive the live overlay in the UI.
Honest baseline
BM25 over Elasticsearch is the strawman. A win only counts if it beats a real lexical engine, not a toy one.
Reproducible
17 pytest tests, deterministic scoring, and a fixed catalog so runs are comparable across changes.