#bm25

5 postswith this tag

August 13, 2026

From finding documents to finding evidence: five tools for querying the DOF

How we separated document search from passage retrieval and built five deterministic, traceable, and evaluable tools on top of the DOF.

#dof-rag #retrieval #bm25 #tools #evaluation #evidence

Joaquín Bravo ContrerasRead more

August 9, 2026

Eval v4: finding the document is not enough; the evidence must be found too

We built a hand-crafted 42-question evaluation to measure complete lists, effective dates, legal references, multi-document queries, monitoring, and false premises. We ran it against full BM25 and a 38.2% vector index to set a baseline before finishing the embeddings.

#dof-rag #evaluation #retrieval #bm25 #vector-search #hybrid-search #rag

Joaquín Bravo ContrerasRead more

August 6, 2026

The First Real-Scale Evaluation: BM25 Against 657,867 Documents, from 0.170 to 0.366 After Fixing the Set

While the embedding run advances over 6.7 million chunks, we built the full-corpus FTS5 index (2.7 GiB) and ran the first real-scale BM25 evaluation. The v2 query set produced an MRR of 0.170; after fixing fake titles and ambiguous queries, v3 reached 0.366 and the partial hybrid smoke test 0.402. Along the way: a COUNT(*) that lies, 32 documents almost invisible to the index, and a token-pruning step that turned 21 hours of queries into 34 minutes without changing the metrics.

#dof-rag #evaluation #bm25 #fts5 #sqlite #vector-search #embedding

Joaquín Bravo ContrerasRead more

August 2, 2026

Hybrid search: how to combine BM25 and embeddings (and make them fit on a laptop)

Third installment of the benchmark: we merge the BM25 and embedding rankings and the result beats both individually. We also measure how to index the full corpus from the Mac M3: what works (GGUF/Metal), what does not (large batches, fp16), and why jina's binary quantization decides the storage architecture.

#dof-rag #embeddings #benchmark #bm25 #hybrid-retrieval #gguf #apple-silicon

Joaquín Bravo ContrerasRead more

August 1, 2026

Embedding benchmark round 2: BM25, late chunking, and LLM-generated queries

Second round of the benchmark: 499 documents from the full corpus (1999–2026), 3,023 LLM-generated queries across 6 types, a BM25 baseline with FTS5, and evaluation of late chunking for pplx-embed-context. BM25 and embeddings turn out to be complementary by query type.

#dof-rag #embeddings #benchmark #bm25 #rag #hybrid-retrieval

Joaquín Bravo ContrerasRead more

View all tags