4 postswith this tag
How we built a local application to test the agent with new questions, stream its verifiable process, and store answers and feedback without waiting for the vector index.
While the embedding run advances over 6.7 million chunks, we built the full-corpus FTS5 index (2.7 GiB) and ran the first real-scale BM25 evaluation. The v2 query set produced an MRR of 0.170; after fixing fake titles and ambiguous queries, v3 reached 0.366 and the partial hybrid smoke test 0.402. Along the way: a COUNT(*) that lies, 32 documents almost invisible to the index, and a token-pruning step that turned 21 hours of queries into 34 minutes without changing the metrics.
We built the production foundations on the 657,867 documents of the Official Journal of the Federation (DOF): the full compressed corpus occupies 3.52 GiB, the chunk index holds 6.73 million 91-byte recipes, and the binary-vector pilot confirms the vector index will fit in less than 1 GiB. Along the way: a GROUP BY that consumed 35 GB of disk, a document that became 21 GB of chunks, and a lesson in result parity.
Fourth installment of the benchmark: we built the compressed corpus on 10,000 real documents. sqlite-zstd compresses 10.9x with random access, chunks are stored as 110-byte recipes instead of text, the word-search index ended up 15x smaller than feared, and TurboQuant quantization ties full-vector quality. Everything fits in ~11 GB.
