The recorded runs need JavaScript to be shown. The raw file is recorded/runs.json.
Applied AI · hybrid RAG · verified citations
Questions about SOPs. Answers you can audit.
This service answers questions over dense procedures and regulatory manuals. It finds the governing passages with vector and keyword search, re-ranks them with a cross-encoder, and lets a language model answer only by quoting the text. Every quote is then checked against the stored document, and the page number comes from the database, not from the model. When the documents do not say it, the answer says so.
Built to be read. Each recorded answer below opens into a step-by-step walkthrough with its own numbers, including a map of where the question landed among the stored chunks and how the four rankings disagreed about them. The rank-fusion lab lets you try the core formula, and every concept links to the code that implements it.
FastAPI · PostgreSQL · pgvector (HNSW) · tsvector (GIN) · Reciprocal Rank Fusion · cross-encoder re-ranking · strict JSON schema
Open the live demo
Code on GitHub
The live demo runs on free hosting and sleeps between visits. Opening it wakes it, which takes about a minute.
Runs at zero cost: free OpenRouter models for generation, local ONNX models for embeddings and re-ranking, serverless Postgres for storage.
Recorded runs
Real answers from the service, saved as it returned them.
- –answerable questions answered with verified citations
- –out-of-corpus questions correctly declined
- –median retrieval time (embed + SQL + re-rank)
- –median end-to-end time on free models
A handful of recorded questions shows that the pipeline works, not how often it is right. The questions were written with the corpus; the answers were not edited.
How a question is answered
Step 1 happens once per document, when it is ingested. Steps 2 to 5 happen for every question; the walkthrough under each recorded run above shows them with real numbers.
- 1
Ingest with context
PDFs are split along headings and paragraphs with a sliding overlap. Each chunk gets a short situational preamble (document, section, pages) before it is embedded and indexed, so a line like “the limit is 10 ppm” stays findable on its own.
- 2
Search two ways at once
One SQL statement runs a pgvector cosine search over an HNSW index and a PostgreSQL full-text search over a GIN index, then fuses both lists with Reciprocal Rank Fusion.
- 3
Re-rank
A cross-encoder reads the question and each candidate together and keeps the three best. This is what lifts the right clause above look-alikes.
- 4
Answer under a schema
The model must return JSON: an answer, a confidence score, and quotes copied character for character, or an explicit “insufficient context”.
- 5
Verify
The server looks for every quote in the stored chunk and drops what does not match. Titles and page numbers are filled from the database. An answer that claims grounding but has no surviving quote is withheld.
Read the rules the language model answers under
Why plain vector search is not enough
Context collapse
A chunk cut out of a manual loses its document, its section and its subject. Embedding it that way makes two near-identical clauses from different SOPs indistinguishable. Prepending context before indexing fixes both retrievers at once.
Keyword blind spots
Embeddings capture meaning and blur identifiers. Compliance questions are full of them: 21 CFR Part 11, ISO 2859-1, SOP-QA-017. Full-text search catches the exact match; rank fusion merges the two lists without calibrating their scores.
Lost in the middle
Handing a model fifty passages degrades its attention to the ones in the middle. Re-ranking down to three passages raises faithfulness and cuts the prompt to a fraction.
Ungrounded answers
Even with the right passage, a model can misquote a number or cite a page it never saw. Here a quote that is not in the stored text is discarded, and the model never supplies a page number at all.
Rank fusion lab
The lab needs JavaScript.
Concepts, one at a time
Each idea in a few sentences, with the file that implements it.
The glossary needs JavaScript.
What it runs on
| Stage | This demo (free) | Drop-in alternative |
|---|---|---|
| Generation | An ordered chain of free OpenRouter models, with fallback when one fails | Claude via the Anthropic API, OpenAI |
| Embeddings | Local ONNX model | OpenAI text-embedding-3-small |
| Re-ranking | Local ONNX cross-encoder | Cohere Rerank v3.5 |
| Storage and search | PostgreSQL with pgvector (HNSW, m=16, ef_construction=64) and tsvector (GIN) | Any PostgreSQL 16+ with pgvector |
| Hosting | Render free web service, Neon serverless Postgres, GitHub Pages | Any container host |
The public demo is deliberately closed in three ways: uploads need the operator's key, questions are rate limited per visitor and capped per day, and the corpus is a set of fictional quality documents written for it.
Read the README How it is deployed Phase 1: Invoice Extraction Service