RAG pipeline, end to end

Ask a question

Beginner

1. You ask: “How many vacation days do I get?”
2. The app asks the translator to turn your question into numbers.
3. Your question is now a list of numbers that captures its meaning.
4. The app looks for passages whose meaning is closest to your question.
5. It finds the paragraphs from the vacation policy.
6. The app gives the writer your question plus those paragraphs: “Answer using only this.”
7. The writer drafts an answer based on the policy.
8. You get the answer, with a link to the policy it came from.

Practitioner

1. The user’s question reaches the RAG app.
2. The app sends the question to the embedding model.
3. The model returns the question’s vector.
4. The app searches the vector store for the nearest chunks.
5. The best-matching chunks come back.
6. The app sends the question and retrieved chunks to the LLM.
7. The LLM returns an answer grounded in the chunks.
8. The user receives the answer and its sources.

Architect

1. Query received; it may be rewritten using chat history first.
2. Query embedded with the same model used at indexing time.
3. Dense query vector returned.
4. Top-k similarity search, with metadata filters such as team or permissions.
5. Top-k chunks returned, optionally reranked for precision.
6. Prompt = instructions + context + question; the context window bounds how much fits.
7. Grounded completion with citations back to source chunks.
8. Response with citations; logged for evaluation of retrieval and answer quality.

Add documents

Beginner

1. A new policy document is added.
2. Each piece is sent to the translator.
3. Every piece now has its own list of numbers.
4. The pieces and their numbers are filed in the catalogue, ready to be found.

Practitioner

1. The app loads a new document and splits it into chunks.
2. Each chunk is sent to the embedding model.
3. The model returns a vector per chunk.
4. Chunks and vectors are stored in the vector store.

Architect

1. Ingestion: parse, chunk (with overlap) and attach metadata.
2. Batch embedding of chunks.
3. Vectors returned.
4. Upsert into the index; new knowledge is searchable without retraining the model.

Nothing relevant

Beginner

1. You ask something the documents don’t cover.
2. The question is turned into numbers as usual.
3. The numbers come back.
4. The app searches the catalogue.
5. Nothing close enough turns up.
6. Instead of guessing, the app says “I don’t know”. That honesty is the point.

Practitioner

1. An out-of-scope question arrives.
2. The question is embedded.
3. The vector is returned.
4. The app searches for matching chunks.
5. No chunk passes the relevance threshold.
6. The app declines rather than letting the model make something up.

Architect

1. Query outside the corpus.
2. Query embedded.
3. Vector returned.
4. Similarity search.
5. All scores fall below the similarity threshold.
6. Refusal path skips generation entirely, which prevents an ungrounded answer.