RAG Tutorial

This RAG tutorial teaches Retrieval-Augmented Generation from first principles. RAG is a technique where an application first retrieves relevant information from your own data and then gives it to a large language model (LLM) so the model can generate an answer grounded in that information.

LLMs know a lot, but they do not know your company handbook, last week's tickets or your private PDFs, and they can confidently make things up. RAG fixes both problems without retraining the model.

What is Retrieval-Augmented Generation?

The name describes the three steps. Retrieval: search a knowledge base for passages related to the question. Augmentation: put those passages into the prompt. Generation: the LLM writes an answer using them.

Think of it as an open-book exam. Instead of answering from memory, the model is handed the right pages first.

Why Use RAG?

  • Answers based on private or up-to-date data the model was never trained on.
  • Fewer hallucinations because the model is told to use the given context.
  • Citations: you know which document each answer came from.
  • Cheap to update: add a document to the index, no training needed.
  • Access control: only retrieve documents the user is allowed to see.

A Tiny RAG Example

This toy version uses keyword overlap instead of a real search engine, but it shows the whole idea in a few lines of plain Python:

docs = [
    "Our office is open Monday to Friday, 9 am to 5 pm.",
    "Refunds are processed within 7 business days.",
    "The Wi-Fi password is printed on the kitchen fridge.",
]
question = "How long do refunds take?"

def score(doc):
    return len(set(question.lower().split()) & set(doc.lower().split()))

best = max(docs, key=score)                       # 1. retrieve
prompt = f"Context: {best}\nQuestion: {question}"  # 2. augment
print(prompt)                                     # 3. send to an LLM to generate
Context: Refunds are processed within 7 business days.
Question: How long do refunds take?

A real system replaces the keyword score with embeddings and a vector database, and step 3 sends the prompt to a model such as Llama running in Ollama. You will build exactly that in this tutorial.

What This RAG Tutorial Covers

  • How RAG works and when to use it
  • Embeddings, chunking strategies and similarity search
  • Open-source vector databases: Chroma, pgvector and Qdrant
  • A RAG pipeline from scratch in Python with Ollama and Chroma
  • The same pipeline with LangChain
  • Reranking, hybrid search, query rewriting and citing sources
  • Evaluation, production tips and a chat-with-your-PDFs project

Prerequisites

You need basic Python 3.10+ and a computer that can run Ollama with a small model (8 GB RAM is enough for llama3.2). Everything in this tutorial uses free, open-source tools.

Frequently asked questions

What is RAG in AI?

Retrieval-Augmented Generation is a pattern where relevant documents are retrieved and added to the prompt so an LLM answers based on that information instead of only its training data.

Is RAG better than fine-tuning?

For adding knowledge, usually yes: RAG is cheaper, easier to update and can cite sources. Fine-tuning is better for teaching a style, format or specialised behaviour. They can be combined.

Does RAG eliminate hallucinations?

It reduces them a lot but not completely. Good retrieval, clear prompts and evaluation are still needed.

Do I need LangChain to build RAG?

No. This tutorial builds RAG from scratch first, then shows LangChain as an optional framework.