RAG with LangChain

RAG with LangChain rebuilds the pipeline from the previous lesson using LangChain 1.x components. The steps are identical; LangChain just provides ready-made loaders, splitters, vector store wrappers and composable chains, and makes it easy to swap models or databases later.

Install the Packages

pip install -U langchain langchain-ollama langchain-chroma \
    langchain-community langchain-text-splitters pypdf
Successfully installed langchain-1.x langchain-ollama-1.x langchain-chroma-1.x ...

Indexing with LangChain

from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_ollama import OllamaEmbeddings
from langchain_chroma import Chroma

pages = PyPDFLoader("handbook.pdf").load()
chunks = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=120).split_documents(pages)

store = Chroma(
    collection_name="handbook",
    embedding_function=OllamaEmbeddings(model="nomic-embed-text"),
    persist_directory="./lc_db",
)
store.add_documents(chunks)
print(len(pages), "pages,", len(chunks), "chunks")
Sample output:
24 pages, 71 chunks

Each chunk keeps the metadata from its page (source and page), so citations come for free.

Two-Step RAG Chain with LCEL

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_ollama import ChatOllama

retriever = store.as_retriever(search_kwargs={"k": 4})

def format_docs(docs):
    return "\n\n".join(f"[p.{d.metadata['page'] + 1}] {d.page_content}" for d in docs)

prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer only from the context. Cite pages like [p.N]. "
               "If the answer is missing, say you don't know."),
    ("human", "Context:\n{context}\n\nQuestion: {question}"),
])
chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt | ChatOllama(model="llama3.2", temperature=0) | StrOutputParser()
)
print(chain.invoke("How many sick days do employees get?"))
Sample output:
Employees get 10 paid sick days per year [p.6].

Agentic RAG with create_agent

Instead of always retrieving once, you can give the model a retrieval tool and let it decide when and how often to search. This handles follow-ups and multi-part questions better, at the cost of more model calls.

from langchain.agents import create_agent
from langchain.tools import tool

@tool
def search_handbook(query: str) -> str:
    """Search the employee handbook."""
    return format_docs(retriever.invoke(query))

agent = create_agent("ollama:qwen3", tools=[search_handbook],
                     system_prompt="Use search_handbook for policy questions and cite pages.")
out = agent.invoke({"messages": [{"role": "user", "content": "Compare sick leave and annual leave."}]})
print(out["messages"][-1].content)
Sample output:
Employees get 20 days of annual leave [p.4] and 10 paid sick days [p.6]. Unused annual leave can be carried over, but sick days cannot [p.5].

Two-Step Chain vs Agent

  • Two-step chain: one retrieval per question, fast, predictable, works with small models.
  • Agent: can search multiple times or skip search, better for complex questions, slower.
  • Start with the chain; move to an agent when users ask multi-hop questions.

Frequently asked questions

Is RetrievalQA deprecated in LangChain?

Yes. The old chains like RetrievalQA moved to langchain-classic. Use an LCEL chain or create_agent with a retrieval tool instead.

Can I use pgvector or Qdrant with LangChain instead of Chroma?

Yes. Use PGVector from langchain-postgres or QdrantVectorStore from langchain-qdrant; the retriever and chain code stays the same.

What is agentic RAG?

RAG where an agent decides when to retrieve, what to search for and whether to search again, rather than following a fixed retrieve-then-generate path.