LangChain Project: Document Q&A Agent

This LangChain project combines everything from the tutorial into one app: a command-line assistant that answers questions about your own documents, runs fully locally with Ollama, remembers the conversation, and cites its sources.

Step 1: Index the Documents

from langchain_community.document_loaders import DirectoryLoader, PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_ollama import OllamaEmbeddings
from langchain_chroma import Chroma

docs = DirectoryLoader("docs/", glob="**/*.pdf", loader_cls=PyPDFLoader).load()
chunks = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=120).split_documents(docs)
store = Chroma(collection_name="kb", embedding_function=OllamaEmbeddings(model="nomic-embed-text"),
               persist_directory="./kb_db")
store.add_documents(chunks)
print(f"Indexed {len(chunks)} chunks from {len(docs)} pages")

Sample output:
Indexed 143 chunks from 52 pages

Step 2: A Retrieval Tool

from langchain.tools import tool

@tool
def search_docs(query: str) -> str:
    """Search the company documents. Use for any question about policies or products."""
    hits = store.similarity_search(query, k=4)
    return "\n\n".join(
        f"[{d.metadata.get('source')} p.{d.metadata.get('page', 0) + 1}] {d.page_content}"
        for d in hits
    )

(no output: defines the tool)

Putting source and page into the tool output lets the model cite them. Adding 1 converts the zero-based page number for humans.

Step 3: The Agent with Memory

from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="ollama:qwen3",
    tools=[search_docs],
    system_prompt=(
        "You answer questions about company documents. Always call search_docs first. "
        "Answer only from the results and cite sources like [file p.N]. "
        "If the answer is not there, say you don't know."
    ),
    checkpointer=InMemorySaver(),
)

(no output: creates the agent)

Step 4: A Streaming Chat Loop

cfg = {"configurable": {"thread_id": "cli"}}
while True:
    q = input("\nYou: ")
    if q in {"exit", "quit"}:
        break
    print("Bot: ", end="")
    for token, meta in agent.stream({"messages": [{"role": "user", "content": q}]}, cfg, stream_mode="messages"):
        if meta.get("langgraph_node") == "model" and token.content:
            print(token.content, end="", flush=True)

Sample session:
You: How many leave days do I get?
Bot: Full-time employees get 20 days of paid leave per year [docs/handbook.pdf p.4].
You: Can I carry them over?
Bot: Up to 5 unused days can be carried into the next year [docs/handbook.pdf p.5].

Ideas to Extend the Project

• Turn on LangSmith tracing and build a small evaluation dataset.
• Swap InMemorySaver for SqliteSaver so memory survives restarts.
• Add a second tool, for example a calculator or a web search.
• Wrap the agent in FastAPI or Streamlit for a web UI.
• Try init_chat_model with a hosted provider and compare answer quality.
• Port it to LangChain.js with the same concepts.

Frequently asked questions

What is a good first LangChain project?

A chatbot over your own documents is ideal: it touches loaders, splitters, embeddings, vector stores, tools, agents and memory.

Why use an agent instead of a fixed RAG chain?

An agent can decide whether to search, search several times with different queries, or answer follow-ups from memory. A fixed chain is simpler and faster when every question needs exactly one search.

How do I deploy a LangChain app?

Wrap it in a web framework like FastAPI, use persistent storage for the vector store and checkpointer, and run Ollama or a hosted model behind it. LangGraph Platform is another option for agents.