LangChain Streaming

LangChain streaming shows output while the model is still generating, so users see the first words in a fraction of a second instead of waiting for the full answer. Every Runnable has stream and astream methods.

Streaming from a Chat Model

from langchain_ollama import ChatOllama

llm = ChatOllama(model="llama3.2")
for chunk in llm.stream("Write two sentences about the ocean."):
    print(chunk.content, end="", flush=True)
print()

Sample output (appears word by word):
The ocean covers more than 70 percent of Earth. Its depths remain one of the least explored places on the planet.

Each chunk is an AIMessageChunk. Chunks can be added together with + to build the full message.

Streaming an LCEL Chain

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

chain = ChatPromptTemplate.from_template("Explain {x} in 3 bullet points.") | llm | StrOutputParser()
for piece in chain.stream({"x": "recursion"}):
    print(piece, end="", flush=True)

Sample output:
- A function that calls itself
- Needs a base case to stop
- Useful for trees and divide-and-conquer problems

Streaming Agents

Agents can stream different kinds of events using stream_mode. "updates" yields the result of each step, "messages" yields LLM tokens, and "values" yields the full state after each step.

for token, meta in agent.stream(
    {"messages": [{"role": "user", "content": "What is 5 * 8?"}]},
    stream_mode="messages",
):
    if token.content:
        print(token.content, end="", flush=True)

Sample output:
40.05 * 8 = 40.

Notice that the tool result (40.0) also streams as a message, glued to the answer. Filter by meta["langgraph_node"] == "model" to show only model tokens to users.

Async Streaming in Web Apps

import asyncio

async def main():
    async for piece in chain.astream({"x": "APIs"}):
        print(piece, end="")

asyncio.run(main())

Sample output:
- APIs let programs talk to each other ...

Streaming Tips

• Use astream inside FastAPI with a StreamingResponse or server-sent events.
• Parsers like JsonOutputParser stream partial objects.
• astream_events gives fine-grained events from every step of a chain.

Frequently asked questions

Why is my LangChain stream returning everything at once?

A step in the chain does not support streaming (for example a custom function that needs the whole input). Make sure streaming-friendly components are last, or use astream_events.

How do I stream LangChain responses in FastAPI?

Iterate over chain.astream(...) inside an async generator and return it with StreamingResponse.

What stream_mode should I use for agents?

Use "messages" for token-by-token chat UIs and "updates" to show progress of each agent step.