LangChain Chat Models

LangChain chat models are the heart of every LangChain app. A chat model takes a list of messages and returns a new message. LangChain wraps each provider in a class with the same methods, so code written for one model works with another.

Creating a Chat Model

The simplest way is to instantiate the provider class directly. Common parameters are model, temperature (randomness, 0 is most predictable) and, for Ollama, num_ctx (the context window size).

from langchain_ollama import ChatOllama

llm = ChatOllama(model="llama3.2", temperature=0.2, num_ctx=8192)
msg = llm.invoke("Give me one tip for writing clean Python.")
print(msg.content)
print(msg.usage_metadata)
Sample output:
Use descriptive variable names so your code explains itself.
{'input_tokens': 36, 'output_tokens': 12, 'total_tokens': 48}

init_chat_model: One Line to Switch Providers

init_chat_model creates a model from a "provider:model" string. This is handy when the model comes from configuration:

from langchain.chat_models import init_chat_model

llm = init_chat_model("ollama:llama3.2", temperature=0)
# llm = init_chat_model("openai:gpt-5-mini")        # needs langchain-openai
# llm = init_chat_model("anthropic:claude-sonnet-5-5")  # needs langchain-anthropic

print(llm.invoke("What is 12 * 7?").content)
Sample output:
12 * 7 = 84.

The commented lines show the swap. You only change the string and install the matching integration package; prompts, chains and agents keep working.

invoke, batch and stream

  • invoke(input): one request, returns one AIMessage
  • batch([input1, input2, ...]): runs several inputs in parallel, returns a list
  • stream(input): yields chunks as the model generates them (see the streaming lesson)
  • ainvoke, abatch, astream: async versions for use with asyncio

Batching Example

questions = ["Capital of Japan?", "Capital of Canada?", "Capital of Kenya?"]
answers = llm.batch(questions, config={"max_concurrency": 3})
for q, a in zip(questions, answers):
    print(q, "->", a.content)
Sample output:
Capital of Japan? -> Tokyo
Capital of Canada? -> Ottawa
Capital of Kenya? -> Nairobi

Choosing a Local Model

Small models (1B to 4B parameters) are fast but make more mistakes. Mid-size models (7B to 14B) are a good balance on a decent laptop. For tool calling and structured output, pick a model that Ollama lists with the "tools" capability, such as llama3.2, qwen3 or mistral.

Frequently asked questions

What is the difference between an LLM and a chat model in LangChain?

Older "LLM" classes took a plain string and returned a string. Chat models work with lists of messages and roles, which is what modern models use. New code should use chat models.

How do I change the Ollama server address?

Pass base_url to ChatOllama, for example base_url="http://192.168.1.10:11434", or set the OLLAMA_HOST environment variable.

What temperature should I use?

Use 0 to 0.3 for factual answers, extraction and tool calling. Use higher values like 0.7 to 1.0 for creative writing.