Laravel Ollama Chatbot with Streaming Responses

Build a Laravel Ollama chatbot that streams tokens to the browser: a service class, a streamed response, chat history, rate limiting and Nginx buffering fixes.

A Laravel Ollama chatbot lets you add AI chat to your app without paying per token or sending customer data to an external API. The model runs on your own server, and Laravel talks to it over HTTP like any other service.

The tricky part is streaming. Users expect words to appear as the model writes them, not a 20-second spinner followed by a wall of text. Getting that right in PHP means dealing with Guzzle streams, output buffering, PHP-FPM and Nginx all at once.

In this tutorial I build the exact setup I use: a small OllamaService, a controller that returns a streamed response, a minimal JavaScript client that renders tokens as they arrive, conversation history stored in the session, and the server settings that stop Nginx from buffering everything. It works with Laravel 11 and 12 on PHP 8.2+.

Prerequisites

• A Laravel 11 or 12 app on PHP 8.2 or newer
• Ollama installed and running on the same machine or private network (see my Ollama guide)
• A pulled chat model, for example: ollama pull qwen3:8b
• Nginx + PHP-FPM, or php artisan serve for local testing

Step 1: Configure the Ollama connection

Keep the URL and model in config so you can switch models per environment without touching code.

// config/services.php
'ollama' => [
    'url' => env('OLLAMA_URL', 'http://127.0.0.1:11434'),
    'model' => env('OLLAMA_MODEL', 'qwen3:8b'),
    'timeout' => (int) env('OLLAMA_TIMEOUT', 120),
],

# .env
OLLAMA_URL=http://127.0.0.1:11434
OLLAMA_MODEL=qwen3:8b

Step 2: Create an Ollama service with streaming

Ollama's /api/chat endpoint streams newline-delimited JSON. Each line contains a message.content fragment, and the last line has done set to true. With Laravel's HTTP client I pass the stream option to Guzzle and read the body in chunks, splitting on newlines. A partial line can arrive at the end of a chunk, so I keep a buffer.

<?php
// app/Services/OllamaService.php
namespace App\Services;

use Illuminate\Support\Facades\Http;

class OllamaService
{
    public function streamChat(array $messages, callable $onToken): string
    {
        $response = Http::baseUrl(config('services.ollama.url'))
            ->timeout(config('services.ollama.timeout'))
            ->withOptions(['stream' => true])
            ->post('/api/chat', [
                'model' => config('services.ollama.model'),
                'messages' => $messages,
                'stream' => true,
                'options' => ['temperature' => 0.4, 'num_ctx' => 8192],
            ]);

        $response->throw();

        $body = $response->toPsrResponse()->getBody();
        $buffer = '';
        $full = '';

        while (! $body->eof()) {
            $buffer .= $body->read(1024);

            while (($pos = strpos($buffer, "\n")) !== false) {
                $line = trim(substr($buffer, 0, $pos));
                $buffer = substr($buffer, $pos + 1);

                if ($line === '') {
                    continue;
                }

                $data = json_decode($line, true);

                if (isset($data['error'])) {
                    throw new \RuntimeException($data['error']);
                }

                $token = $data['message']['content'] ?? '';
                if ($token !== '') {
                    $full .= $token;
                    $onToken($token);
                }

                if (! empty($data['done'])) {
                    return $full;
                }
            }
        }

        return $full;
    }
}

Step 3: Return a streamed response from the controller

The controller validates the message, builds the conversation from the session, and returns response()->stream(). Inside the callback I echo each token and flush immediately. I send plain text rather than full Server-Sent Events because fetch with a reader is simpler for POST requests, which EventSource does not support.

Two details matter. First, I save the session before streaming, because a long-running stream otherwise keeps the session locked. Second, the X-Accel-Buffering header tells Nginx not to buffer this response.

<?php
// app/Http/Controllers/ChatController.php
namespace App\Http\Controllers;

use App\Services\OllamaService;
use Illuminate\Http\Request;

class ChatController extends Controller
{
    public function stream(Request $request, OllamaService $ollama)
    {
        $data = $request->validate([
            'message' => ['required', 'string', 'max:4000'],
        ]);

        $history = $request->session()->get('chat', []);
        $history[] = ['role' => 'user', 'content' => $data['message']];

        // Keep the last 12 messages to fit the context window
        $history = array_slice($history, -12);

        $messages = array_merge([
            ['role' => 'system', 'content' => 'You are a helpful support assistant for our app. Be concise.'],
        ], $history);

        $session = $request->session();
        $session->put('chat', $history);
        $session->save();

        return response()->stream(function () use ($ollama, $messages, $history, $session) {
            try {
                $answer = $ollama->streamChat($messages, function (string $token) {
                    echo $token;
                    if (ob_get_level() > 0) {
                        ob_flush();
                    }
                    flush();
                });

                $history[] = ['role' => 'assistant', 'content' => $answer];
                $session->put('chat', $history);
                $session->save();
            } catch (\Throwable $e) {
                report($e);
                echo "\n[The assistant is unavailable, please try again.]";
                flush();
            }
        }, 200, [
            'Content-Type' => 'text/plain; charset=UTF-8',
            'Cache-Control' => 'no-cache',
            'X-Accel-Buffering' => 'no',
        ]);
    }
}

// routes/web.php
use App\Http\Controllers\ChatController;

Route::post('/chat/stream', [ChatController::class, 'stream'])
    ->middleware(['auth', 'throttle:20,1']);

Step 4: Render tokens in the browser

On the front end I use fetch and a ReadableStream reader. This works in a Blade view, Vue or React. The CSRF token comes from the meta tag that Laravel's default layout includes.

<!-- resources/views/chat.blade.php -->
<meta name="csrf-token" content="{{ csrf_token() }}">
<div id="log" style="white-space: pre-wrap"></div>
<form id="chat">
  <input id="msg" autocomplete="off" placeholder="Ask something..." />
  <button>Send</button>
</form>

<script>
const form = document.getElementById('chat');
const log = document.getElementById('log');

form.addEventListener('submit', async (e) => {
  e.preventDefault();
  const input = document.getElementById('msg');
  const text = input.value.trim();
  if (!text) return;
  input.value = '';

  log.textContent += '\nYou: ' + text + '\nBot: ';

  const res = await fetch('/chat/stream', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Accept': 'text/plain',
      'X-CSRF-TOKEN': document.querySelector('meta[name="csrf-token"]').content,
    },
    body: JSON.stringify({ message: text }),
  });

  if (!res.ok) {
    log.textContent += '[Error ' + res.status + ']';
    return;
  }

  const reader = res.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    log.textContent += decoder.decode(value, { stream: true });
  }
});
</script>

Step 5: Make streaming work behind Nginx and PHP-FPM

Locally with php artisan serve everything streams. In production, the response often arrives all at once. That is buffering, and it can happen in three places: PHP output buffering, FastCGI buffering in Nginx, and gzip. The X-Accel-Buffering header handles most of it, but I also set it explicitly for the route.

# /etc/nginx/sites-available/app
location = /chat/stream {
    include snippets/fastcgi-php.conf;
    fastcgi_param SCRIPT_FILENAME $realpath_root/index.php;
    fastcgi_pass unix:/run/php/php8.3-fpm.sock;
    fastcgi_buffering off;
    gzip off;
    fastcgi_read_timeout 300s;
}

; php.ini or the FPM pool
output_buffering = Off

Common errors

• The whole answer appears at once: buffering. Check X-Accel-Buffering, fastcgi_buffering off, gzip off for that route, and output_buffering in php.ini. Cloudflare can also buffer; text/event-stream responses are less likely to be held.
• 419 Page Expired: the X-CSRF-TOKEN header is missing. Make sure the meta tag exists and the header is sent.
• cURL error 28 timeout: the model is loading or too big for the hardware. Raise OLLAMA_TIMEOUT and set OLLAMA_KEEP_ALIVE so the model stays in memory.
• Other requests hang while a chat streams: session locking or too few PHP-FPM workers. Save the session before streaming and raise pm.max_children.
• model not found error: the OLLAMA_MODEL tag is not pulled on the server. Run ollama pull with the exact tag.

Best practices and security

• Keep Ollama bound to 127.0.0.1 or a private network. Laravel is the only thing that should talk to it.
• Rate limit the chat route per user. Each request ties up GPU time and a PHP worker.
• Limit input length and trim history so prompts never exceed the model's context window.
• Never trust the model with permissions: if it suggests actions, validate them in PHP before doing anything.
• Escape output when rendering. I use textContent on the client; if you render Markdown, sanitise the HTML.
• For many concurrent users, consider running streams through a queue and broadcasting tokens with Laravel Reverb instead of holding PHP-FPM workers open.

Conclusion

A streaming Laravel Ollama chatbot needs only a small service, a streamed response and a few server settings. Once the buffering is sorted, it feels as responsive as a hosted AI chat, while every token stays on your infrastructure.

The next step is giving the bot knowledge of your own content with retrieval-augmented generation, which I cover in my RAG chatbot guide.

Frequently asked questions

How do I stream an AI response in Laravel?

Return response()->stream() with a callback that reads the upstream stream, echoes each chunk and calls flush(). Disable buffering in Nginx with the X-Accel-Buffering: no header or fastcgi_buffering off.

Why does my Laravel streamed response arrive all at once?

Something is buffering it: PHP output_buffering, Nginx FastCGI buffering, gzip or a CDN. Disable each for the streaming route and test with curl -N.

Can I use Ollama with Laravel in production?

Yes, as long as the server has enough RAM or VRAM for the model, Ollama is only reachable privately, and you rate limit requests. For high concurrency, a dedicated GPU server is recommended.

Is there a PHP package for Ollama?

Community packages exist, but Laravel's HTTP client is enough. The API is plain JSON over HTTP, and Ollama's OpenAI-compatible endpoint also works with OpenAI PHP clients.