Whisper Speech to Text Locally: Python & Node.js Guide

Run Whisper speech to text locally with faster-whisper in Python and whisper.cpp in Node.js: install, transcribe, add timestamps and fix common errors.

Whisper speech to text runs perfectly well on your own machine. OpenAI released the Whisper models and code under the MIT license, and the community has since built faster runtimes around them. You can transcribe meetings, voice notes and support calls without uploading a single file to a cloud API.

I use local Whisper in client projects where audio contains personal data, and for generating subtitles for video content. In this guide I show the two setups I actually use: faster-whisper for Python scripts and services, and whisper.cpp called from Node.js for JavaScript backends. I also cover model sizes, GPU vs CPU, timestamps and subtitles, and the errors you are most likely to hit.

Prerequisites

  • Python 3.9+ for the Python section, Node.js 20+ for the Node section
  • ffmpeg installed and on your PATH (used to decode and convert audio)
  • A C/C++ toolchain and CMake to build whisper.cpp
  • Optional: an NVIDIA GPU with CUDA for much faster transcription
  • Some test audio: an .mp3, .m4a or .wav file

Choosing a Whisper model size

Whisper comes in several sizes. Larger models are more accurate, especially with accents, noise and less common languages, but slower and hungrier for memory. The .en models are English-only and slightly better for English at small sizes. large-v3-turbo is a pruned version of large-v3 that is much faster with only a small loss in accuracy, and it is my default when I have a GPU.

  • tiny / base: very fast, fine for clear English voice notes, weak on noise
  • small: a good balance on CPU
  • medium: more accurate, slow on CPU
  • large-v3: best accuracy, wants a GPU
  • large-v3-turbo: close to large-v3 quality at much higher speed

Option 1: faster-whisper in Python

faster-whisper reimplements Whisper on top of CTranslate2. It is considerably faster than the original openai-whisper package and uses less memory, and it supports int8 quantization on CPU. It downloads the model automatically the first time.

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install faster-whisper

# transcribe.py
import sys
from faster_whisper import WhisperModel

audio = sys.argv[1]

# CPU: device="cpu", compute_type="int8"
# NVIDIA GPU: device="cuda", compute_type="float16"
model = WhisperModel("small", device="cpu", compute_type="int8")

segments, info = model.transcribe(
    audio,
    beam_size=5,
    vad_filter=True,          # skip long silences
    language=None,            # auto-detect, or e.g. "en"
)

print(f"Detected language: {info.language} ({info.language_probability:.2f})")

# segments is a generator: transcription runs while you iterate
for seg in segments:
    print(f"[{seg.start:7.2f} -> {seg.end:7.2f}] {seg.text.strip()}")

# Run it
python transcribe.py meeting.mp3

Generate SRT subtitles in Python

Subtitles are just numbered segments with timestamps in HH:MM:SS,mmm format. This helper writes an .srt file you can load in any video player or upload to YouTube.

# srt.py
import sys
from faster_whisper import WhisperModel

def ts(seconds: float) -> str:
    ms = int(round(seconds * 1000))
    h, ms = divmod(ms, 3_600_000)
    m, ms = divmod(ms, 60_000)
    s, ms = divmod(ms, 1000)
    return f"{h:02}:{m:02}:{s:02},{ms:03}"

audio, out = sys.argv[1], sys.argv[2]
model = WhisperModel("large-v3-turbo", device="cuda", compute_type="float16")
segments, _ = model.transcribe(audio, vad_filter=True)

with open(out, "w", encoding="utf-8") as f:
    for i, seg in enumerate(segments, start=1):
        f.write(f"{i}\n{ts(seg.start)} --> {ts(seg.end)}\n{seg.text.strip()}\n\n")

print("Wrote", out)

Option 2: whisper.cpp from Node.js

For Node.js backends I use whisper.cpp, a C/C++ port of Whisper with no Python dependency. It runs well on CPUs, Apple Silicon and GPUs. I build it once, download a GGML model, and call the whisper-cli binary from Node with child_process. It expects 16 kHz mono WAV input, so I convert with ffmpeg first.

git clone https://github.com/ggml-org/whisper.cpp
cd whisper.cpp
cmake -B build              # add -DGGML_CUDA=1 for NVIDIA GPUs
cmake --build build -j --config Release

# Download a model into ./models
sh ./models/download-ggml-model.sh base.en

# Quick test
ffmpeg -i ../meeting.mp3 -ar 16000 -ac 1 -c:a pcm_s16le ../meeting.wav
./build/bin/whisper-cli -m models/ggml-base.en.bin -f ../meeting.wav

Node.js transcription service

This module converts any input to WAV, runs whisper-cli with JSON output, and returns the text and segments. I use execFile rather than exec, so file names are passed as arguments and never interpreted by a shell.

// whisper.js
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { readFile, rm, mkdtemp } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";

const run = promisify(execFile);
const WHISPER_DIR = process.env.WHISPER_DIR ?? "/opt/whisper.cpp";
const BIN = path.join(WHISPER_DIR, "build/bin/whisper-cli");
const MODEL = path.join(WHISPER_DIR, "models/ggml-base.en.bin");

export async function transcribe(inputPath) {
  const dir = await mkdtemp(path.join(tmpdir(), "stt-"));
  const wav = path.join(dir, "audio.wav");
  const outBase = path.join(dir, "result");

  try {
    await run("ffmpeg", ["-y", "-i", inputPath, "-ar", "16000", "-ac", "1", "-c:a", "pcm_s16le", wav]);

    await run(BIN, ["-m", MODEL, "-f", wav, "-oj", "-of", outBase, "-t", "4"], {
      maxBuffer: 50 * 1024 * 1024,
    });

    const json = JSON.parse(await readFile(outBase + ".json", "utf8"));
    const segments = json.transcription.map((s) => ({
      from: s.timestamps.from,
      to: s.timestamps.to,
      text: s.text.trim(),
    }));

    return { text: segments.map((s) => s.text).join(" "), segments };
  } finally {
    await rm(dir, { recursive: true, force: true });
  }
}

// usage.js
import { transcribe } from "./whisper.js";
const result = await transcribe("./meeting.mp3");
console.log(result.text);

Common errors

  • ffmpeg not found: install it (apt install ffmpeg, brew install ffmpeg or the Windows build) and make sure it is on PATH for the process running your app.
  • whisper.cpp output is garbage or empty: the input was not 16 kHz mono WAV. Always convert with ffmpeg first.
  • Could not load library libcudnn or CUDA errors in faster-whisper: the CUDA and cuDNN versions do not match what CTranslate2 expects. Use CPU with int8 to confirm everything else works, then fix the CUDA libraries.
  • Repeated phrases or hallucinated text in silent parts: enable vad_filter in faster-whisper, trim silence, or try a larger model.
  • Wrong language detected on short clips: pass the language explicitly.
  • Very slow on CPU: use a smaller model, int8, and more threads; or move to a GPU.

Best practices and privacy

  • Process audio in a background queue; long files can take minutes, so never transcribe inside a web request.
  • Delete temporary audio files after processing, as the Node example does in finally.
  • Validate uploads: check file size and type before passing them to ffmpeg, and run workers with limited permissions.
  • Store transcripts with the same care as the audio. They often contain personal data.
  • For speaker labels (who said what), add a diarization tool such as pyannote; Whisper alone does not identify speakers.
  • Keep model files on local disk and pin versions so results stay consistent between deployments.

Conclusion

Local Whisper speech to text is mature, fast and free. faster-whisper is my choice for Python services, and whisper.cpp is the easiest path for Node.js and edge devices. Pick the smallest model that is accurate enough for your audio, process files in the background, and you get private, offline transcription with no per-minute bill.

Frequently asked questions

Can I run Whisper offline?

Yes. After the model file is downloaded once, faster-whisper, openai-whisper and whisper.cpp all run completely offline.

Is Whisper free for commercial use?

The Whisper code and model weights are released under the MIT license, which allows commercial use. Check the licenses of any additional tools you add.

Which is faster, faster-whisper or whisper.cpp?

Both are much faster than the original Python implementation. faster-whisper is usually the better fit on NVIDIA GPUs from Python, while whisper.cpp shines on CPUs, Apple Silicon and when you want no Python dependency.

Which Whisper model should I use?

Start with small on CPU or large-v3-turbo on a GPU. Use .en models for English-only audio at small sizes, and move up a size if accuracy is not good enough.

Can Whisper identify different speakers?

No, Whisper only transcribes. Combine it with a diarization library such as pyannote to label speakers.