How to Self-Host OpenAI Whisper in 2026: Private Offline Transcription

Self-hosting Whisper can keep transcription on hardware you control, but “local” is not the same as “automatically private.” A dependable setup has four parts: a suitable Whisper implementation, downloaded model files, a repeatable audio pipeline, and a security boundary that covers temporary files, logs, backups, and any local API you expose.

This guide gives you three current paths: OpenAI’s reference Python package, faster-whisper for optimized Python inference, and whisper.cpp for a lightweight C/C++ deployment. The commands and model guidance below were checked against the projects’ official documentation on August 28, 2026.

What self-hosting Whisper does—and does not—mean

Whisper is a speech-recognition model and open-source codebase. When you run an implementation on your own computer, the inference step can occur without sending the recording to a hosted transcription API. After the required packages and model assets are present, transcription can also run without an active internet connection.

That is a useful privacy boundary, not a complete security program. Local inference does not automatically secure temporary files, command history, application logs, exported transcripts, operating-system backups, shared folders, or a service listening on a network port. Model installers may also download weights the first time a model name is requested.

Before calling a workflow “offline,” test it with the network disabled and confirm that:

• the application starts without trying to retrieve a package or model;
• the selected model loads from a known local path;
• input audio and generated text stay in approved directories;
• temporary files are deleted according to your retention policy; and
• no HTTP server is reachable beyond the interface you intended.

Choose the implementation before choosing a model

All three paths run Whisper-family models, but they do not have identical dependencies, command names, model formats, or performance characteristics. Pick the runtime that matches your operating environment instead of copying a command written for another implementation.

Choose a model and size the hardware

OpenAI currently lists six model sizes. Its README gives approximate VRAM needs and relative speed measured on a specific reference GPU. Those figures are useful for screening, not guarantees: actual memory and speed depend on implementation, precision, batch size, beam size, language, audio length, and hardware.

OpenAI’s table describes turbo as roughly eight times the reference speed of large in its test comparison. Do not convert that into an “eight times faster on every computer” promise. Turbo is not trained for translation. If the job is to translate non-English speech into English, OpenAI recommends a multilingual model such as medium or large instead.

For English-only recordings, the .en variants of tiny, base, small, and medium may be worth testing. For production sizing, run the same representative file on two candidate models and record peak memory, wall-clock time, transcription errors, and correction time.

Prepare the system and a clean test file

Create a dedicated project directory and virtual environment so that package versions are visible and reversible. If you use the OpenAI Python implementation with a GPU, select the current PyTorch command for your operating system and CUDA setup from the official PyTorch installer. Avoid copying an old CUDA wheel URL from a tutorial.

OpenAI Whisper requires the FFmpeg command-line tool. Install it through a supported package manager or the official FFmpeg download page, then verify:

ffmpeg -version
python --version

FFmpeg is not a universal mandatory dependency for every Whisper implementation. For example, whisper.cpp can decode supported input through its own example tooling, while optional FFmpeg integration broadens format support. To create a predictable 16 kHz mono 16-bit WAV test file for whisper.cpp, use:

ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le output.wav

Use a short, consented sample containing a name, a number, two speakers, a quiet gap, and one domain-specific term. That single fixture will expose more deployment problems than a perfectly clean demo clip.

Path 1: Install OpenAI Whisper

Open a terminal inside your project directory, create a virtual environment, activate it, and install the current package:

python -m venv .venv

# Windows PowerShell
.\.venv\Scripts\Activate.ps1

# macOS or Linux
source .venv/bin/activate

pip install -U openai-whisper

Run the first transcription with the official CLI:

whisper audio.mp3 --model turbo

The first run may download the selected model. After it completes, check that expected output files were created and read several timestamps against the source audio. For non-English same-language transcription, specify the language if automatic detection is unreliable:

whisper japanese.wav --language Japanese --model turbo

For translation into English, do not use turbo:

whisper japanese.wav --model medium --language Japanese --task translate

A minimal Python call is equally direct:

import whisper

model = whisper.load_model("turbo")
result = model.transcribe("audio.mp3")
print(result["text"])

Path 2: Install faster-whisper

faster-whisper runs Whisper models through CTranslate2. It is useful when you want a Python integration with configurable precision, batching, word timestamps, or built-in VAD filtering.

python -m venv .venv
source .venv/bin/activate  # Windows: .\.venv\Scripts\Activate.ps1
pip install faster-whisper

Start with an explicit device and compute type rather than assuming a GPU path:

from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")
segments, info = model.transcribe(
    "audio.mp3",
    beam_size=5,
    vad_filter=True,
)

print("Detected language:", info.language)
for segment in segments:
    print(f"[{segment.start:.2f} - {segment.end:.2f}] {segment.text}")

The returned segments are generated lazily, so iterate over them to perform the transcription. On an NVIDIA deployment, follow the current faster-whisper repository’s GPU library requirements and verify the installed versions rather than relying on an older CUDA/CUDNN combination from a blog post.

Benchmarks need their conditions. The faster-whisper project currently reports an example where large-v2 with INT8 used 2,926 MB on an RTX 3070 Ti 8GB, with beam size 5 and the repository’s stated software stack. That result demonstrates a tested configuration; it is not a promise that every model, GPU, batch size, or audio file will use the same memory.

Path 3: Build whisper.cpp

The current project lives under ggml-org/whisper.cpp and uses CMake in its documented quick start. Clone the repository, download a converted model, and build the CLI:

git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
sh ./models/download-ggml-model.sh base.en
cmake -B build
cmake --build build -j --config Release

Run a transcription with an explicit model path:

./build/bin/whisper-cli \
  --model models/ggml-base.en.bin \
  --file output.wav

On Windows, the executable may be inside a configuration-specific folder such as build\bin\Release. Use whisper-cli --help from the actual build output to confirm flags for your version.

whisper.cpp also supports quantized GGML models, which can reduce memory and disk use. Quantization can change quality and speed, so compare a quantized candidate against its non-quantized counterpart on your own fixture before standardizing it.

Prepare an air-gapped deployment

An air-gapped system cannot fetch missing Python wheels, GPU libraries, model weights, VAD assets, or container layers. Build a complete manifest on a connected staging machine first.

1. Record operating system, CPU architecture, GPU model, driver version, and Python version.
2. Download the exact packages, model files, optional VAD model, and their checksums.
3. Transfer assets through your organization’s approved process.
4. Install from local paths and configure an explicit local model directory.
5. Disconnect the network before the acceptance test.
6. Transcribe the fixture, confirm outputs, and monitor for failed connection attempts.

Do not assume that a successful online first run proves an offline deployment. It may have filled a user-specific cache that will not exist under a service account or in a new container. Test using the same identity, directories, and launch method that production will use.

Run a local API or Docker service safely

whisper.cpp includes whisper-server, an HTTP server with an OpenAI-like transcription route. Build the project first, then inspect the server’s current options:

./build/bin/whisper-server --help

./build/bin/whisper-server \
  --host 127.0.0.1 \
  --port 8080 \
  --model models/ggml-base.en.bin

Bind the service to localhost unless another machine genuinely needs access. Docker’s documentation warns that published ports can be reachable outside the host when bound broadly. If you intentionally expose transcription on a LAN, add authentication, TLS, request-size limits, timeouts, upload isolation, and a retention policy; a local process is not an access-control layer.

The whisper.cpp project also publishes project Docker images. Persist the model directory so a recreated container does not download assets again:

docker run --rm \
  -v /absolute/path/models:/models \
  -v /absolute/path/audio:/audios \
  ghcr.io/ggml-org/whisper.cpp:main \
  "whisper-cli -m /models/ggml-base.en.bin -f /audios/output.wav"

Pin an image digest or tested release for production. Treat community wrappers as separate projects: verify their repository, API schema, image publisher, license, update cadence, and default network settings instead of presenting them as official OpenAI services.

Use VAD without hiding real speech

Long nonspeech sections can produce unwanted output in automated transcription. Voice activity detection can skip regions classified as silence or nonspeech before decoding, reducing wasted work and some silence-related errors. It cannot guarantee that every hallucination disappears, and aggressive thresholds can remove quiet words.

faster-whisper exposes Silero VAD through vad_filter=True. Start with the project defaults, then test pauses, soft speakers, music, and noisy recordings before changing parameters.

Current whisper.cpp documentation provides a downloadable Silero model and explicit VAD flags:

./models/download-vad-model.sh silero-v6.2.0

./build/bin/whisper-cli \
  --file output.wav \
  --model models/ggml-base.en.bin \
  --vad \
  --vad-model models/ggml-silero-v6.2.0.bin

Review the cut points against the waveform. If the first syllable after a pause disappears, reduce the aggressiveness or add padding around detected speech rather than accepting a cleaner-looking but incomplete transcript.

Validate outputs, timestamps, and quality

A command that exits successfully is not enough. Preserve the raw source audio and review the transcript with a small acceptance checklist:

• Completeness: Are the first and last words present? Were quiet phrases cut?
• Names and numbers: Check people, products, dates, measurements, and amounts.
• Language: Confirm detected language and whether the task was transcribe or translate.
• Timestamps: Seek to several segment boundaries and verify alignment.
• Silence behavior: Inspect long pauses for repeated or invented phrases.
• Reproducibility: Record implementation version, model, precision, flags, hardware, and runtime.

For subtitles, set a line-length policy and inspect the final SRT or VTT in a player. For searchable notes, retain timestamps or segment IDs so a reviewer can return to the audio. For sensitive material, separate the immutable source recording from generated text and apply a defined retention period to both.

If you are building an offline capture-to-transcription workflow, also see how offline AI edge processing works and the comparison of offline AI voice recorders.

Troubleshoot common failures

Frequently asked questions

Can Whisper run completely offline?

Yes, inference can run without an internet connection after the required software and model assets are stored locally. Prove the boundary by testing with the network disabled and checking for missing downloads, temp files, logs, and exposed services.

Which Whisper implementation should I self-host?

Use OpenAI Whisper for the reference Python CLI, faster-whisper for an optimized Python service with quantization or VAD, and whisper.cpp for a native build, quantized GGML models, Apple/CPU deployment, Docker, or its included local server.

Does Whisper require FFmpeg?

OpenAI’s Python implementation requires the FFmpeg command-line tool. That does not make FFmpeg mandatory for every implementation: whisper.cpp has its own supported decoding path and optional FFmpeg integration for broader formats.

How much VRAM does the turbo model need?

OpenAI’s model table lists approximately 6 GB for turbo. Treat this as a planning estimate for the reference implementation; actual use changes with runtime, precision, batch size, hardware, and concurrent workloads.

Can the turbo model translate speech into English?

No. OpenAI states that turbo is not trained for translation. Use a multilingual model such as medium or large when translating non-English speech into English.

Does VAD prevent every Whisper hallucination?

No. VAD can reduce nonspeech sent to the model and may reduce silence-related output, but it can also remove quiet speech if configured too aggressively. Validate thresholds and segment boundaries on representative audio.

Is a local Whisper API private by default?

No. Local processing avoids a hosted transcription API, but the service still needs interface binding, authentication when shared, encrypted transport where appropriate, upload isolation, logging controls, and a retention policy.