pipx install voxa-dub
voxa talk.mp4 --target_lang ru        # → talk_dubbed_ru.mp4

Overview

Most dubbing tools concatenate synthesized clips one after another. A clip that runs slightly long pushes the next one later, the error accumulates, and three minutes in the dub is a sentence behind the speaker.

Voxa places every clip anchored to the source timeline. The slot for a line runs from its own onset to the next line's onset. A clip that over-runs is trimmed to fit; a short one is padded with silence. The cursor is assigned, never accumulated — so drift is structurally impossible.

Everything else follows from that decision: the translator is given a character budget so the line fits its slot at a natural pace, speech is only ever sped up (never slowed down, which sounds dragged), and segment starts are tightened to the first word's timestamp so the dub does not begin before the speaker does.

voxa talk.mp4 --target_lang ru
# → talk_dubbed_ru.mp4 + subtitles_ru.srt

No API key is required. The defaults work out of the box.

Features

[!NOTE] Voxa ships no model weights and vendors no third-party code. It orchestrates tools you install yourself. See NOTICE.md before using it commercially.

Supported Languages

Which engine for which language?

There is no universal answer, and the cloud engine does not always win — a voice that genuinely covers a language usually beats a larger model that only approximates it. Which is why Voxa ships the measurement rather than a verdict.

--quality-gate transcribes every synthesized clip back with a second ASR model and scores it by word error rate, so you can check the engines against your content:

voxa clip.mp4 --target_lang az --tts edge --quality-gate --gate-model base

docs/BENCHMARK.md has a reproducible script, a worked run across six languages, and — just as important — the limits of what a number like this can tell you.

[!TIP] If a language has a dedicated native neural voice in Edge, try it first. For a low-resource language use --gate-model base or larger: a tiny gate model misreads a perfectly good voice and reports a worse score than it deserves.

[!IMPORTANT] A WER figure only means something next to the clip it came from. The same engine and language score very differently on different source material — one proper noun in a short clip can move the average enormously. Treat any published number, including ours, as a starting point for your own measurement rather than a ranking.

Supported Models

Architecture Overview

                         Video
                           │
                           ▼
              ┌─────────────────────────┐
              │   Speech Recognition    │  Whisper / faster-whisper
              │                         │  + word-onset refinement
              │                         │  + non-speech filtering
              └────────────┬────────────┘
                           ▼
              ┌─────────────────────────┐
              │       Translation       │  Google · Ollama · OpenAI · Anthropic · OpenRouter
              │                         │  context-aware, length-budgeted
              └────────────┬────────────┘
                           ▼
              ┌─────────────────────────┐
              │     Text-to-Speech      │  Edge · OpenAI · Piper · XTTS
              │                         │  one shared timeline driver
              └────────────┬────────────┘
                           ▼
              ┌─────────────────────────┐
              │    Audio Processing     │  fit (speed up only)
              │                         │  anchored placement
              │                         │  2-pass loudnorm
              └────────────┬────────────┘
                           ▼
                      Dubbed Video

Every stage is checkpointed in <video>_work/. See docs/ARCHITECTURE.md for why the cursor is assigned rather than accumulated, how the two provider registries work, and where each concern lives in the code.

Demo

One 15-second English clip, dubbed into three languages from a single transcription. Every dub below was produced with the default engines and no API key:

voxa clip.mp4 --target_lang tr        # then --target_lang az, --target_lang fr

The second and third languages reused the cached transcription, so only translation and speech were regenerated — the timeline is identical across all three.

Installation

1. FFmpeg — Voxa checks for it at startup and tells you if it's missing.

sudo apt install ffmpeg      # Debian / Ubuntu
sudo dnf install ffmpeg      # Fedora / RHEL
brew install ffmpeg          # macOS
winget install Gyan.FFmpeg   # Windows

2. Voxa — Python 3.9 or newer.

pipx install voxa-dub          # recommended: isolated, puts `voxa` on your PATH

Or run it once without installing anything:

uvx voxa-dub talk.mp4 --target_lang ru

git clone https://github.com/akshinmrv/Voxa
cd Voxa
python3 -m venv venv && source venv/bin/activate   # Windows: venv\Scripts\activate

# The CPU-only torch wheel keeps the install considerably smaller.
# Have an NVIDIA GPU? Install the CUDA wheel instead — see "4. GPU acceleration" below.
pip install torch --index-url https://download.pytorch.org/whl/cpu

pip install .

[!NOTE] The distribution is named voxa-dub (the voxa name on PyPI belongs to an unrelated, abandoned package). The command it installs is voxa — that is what every example below uses. Running the script directly, python voxa.py …, is equivalent.

docker run --rm -v "$PWD:/data" ghcr.io/akshinmrv/voxa talk.mp4 --target_lang ru

docker run --rm -v "$PWD:/data" -v voxa-cache:/cache \
    ghcr.io/akshinmrv/voxa talk.mp4 --target_lang ru

3. Optional engines — install only what you use.

[!NOTE] Multiple speakers (--diarize) detects who speaks when and gives each speaker a distinct voice, so an interview or podcast isn't dubbed in one voice. It needs a Hugging Face token (--hf-token or HF_TOKEN) and a one-time acceptance of the model terms at hf.co/pyannote/speaker-diarization-3.1 — the model is MIT-licensed and free to ship, only gated. Pass --num-speakers N if you know the count. Edge and OpenAI TTS get per-speaker voices; Piper and XTTS stay single-voice for now, and --diarize isn't combinable with --subtitles (diarization needs the source audio).

[!NOTE] Dubbing from a URL (voxa "https://…" --target_lang de) downloads the video with yt-dlp and then runs the normal pipeline. Downloading may breach the source platform's Terms of Service — you are responsible for having the right to the content; the CLI prints this before it fetches. Local files behave exactly as before.

[!WARNING] voxa-dub[xtts] installs coqui-tts, the maintained community fork. The XTTS-v2 model weights are non-commercial (CPML), and Coqui Inc. no longer exists to sell a commercial licence. For commercial voice cloning, drive an MIT-licensed engine through --openai-tts-base-url. If import TTS fails after install, pip picked a transformers coqui-tts can't use — pin a compatible one, e.g. pip install "transformers==4.53.0".

4. GPU acceleration — optional, NVIDIA only.

Voxa detects CUDA on its own; there is no flag to turn it on. What it speeds up:

Install the CUDA build of PyTorch instead of the CPU wheel from step 2:

# Match the CUDA version your driver supports — https://pytorch.org/get-started/locally/
pip install torch --index-url https://download.pytorch.org/whl/cu124

Check it took:

python -c "import torch; print(torch.cuda.is_available())"   # → True

[!NOTE] A brand-new GPU can outrun its PyTorch build: torch.cuda.is_available() returns True, yet transcription fails with "no kernel image is available for execution on the device" — the installed torch simply has no compiled kernels for that architecture yet. Install a newer CUDA build (a higher cuXXX), or sidestep torch altogether with --whisper-backend faster (voxa-dub[faster], CTranslate2): it transcribes without the PyTorch/CUDA version matching, and is also 2–4× faster on CPU.

When the GPU is in use, the run log prints ⚙️ Using device: CUDA at the transcription step (it prints CPU otherwise). Rough VRAM per Whisper model: tiny/base ≈ 1 GB, small ≈ 2 GB, medium ≈ 5 GB, turbo ≈ 6 GB, large ≈ 10 GB; XTTS wants roughly 4 GB more.

[!NOTE] --whisper-backend faster (CTranslate2) runs on the GPU too, but needs NVIDIA's cuBLAS and cuDNN libraries: pip install nvidia-cublas-cu12 nvidia-cudnn-cu12. Voxa checks for CUDA only — Apple Silicon (MPS) and AMD GPUs fall back to CPU.

Configuration

API keys. Copy .env.example to .env (gitignored) and fill it in. Voxa loads it on startup; real environment variables always win. Prefer this over --openai_api_key, which lands in your shell history and the process list.

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
OPENROUTER_API_KEY=sk-or-...

Defaults. Put common options in a JSON file. Keys are the long option names with dashes as underscores; explicit flags override the file. A complete example lives in examples/config.json.

voxa video.mp4 --config examples/config.json

Logging. --log-format json emits one JSON object per line. --verbose raises the level to DEBUG and also surfaces third-party libraries.

Cost estimates. The per-job summary reports usage and — when a price is known — an estimated cost. For an LLM translator, a couple of low-cost defaults are built in; set prices for the models you use with --llm-prices (best kept in the config file), since list prices change. For OpenAI TTS, --tts-price (USD per 1M characters) turns the reported character count into a cost.

{ "llm_prices": { "gpt-5": [1.25, 10], "claude-opus-4-8": [15, 75] }, "tts_price": 12.0 }

All options: voxa --help. It is generated from the code, so unlike a README table it can never go stale.

Quick Start

# Simplest possible run — no API key, no configuration
voxa video.mp4 --target_lang ru

# Natural, context-aware translation with an LLM
export OPENAI_API_KEY="sk-..."
voxa video.mp4 --target_lang de --translator openai

# Hundreds of models (DeepSeek, Gemini, Llama…) through one OpenRouter key — no extra to install
export OPENROUTER_API_KEY="sk-or-..."
voxa video.mp4 --target_lang az --translator openrouter --openrouter_model deepseek/deepseek-chat

# Clone the original speaker's voice
voxa video.mp4 --target_lang tr --tts xtts

# Fully offline: local LLM translation, offline speech
voxa video.mp4 --target_lang fr --translator ollama --tts piper

# See what a run would do — engines, output path, cached steps — without running it
voxa video.mp4 --target_lang ru --dry-run

# Subtitles only, no synthesis
voxa video.mp4 --target_lang es --subtitles-only

# Dub from an existing SRT — skip Whisper (hand-corrected captions, or subtitles you already have)
voxa video.mp4 --target_lang de --subtitles captions.srt

# Give every speaker their own voice (interviews, podcasts) — needs a Hugging Face token
voxa video.mp4 --target_lang de --diarize --num-speakers 2 --hf-token hf_...

# Several videos in one command
voxa a.mp4 b.mp4 c.mp4 --target_lang ru

# Score the result instead of guessing
voxa video.mp4 --target_lang az --quality-gate --gate-model base

# Self-hosted speech: any OpenAI-compatible server, no API key
voxa video.mp4 --target_lang tr --tts openai \
     --openai-tts-base-url http://localhost:8004/v1

[!TIP] A wordier target language (Turkish or Russian for an English source) reads most naturally with an LLM translator (--translator openai / anthropic / openrouter): each line gets a length budget so it fits its slot at a natural pace. Lines also breathe into the pause that follows instead of being sped up. google translates literally, so a long line can still be compressed.

Optional local console (voxa serve)

The CLI is the product. If you'd rather click than type, voxa serve starts a local console that shells out to the very same voxa command — so the UI cannot drift away from the engine, because it is the engine. It binds to 127.0.0.1; nothing is uploaded anywhere.

_{Upload a video, pick engines, watch the seven-step pipeline live, download the dub.}

# Backend: REST + SSE, drives the same pipeline per job
pipx install "voxa-dub[serve]"
voxa serve                              # http://127.0.0.1:8000

# Frontend (separate terminal)
cd web && npm install && npm run dev    # http://localhost:3000  →  /en/app

It also carries the settings the CLI reads: API keys (kept in your local .env, never echoed back), per-provider models with a connection test, translation style, and speech style presets.

The repository also contains the public landing site in web/ — a trilingual (EN/AZ/TR) showcase, deployable as a static site. See web/README.md for development, environment variables, and deployment.

Project Structure

Voxa/
├── voxa.py                     # The entire tool: pipeline, engines, registries, CLI
├── voxa_server.py              # `voxa serve` operator backend (FastAPI; optional [serve] extra)
├── web/                        # Web frontend: public landing + local operator app
├── pyproject.toml              # Packaging, extras, ruff and pytest configuration
├── requirements.txt            # Core dependencies (optional engines live in extras)
│
├── tests/
│   ├── test_voxa.py            # Unit tests: parsing, timing maths, scoring, adapters
│   ├── test_golden.py          # Golden harness: the same functions composed
│   └── golden/                 # Recorded inputs and expected outputs
│
├── docs/
│   ├── ARCHITECTURE.md         # Design decisions and where each concern lives
│   ├── RELEASING.md            # Release checklist
│   └── assets/                 # Demo videos and images
│
├── examples/
│   └── config.json             # A complete, verified configuration file
│
├── .github/
│   ├── workflows/ci.yml        # ruff + pytest on Python 3.9–3.12
│   ├── workflows/release.yml   # A tag becomes a release only if it passes
│   ├── ISSUE_TEMPLATE/         # Bug report and feature request forms
│   └── dependabot.yml
│
├── README.md
├── CHANGELOG.md
├── CONTRIBUTING.md
├── SECURITY.md
├── CODE_OF_CONDUCT.md
├── NOTICE.md                   # Third-party licences and commercial-use guidance
└── LICENSE                     # MIT

Roadmap

Direction, not a promise. Anything requiring hardware or a paid key waits until it can be tested honestly.

Have a use case that isn't covered? Open an issue.

FAQ

voxa video.mp4 --tts openai --openai-tts-base-url http://localhost:8004/v1

Contributing

Issues and pull requests are welcome, and a first pull request gets a response within 24 hours — even if that response is a request for changes. A good first issue is the easiest place to start. CONTRIBUTING.md covers the test suite, how to re-record the golden files, and the two dependency rules that matter: no GPL-licensed required dependency, and engine-specific packages belong in extras.

Adding a speech engine is one adapter plus one registry line — the timeline, placement, drift tracking and scoring are already handled for you.

pip install -e ".[dev]"
ruff check .
pytest

License

Voxa is released under the MIT License.

Voxa ships no model weights and vendors no third-party code, but the engines it drives have their own licences — some stricter than Voxa's. Read NOTICE.md before commercial use.

Acknowledgements

Voxa stands on work done by others.

Author

Voxa is built and maintained by Akshin Miranov under the Servoogle name.

Servoogle exists to build practical AI tooling — software that solves a real problem end to end rather than demonstrating a model — and to release the parts that other people can build on. Voxa is one of those parts. Dubbing a video should not require a studio, a licence negotiation, or a proprietary pipeline; it should require one command and a machine you already own.

That is why Voxa is MIT, why it carries no copyleft dependency, why its licence obligations are documented rather than glossed over, and why every engine sits behind a registry that anyone can extend. The interesting problems in this space — keeping a dub in sync, making a synthetic voice sound unhurried, knowing whether the output is actually good — are worth solving in the open.

If Voxa is useful to you, the most valuable thing you can send back is a bug report with the log attached.