VibeVoice TTS Server

Local OpenAI-compatible text-to-speech API server powered by VibeVoice-7B. Generates up to 45 minutes of speech in a single request, with voice cloning from reference audio.

Features

• OpenAI-compatible API — drop-in replacement for POST /v1/audio/speech
• Voice cloning — upload reference audio to clone any speaker's voice
• Long-form generation — natively supports up to ~45 minutes per request (full podcast episodes)
• On-demand model loading — loads the 7B model on first request, auto-unloads after idle timeout to free VRAM
• Multiple formats — MP3, WAV, OPUS, FLAC, AAC, PCM
• MCP server — use as a Claude Code tool for speech synthesis
• Platform detection — CUDA > MPS > CPU with optional 4-bit quantization

Quickstart

1. Install

pip install -e .

# With CUDA flash attention:
pip install -e ".[cuda]"

# With 4-bit quantization:
pip install -e ".[quant]"

Requires ffmpeg for MP3/OPUS/AAC encoding:

# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt install ffmpeg

2. Start the server

vibevoice-tts-server

The model downloads on first request (~14 GB) and loads into VRAM/RAM. Subsequent requests reuse the loaded model.

Options:

vibevoice-tts-server --device cuda --port 8100 --idle-timeout 600

3. Generate speech

curl -X POST http://localhost:8101/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello world, this is VibeVoice!", "voice": "alloy"}' \
  --output hello.mp3

4. Voice cloning with reference audio

Upload a reference audio file to clone a speaker's voice:

curl -X POST http://localhost:8101/v1/audio/speech/upload \
  -F "input=Welcome to the show, I'm your host." \
  -F "voice=alloy" \
  -F "response_format=wav" \
  -F "reference_audio=@speaker_sample.wav" \
  --output cloned.wav

5. Multi-speaker generation (one-shot)

VibeVoice natively generates multi-speaker audio in a single pass. Format the input with Speaker N: prefixes and upload one reference audio file per speaker:

curl -X POST http://localhost:8101/v1/audio/speech/upload \
  -F "input=Speaker 1: Welcome to the show, I'm your host.
Speaker 2: Thanks for having me, great to be here.
Speaker 1: Let's dive right in." \
  -F "response_format=wav" \
  -F "reference_audio=@host_voice.wav" \
  -F "reference_audio=@guest_voice.wav" \
  --output podcast.wav

Or via the JSON endpoint with file paths:

curl -X POST http://localhost:8101/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Speaker 1: Welcome to the show.\nSpeaker 2: Thanks for having me.",
    "instructions": "{\"reference_audio\": [\"/path/to/host.wav\", \"/path/to/guest.wav\"]}"
  }' \
  --output podcast.mp3

Single-speaker voice cloning also works with a single file path:

curl -X POST http://localhost:8101/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Welcome to the show.",
    "voice": "alloy",
    "instructions": "{\"reference_audio\": \"/path/to/speaker_sample.wav\"}"
  }' \
  --output cloned.mp3

API Reference

POST /v1/audio/speech

JSON body (OpenAI-compatible):

Instructions JSON fields:

Returns raw audio bytes with the appropriate Content-Type header.

POST /v1/audio/speech/upload

Multipart form — same fields as above, plus:

GET /v1/audio/voices

List available voice presets.

GET /v1/models

List available models.

GET /health

Server status, model load state, device info.

Voice Presets

Configuration

All settings can be set via environment variables with the VIBEVOICE_TTS_ prefix:

VIBEVOICE_TTS_HOST=0.0.0.0
VIBEVOICE_TTS_PORT=8101
VIBEVOICE_TTS_MODEL_ID=vibevoice/VibeVoice-7B
VIBEVOICE_TTS_DEVICE=auto          # auto, cuda, mps, cpu
VIBEVOICE_TTS_DTYPE=auto           # auto, bfloat16, float32
VIBEVOICE_TTS_IDLE_TIMEOUT=300     # seconds before unloading model (0 = never)
VIBEVOICE_TTS_MAX_NEW_TOKENS=0     # 0 = unlimited; tokens are audio frames at 7.5 Hz
VIBEVOICE_TTS_CFG_SCALE=1.3
VIBEVOICE_TTS_N_DIFFUSION_STEPS=10
VIBEVOICE_TTS_QUANTIZE_4BIT=false

MCP Server

Use as a Claude Code tool:

vibevoice-tts-mcp

Add to your Claude Code MCP config:

{
  "mcpServers": {
    "vibevoice-tts": {
      "command": "vibevoice-tts-mcp",
      "args": ["--device", "auto"]
    }
  }
}

Tools: synthesize_speech, list_voices, get_tts_status

Running Tests

pip install -e ".[test]"
pytest tests/ -v

Audio Token Math

VibeVoice generates audio tokens at 7.5 Hz (7.5 tokens per second of audio). The 7B model has a 32K context window shared between text input tokens and audio output tokens.

By default, max_new_tokens=0 (unlimited), allowing the model to generate until it finishes the input text naturally.

Licensing

This server code is released under the MIT License.

Model license note: Microsoft released VibeVoice-7B under the MIT License. However, Microsoft's model card states the model is "limited to research purpose use" and later removed the TTS code from their official repository citing misuse concerns. Community forks and model weights remain available under MIT. Users should review the model card and applicable terms before deploying in production.

This project is an independent wrapper and is not affiliated with or endorsed by Microsoft.