README

WLK: Ultra-low-latency, self-hosted speech-to-text pipeline












  









 Powered by Leading Research:

- Simul-Whisper/Streaming (SOTA 2025) - Ultra-low latency transcription using AlignAtt policy. 
- NLLW (2025), based on distilled NLLB (2022, 2024) - Simulatenous translation from & to 200 languages.
- WhisperStreaming (SOTA 2023) - Low latency transcription using LocalAgreement policy
- Streaming Sortformer (SOTA 2025) - Advanced real-time speaker diarization
- Qwen3-ASR-causal (2026) - Causal streaming audio encoder for Qwen3-ASR: each audio block is encoded exactly once, constant compute per audio second, append-only transcripts.
- AlignAtt4LLM (IWSLT 2026) - Simultaneous translation with decoder-only LLMs: attention-gated commits, append-only output.


 Why not just run a simple Whisper model on every audio batch? Whisper is designed for complete utterances, not real-time chunks. Processing small segments loses context, cuts off words mid-syllable, and produces poor transcription. WhisperLiveKit uses state-of-the-art simultaneous speech research for intelligent buffering and incremental processing.


 Architecture



The backend supports multiple concurrent users. Voice Activity Detection reduces overhead when no voice is detected.

 Installation & Quick Start

bash
pip install whisperlivekit


 Quick Start

bash

 Start the server — open http://localhost:8000 and start talking
wlk --model base --language en


 Auto-pull model and start server
wlk run whisper:tiny

 Transcribe a file (no server needed)
wlk transcribe meeting.wav

 Generate subtitles
wlk transcribe --format srt podcast.mp3 -o podcast.srt

 Manage models
wlk models                              See what's installed
wlk pull large-v3                       Download a model
wlk rm large-v3                         Delete a model

 Benchmark speed and accuracy
wlk bench


 API Compatibility

WhisperLiveKit exposes compatibility-oriented subsets of popular APIs:

bash
 OpenAI-compatible REST API
curl http://localhost:8000/v1/audio/transcriptions -F [email protected]

 Works with the OpenAI Python SDK
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")

 Deepgram-compatible WebSocket
 See docs/API.md for supported options and current SDK setup

 Native WebSocket for real-time streaming
ws://localhost:8000/asr


Per-session WebSocket query parameters:

| param | example | effect |
|---|---|---|
| language | ?language=fr | transcription language for this session (one shared engine serves mixed-language sessions) |
| target_language | ?target_language=de | translation target for this session (server must run with --target-language) |
| context | ?context=WhisperLiveKit%2C+Qwen3-ASR | terminology, names, or phrase-list text used to condition this session; supported by Whisper-family and SimulStreaming backends |
| mode | ?mode=diff | incremental snapshot/diff protocol instead of resending the full state (experimental, for integrators building their own client,