README

--------------------------------------------------------------------------------


Blog |
Documentation |
Quick Start |
Cookbook |
SGLang |
Join Slack



⭐ Star SGLang-Omni to help more builders discover open infrastructure for multimodal and speech serving!


 News

- [2026/11] 🎙️ Voxt 2.0 redefines open-source voice interaction on Mac: voice input, any-to-any translation, and a local AI assistant, with a 92% shorter wait after long dictation than 1.x. \[Download\] \[Changelog\]
- [2026/11] 🔁 OpenRSI × SGLang-Omni: recursive self-improvement (RSI) scaled to production AI infrastructure, with 14 merged PRs across 8 speech and omni models and +7.6% to +326.5% throughput on H100. \[Blog\]
- [2026/10] 🗺️ A unified runtime for interactive intelligence: native full-duplex models, turn-based multimodal models, and cascaded systems, coordinated through composable graphs and persistent sessions. \[Blog\]
- [2026/10] 🚀 SGLang-Omni v0.2.0 is on PyPI. Install with uv pip install --prerelease=allow "sglang-omni==0.2.0". \[Installation\]
- [2026/10] 🐟 Day-0 support for Fish Audio Drama 3: multi-speaker text-to-speech with voice cloning and natural-language control, streamed on /v1/audio/speech. \[Blog\] \[Cookbook\] \[Technical Report\]
- [2026/10] 🎙️ Introducing Voxt, an open-source voice input tool for Mac, now integrating SGLang-Omni for faster local speech recognition on Apple Silicon. Combine ASR with LLM-based text refinement and personal system prompts for accurate technical terminology and personalized voice input, with fully local or remote inference options. [Download][[Demo]](https://x.com/GenAI_is_real/status/2109419016205017181)


Earlier news

- [2026/09] 🐧 Day-0 support for AuK and AuK-Flash: text + voice instructions → 24 kHz speech on /v1/audio/speech, audio + editing instructions → edited speech on /generate. \[Cookbook\]
- [2026/08] 🎵 Day-0 support for MiniMax Music 3: lyrics + caption → 32 kHz stereo song on /v1/audio/speech. \[Cookbook\]
- [2026/08] 🚀 TTS architecture refactor: shared pipeline state, engine construction, reference encoding, capability metadata, and vocoder scheduling. \[Roadmap\] \[Blog\]
- [2026/06] 🔥 MOSS-TTS Local Transformer v1.5 on SGLang-Omni with native-streaming 48 kHz speech. \[Blog\] \[Cookbook\]
- [2026/06] 🔥 Higgs Audio v3 TTS for real-time, controllable speech. \[Blog\] \[Cookbook\]



 About

SGLang-Omni is a multi-stage serving runtime for omni, speech, and TTS models. Its design target is multi-stage decoding: generation split across heterogeneous stages with different compute patterns, dependency structures, and resource needs. SGLang-Omni owns the pipeline topology, stage lifecycle, inter-stage transport, model-family integration layer, and OpenAI-compatible serving surface, while composing with SGLang for high-performance autoregressive scheduling and model execution where applicable.

- Multi-stage runtime: SGLang-Omni models generation as coordinated stages: preprocessing, encoders, autoregressive eng

Top issues

Discussions

Latest release

Requirements

- A Mac with Apple silicon and macOS 15 or later.

 Install

- Open Voxt-2.0.3-arm64.dmg and drag Voxt to Applications. It replaces Voxt 1.x there.
- This version does not update itself: install a newer one the same way.


 Changed

- Plainer text across the app: about 300 English strings stop chaining clauses with colons and semicolons and leave out details nobody needs, about 360 Chinese strings read as written in Chinese and name each thing one way, and about 50 help texts that only said what the page already shows are gone. In Chinese, Rewrite is now 改写 (转写 also m