README

Easy, fast, and cheap omni-modality model serving for everyone



| Documentation | DeepWiki | User Forum | Developer Slack | WeChat | Technical Report | Paper | Slides |


---

Latest News 🔥

- [2026/10] We released 0.32.0, aligned with vLLM 0.32.0, featuring optional Ray-based multi-node inference, session-aware full-duplex serving, server VAD for turn-based realtime requests, and expanded image, music, and speech generation with YuE2-3B, Qwen-Image-2.1, Kandinsky6, MOSS-Image, and TaoMate-H3. See the release notes for compatibility changes and installation instructions.
- [2026/10] We released the vLLM-Omni Technical Report, describing a unified serving runtime for text, audio, image, video, and action generation.
- [2026/09] We released 0.30.0, rebased onto vLLM 0.30.0, featuring a unified full-duplex serving framework around engine-owned sessions for MiniCPM-o 4.5 and AURA, native cross-stage KV and multimodal payload transfer (Mooncake AR-to-DiT handoff and NIXL connectors), interactive world-model serving with LingBot World, and realtime MiniMax H3 Turbo inference on NVIDIA Blackwell and Ascend 950.
- [2026/08] We released 0.28.0, featuring production-ready MiniMax H3 serving on GPU and NPU, a unified AR/DiT paged KV cache runtime, and enhanced realtime full-duplex serving for the MiniCPM-o series.
- [2026/08] VeRL-Omni v0.2.0 is released: faster diffusion RL powered by vLLM-Omni (request-level/step-wise batching with FA3), rebuilt Qwen3-Omni multimodal training (DPO & GSPO), plus LTX-2.3, Qwen-Image-Edit support and more. See the release notes.
- [2026/08] We released 0.26.0 - aligned with the vLLM 0.26 release line, featuring MiniMax H3 joint video/audio generation, an experimental full-duplex realtime runtime for MiniCPM-o 4.5, distributed layerwise diffusion offload, and broader model, hardware, streaming, TTS, and quantization support.
- [2026/07] We released 0.24.0 - aligned with the vLLM 0.24 release line, expanding production-ready coverage across TTS, speech, diffusion, image/video generation, and robot-policy serving, with major Omni stage runtime refactoring, diffusion request-level batching, async output materialization, quantization/cache/memory improvements, and broad CUDA/ROCm/XPU/NPU support.
- [2026/06] Stable releases 0.14.0, 0.16.0, 0.18.0, 0.20.0, and 0.22.0 aligned with every even-numbered upstream vLLM minor version, expanding omni and world-model serving, TTS, diffusion, distributed execution, quantization, RL integration through VeRL-Omni, and CUDA/ROCm/MUSA/NPU/XPU support.
- [2026/03] Check out our first public project deepdive at the vLLM Hong Kong Meetup!
- [2025/11] vLLM community officially released vllm-project/vllm-omni in order to support omni-modality models serving.

---

 About

vLLM was originally designed to support large language models for text-based autoregressive generation tasks. vLLM-Omni is a framework that extends its support for omni-modality model inference and serving:

- Omni-modality: Text