{"full_name":"sgl-project/sglang","owner":"sgl-project","name":"sglang","url":"https://github.com/sgl-project/sglang","description":"SGLang is a high-performance serving framework for large language models and multimodal models.","homepage":"https://sglang.io","topics":[],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/147780389?v=4","created_at":"2024-01-08T04:15:52Z","pushed_at":"2026-11-10T09:36:31Z","metrics":{"stars":38643,"forks":10315,"watchers":183,"issues":8279,"prs":40341,"releases":64,"issues_open":null,"issues_closed":null,"prs_open":null,"prs_closed":null,"prs_merged":null,"discussions":423},"growth":{"day":{"stars":38,"forks":26,"base_day":"2026-11-09"}},"history":{"dates":["2026-11-09","2026-11-10"],"stars":[38605,38643],"forks":[10289,10315],"watchers":[183,183],"issues_open":[null,null],"prs_open":[null,null]},"boards":{},"same_owner":[{"full_name":"sgl-project/mini-sglang","name":"mini-sglang","boards":{}}],"issues":[{"number":26340,"title":"CUDA Coredump Tracker","comments":360,"created_at":"2026-05-26T03:03:57Z"},{"number":42752,"title":"[CI] Flaky tests and CI infrastructure failures seen while babysitting PRs","comments":187,"created_at":"2026-10-06T10:42:20Z"},{"number":21302,"title":"[AITER-Upgrade] PR readiness","comments":44,"created_at":"2026-03-24T09:28:38Z"},{"number":33522,"title":"[Roadmap]Fast Engine Recovery: Weight Cache Daemon","comments":35,"created_at":"2026-08-04T09:05:53Z"},{"number":19137,"title":"[Roadmap] Apple Device Support (2026 Q2)","comments":30,"created_at":"2026-02-22T03:21:15Z"},{"number":39499,"title":"[Roadmap] SGLang dLLM Serving","comments":29,"created_at":"2026-09-15T01:21:47Z"},{"number":22511,"title":"[Bug] sglang v0.5.10 2 * H100 deploy glm5 report health check failed. server could not get a response from detokenizer for last 20 seconds","comments":27,"created_at":"2026-04-10T09:26:15Z"},{"number":42012,"title":"[Bug] GLM-5.3-Flash on SM120: fa4 attention backend crashes at CUDA-graph capture (hybrid extend reshape) — triton is the only working backend","comments":26,"created_at":"2026-10-01T06:22:43Z"}],"discussions":[{"number":47933,"title":"Should sglang load lazily like vllm?","comments":0,"category":"General","created_at":"2026-11-02T14:09:12Z"},{"number":47143,"title":"VRAM & KV-Cache Sizing Calculator for High-Throughput Serving & RadixAttention Planning (Zero-Dependency CLI & WebApp)","comments":0,"category":"Show and tell","created_at":"2026-10-28T21:10:00Z"},{"number":46804,"title":"Cookbook: are consumer SM89 (RTX 40-series, 16 GB) notes welcome on the Qwen3 page?","comments":3,"category":"Q&A","created_at":"2026-10-27T09:13:49Z"},{"number":46591,"title":"[Case Study] Agent Context Compaction Breaks RadixAttention Prefix Caching (10.1% -> 99.6% Hit Rate)","comments":0,"category":"Show and tell","created_at":"2026-10-26T12:13:53Z"},{"number":46590,"title":"[Case Study] Agent Context Compaction Breaks RadixAttention Prefix Caching (10.1% -> 99.6% Hit Rate)","comments":0,"category":"Show and tell","created_at":"2026-10-26T12:09:05Z"},{"number":45932,"title":"DeepSeek-V4.1-Flash-1017 on two 16 GB RTX 5070 Ti with the routed experts on the CPU (SGLang + KTransformers, 1M context)","comments":0,"category":"Show and tell","created_at":"2026-10-23T01:32:49Z"},{"number":44633,"title":"Handling KV Cache Corruption During L2-to-L1 Promotion in HiCache","comments":1,"category":"General","created_at":"2026-10-16T02:41:37Z"},{"number":43716,"title":"RadixView: a live radix-tree view of SGLang KV-cache events","comments":0,"category":"Show and tell","created_at":"2026-10-11T17:54:18Z"}],"latest_release":{"name":"v0.5.23","tag":"v0.5.23","published_at":"2026-11-02T22:14:10Z","body":"# Highlights\r\n\r\n*1007 PRs from 257 contributors.*\r\n\r\n## New models\r\n\r\n| Model | Type | Cookbook |\r\n|---|---|---|\r\n| Muse Spark Open | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen) |\r\n| Beam-1.0 | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/Reflection/Beam-1_0) |\r\n| MiniMax-M3.1-Flash | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1) |\r\n| MiniCPM-V 4.7 | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/OpenBMB/MiniCPM-V-4_7) |\r\n| Kev | LLM / VLM | [link](https://docs.sglang.io/cookbook/autoregressive/Kev/kev) |\r\n| Cosmos3-Nano-Sim (Bimanual, Transfer, Auto) | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/Cosmos/Cosmos3) |\r\n| FastH3 Omni | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3) |\r\n| FLUX.3 Image Klein | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/FLUX/FLUX.3) |\r\n| HY-Nomas | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/HY-Nomas/HY-Nomas) |\r\n| MOSS-Image | Diffusion | [link](https://docs.sglang.io/cookbook/diffusion/OpenMOSS/MOSS-Image) |\r\n\r\n## What's new\r\n\r\n* DeepSeek-V4.1 on GB300 gets up to 1.5x more prefill throughput under load and about 28% lower first-token latency on 32K to 64K prompts ([#44403](https://github.com/sgl-project/sglang/pull/44403), [#45059](https://github.com/sgl-project/sglang/pull/45059), [#45107](https://github.com/sgl-project/sglang/pull/45107)).\r\n* SGLang low-latency mode: the opt-in NVIDIA [megakernel](https://github.com/sgl-project/sgl-nvidia-mega) and AMD [monokernel](https://github.com/sgl-project/sgl-amd-mono) plugins run DeepSeek-V4.1-Flash decode as fused persistent kernels, for up to 44% faster single-user decode on GB300 and 75% on MI350X ([#44945](https://github.com/sgl-project/sglang/pull/44945), [#45450](https://github.com/sgl-project/sglang/pull/45450), [#44353](https://github.com/sgl-project/sglang/pull/44353), [#44621](https://github.com/sgl-project/sglang/pull/44621)).\r\n* MoE models such as DeepSeek-V4.1-Flash, GLM-5.3-Flash and gpt-oss reach the first token 30% to 58% sooner on 8K prompts on H200 ([#43363](https://github.com/sgl-project/sglang/pull/43363)).\r\n* Paged Experts (opt-in) runs MoE models larger than GPU memory on a single consumer GPU, up to 30x faster than CPU offload ([#42557](https://github.com/sgl-project/sglang/pull/42557), [#45787](https://github.com/sgl-project/sglang/pull/45787)).\r\n* SGLang now runs on Google TPU7x through [SGLang-TorchTPU](https://github.com/sgl-project/sglang-torchtpu), a PyTorch-native plugin for TPU.\r\n* You can now serve LoRA adapters on quantized MoE models (FP8 and NVFP4) with a new opt-in LoRA engine ([#45892](https://github.com/sgl-project/sglang/pull/45892)).\r\n* DeepSeek-style MLA models running decode context parallelism can now keep their prefix cache in HiCache file-based L3 storage ([#41187](https://github.com/sgl-project/sglang/pull/41187)).\r\n\r\n## To upgrade\r\n\r\n```bash\r\nuv pip install --prerelease=allow sglang==0.5.23\r\n```\r\n\r\n| Platform | Docker |\r\n|---|---|\r\n| NVIDIA (CUDA 13) | `docker pull lmsysorg/sglang:v0.5.23` |\r\n| AMD MI35x | `docker pull lmsysorg/sglang:v0.5.23-rocm10-mi35x` |\r\n| AMD MI30x | `docker pull lmsysorg/sglang:v0.5.23-rocm10-mi30x` |\r\n| Intel GPU | `docker pull lmsysorg/sglang:v0.5.23-xpu` |\r\n| Intel CPU | `docker pull lmsysorg/sglang:v0.5.23-xeon` |\r\n| Google TPU7x (TorchTPU) | `docker pull lmsysorg/sglang:v0.1.0-tpu` |\r\n\r\n# Full Release Notes\r\n\r\n## Speculative Decoding\r\n\r\n* [Feat] speculative: Seed rejection-sampling draft proposals for deterministic inference: [#33395](https://github.com/sgl-project/sglang/pull/33395)\r\n* [Fix] observability: Correct speculative FPM timing and decode lengths: [#38711](https://github.com/sgl-project/sglang/pull/38711)\r\n* [Perf] spec: Let the FlashAttention backend capture the draft-extend CUDA graph (about 1.5x decode throughput at batch size 1 with a single-layer MoE drafter): ["},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-10T09:35:25Z","history_start":"2026-09-06"}