{"full_name":"lyogavin/airllm","owner":"lyogavin","name":"airllm","url":"https://github.com/lyogavin/airllm","description":"AirLLM 2.8T LLM inference with single 4GB GPU","homepage":"","topics":[],"language":"Jupyter Notebook","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/1113905?u=fd3d25f2d8e597e495d8ffd10d438ad22eb0e94e&v=4","created_at":"2023-06-12T16:28:41Z","pushed_at":"2026-11-10T06:14:31Z","metrics":{"stars":42081,"forks":4335,"watchers":326,"issues":263,"prs":88,"releases":6,"issues_open":null,"issues_closed":null,"prs_open":null,"prs_closed":null,"prs_merged":null,"discussions":27},"growth":{"day":{"stars":48,"forks":7,"base_day":"2026-11-09"},"week":{"stars":478,"forks":65,"base_day":"2026-11-03"}},"history":{"dates":["2026-10-27","2026-10-28","2026-10-29","2026-10-30","2026-10-31","2026-11-01","2026-11-02","2026-11-03","2026-11-04","2026-11-09","2026-11-10"],"stars":[40536,40556,40756,40978,41105,41227,41333,41603,41666,42033,42081],"forks":[4136,4138,4157,4179,4199,4211,4226,4270,4279,4328,4335],"watchers":[319,314,317,318,321,321,322,323,324,326,326],"issues_open":[null,null,null,null,null,null,null,null,null,null,null],"prs_open":[null,null,null,null,null,null,null,null,null,null,null]},"boards":{},"same_owner":[],"issues":[{"number":258,"title":"[Partially solved] What's the problem with these AI toys? Not one that works at fist try","comments":19,"created_at":"2026-02-11T20:56:10Z"},{"number":318,"title":"AMD ROCM support?","comments":1,"created_at":"2026-08-01T12:09:34Z"},{"number":301,"title":"run this with windows amd 9070 XT ROCm 7.12  VERRY SLOW","comments":3,"created_at":"2026-07-06T12:39:10Z"},{"number":330,"title":"3x speedup compression claim is false/outdated","comments":2,"created_at":"2026-08-04T22:18:47Z"},{"number":319,"title":"grok claude","comments":0,"created_at":"2026-08-01T21:54:28Z"},{"number":44,"title":"More clever batching of layers","comments":13,"created_at":"2023-11-30T11:52:03Z"},{"number":192,"title":"it is run","comments":11,"created_at":"2024-10-24T20:25:50Z"},{"number":328,"title":"Login button is not working on Homepage","comments":0,"created_at":"2026-08-04T10:17:52Z"}],"discussions":[{"number":391,"title":"Show and tell: an interactive visualization of the layer-streaming memory/time tradeoff","comments":0,"category":"Show and tell","created_at":"2026-10-27T08:28:18Z"},{"number":390,"title":"artefact english coach","comments":0,"category":"General","created_at":"2026-10-26T14:38:25Z"},{"number":376,"title":"Collab proposal: optional diagnostics + repro bench (no new deps)","comments":1,"category":"Ideas","created_at":"2026-10-11T11:17:33Z"},{"number":361,"title":"An idea regarding the diffusion models","comments":1,"category":"General","created_at":"2026-09-06T13:24:00Z"},{"number":338,"title":"AirWrapperLLM for AirLLM","comments":0,"category":"Show and tell","created_at":"2026-08-08T15:43:25Z"},{"number":320,"title":"Wouldn’t this be very slow ?","comments":3,"category":"Q&A","created_at":"2026-08-02T08:35:10Z"},{"number":314,"title":"[Bug] Qwen3-235B-A22B (BF16 & FP8) inference fails — streaming loader cannot handle fused MoE expert tensors (Qwen3MoeExperts / FP8Experts)","comments":0,"category":"General","created_at":"2026-07-27T14:16:47Z"},{"number":302,"title":"Added an tui to the tool","comments":0,"category":"Ideas","created_at":"2026-07-09T16:05:47Z"}],"latest_release":{"name":"AirLLM v4.0.0 — train Qwen3.8-Flash-Next under 6GB","tag":"v4.0.0","published_at":"2026-09-05T15:43:11Z","body":"Streamed LoRA: frozen weights stay on disk, adapters stay on GPU. **Qwen3.8-Flash-Next** trains under **6GB** (RTX 3060 Ti); **Qwen3.8-27B** trains in **~2GB** at seq 512.\n\n```bash\npip install -U airllm\npip install git+https://github.com/huggingface/transformers.git\n```\n\nPoint `--data` at a JSONL file (`{\"text\": \"...\"}` or `{\"prompt\": \"...\", \"completion\": \"...\"}`):\n\n```bash\npython air_llm/examples/train_qwen38_flash_next_lora.py   --data my_data.jsonl   --seq-len 512   --epochs 1   --save-adapter qwen38-flash-next-lora.pt\n```\n\nFlash-Next still needs in-tree `qwen4_exp` (GitHub `transformers` main today)."},"readme":true,"readme_zh":false,"news":[{"title":"AirLLM 70B inference with single 4GB GPU","url":"https://github.com/lyogavin/airllm","hn":"https://news.ycombinator.com/item?id=49154228","source":"Hacker News","lang":"en","date":"2026-08-03","points":232,"comments":84},{"title":"AirLLM: Inference 2.8T Kimi K3 on a single 4GB GPU","url":"https://github.com/lyogavin/airllm","hn":"https://news.ycombinator.com/item?id=49142538","source":"Hacker News","lang":"en","date":"2026-08-02","points":2,"comments":0},{"title":"AirLLM optimizes inference memory usage","url":"https://github.com/lyogavin/airllm","hn":"https://news.ycombinator.com/item?id=47293735","source":"Hacker News","lang":"en","date":"2026-03-08","points":2,"comments":0},{"title":"AirLLMで4GBのGPUに70Bモデルは動くのか、1トークンにかかる時間を試算した","url":"https://note.com/snake_dragon/n/nb4f1b7ebefeb","source":"note","lang":"zh","date":"2026-10-28"},{"title":"为什么你的 AirLLM 比别人慢三倍？一个 4GB 老显卡玩家的调优踩坑全复盘","url":"https://blog.csdn.net/gitblog_01114/article/details/168541349","source":"blog.csdn","lang":"zh","date":"2026-10-20"},{"title":"刚刚冲上 GitHub 日榜第7：AirLLM 一天狂揽 541 星，三年老项目为何突然翻红？","url":"https://blog.csdn.net/gitblog_00634/article/details/168425887","source":"blog.csdn","lang":"zh","date":"2026-10-19"},{"title":"这周 GitHub 上涨星最猛的 10 个 AI 开源项目","url":"https://www.bilibili.com/video/BV1UAn36aEX3/","source":"bilibili","lang":"zh","date":"2026-10-17"},{"title":"AirLLM: Große KI-Modelle auf schwachen Grafikkarten starten","url":"https://www.heise.de/ratgeber/AirLLM-Grosse-KI-Modelle-auf-schwachen-Grafikkarten-starten-11476015.html?seite=all","source":"heise online","lang":"en","date":"2026-10-06"},{"title":"I want to use a heavy LLM locally, but I only have 12GB of VRAM! For you, there's AirLLM? I investigated whether it's actually usable.","url":"https://note.com/humble_bobcat51/n/na7f9589db317?hl=en","source":"note","lang":"en","date":"2026-02-09"},{"title":"本周最猛的10个 AI 开源项目","url":"https://news.google.com/rss/articles/CBMiVkFVX3lxTE9WSkYwZnZsLWthRVVfV1B4bno1WWtwQWFFR3JQakxmYkhPUHYxNWIyQWpCZF8wMkpnVWJPb3U3dmx4Zk1ZcVRQV2FzbmVXU3dIY1VXTWlB?oc=5","source":"腾讯新闻","lang":"zh","date":"2026-10-18"}],"stale":false,"snapshot_at":"2026-11-10T09:35:25Z","history_start":"2026-09-06"}