{"full_name":"FareedKhan-dev/kimi-k3-in-c","owner":"FareedKhan-dev","name":"kimi-k3-in-c","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","description":"A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.","homepage":"https://medium.com/@fareedkhandev/building-kimi-k3-in-c-to-run-a-2-8t-model-on-consumer-hardware-a5792cbf3b59","topics":[],"language":"C","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/63067900?u=09fcee42c39e5a66924e5012f966d75b979b7041&v=4","created_at":"2026-08-01T09:29:38Z","pushed_at":"2026-10-02T04:51:44Z","metrics":{"stars":11528,"forks":1796,"watchers":96,"issues":19,"prs":91,"releases":3,"issues_open":null,"issues_closed":null,"prs_open":null,"prs_closed":null,"prs_merged":null,"discussions":0},"growth":{"day":{"stars":164,"forks":19,"base_day":"2026-11-09"},"week":{"stars":2198,"forks":274,"base_day":"2026-11-03"},"month":{"stars":2648,"forks":353,"base_day":"2026-10-11"}},"history":{"dates":["2026-09-06","2026-09-07","2026-09-08","2026-09-09","2026-09-10","2026-09-11","2026-09-12","2026-09-13","2026-09-14","2026-09-15","2026-09-16","2026-09-17","2026-09-18","2026-09-19","2026-09-20","2026-09-21","2026-09-22","2026-09-23","2026-09-24","2026-09-25","2026-09-26","2026-09-27","2026-09-28","2026-09-29","2026-09-30","2026-10-01","2026-10-02","2026-10-03","2026-10-04","2026-10-05","2026-10-06","2026-10-07","2026-10-08","2026-10-09","2026-10-10","2026-10-11","2026-10-12","2026-10-13","2026-10-14","2026-10-15","2026-10-16","2026-10-17","2026-10-18","2026-10-19","2026-10-20","2026-10-21","2026-10-22","2026-10-23","2026-10-24","2026-10-25","2026-10-26","2026-10-27","2026-10-28","2026-10-29","2026-10-30","2026-10-31","2026-11-01","2026-11-02","2026-11-03","2026-11-04","2026-11-09","2026-11-10"],"stars":[6396,6470,6514,6587,6628,6669,6715,6953,7023,7063,7112,7137,7182,7214,7286,7505,7567,7761,7911,7954,7993,8012,8041,8082,8133,8267,8432,8530,8645,8677,8761,8795,8829,8836,8864,8880,8909,8926,8940,8955,8969,9010,9028,9045,9057,9073,9086,9108,9127,9143,9157,9168,9178,9190,9199,9216,9231,9251,9330,9783,11364,11528],"forks":[1041,1053,1061,1074,1079,1086,1095,1126,1132,1143,1154,1160,1172,1181,1185,1205,1215,1243,1276,1280,1282,1288,1297,1301,1313,1328,1347,1371,1394,1399,1418,1424,1435,1438,1442,1443,1456,1459,1465,1468,1470,1476,1478,1479,1481,1481,1483,1486,1487,1491,1493,1495,1500,1502,1502,1506,1508,1509,1522,1573,1777,1796],"watchers":[62,62,60,61,62,62,64,66,66,66,66,65,67,67,67,67,66,68,69,69,70,71,71,71,71,73,72,72,73,74,75,76,76,75,75,75,76,76,76,76,77,79,80,81,81,81,81,81,81,81,81,82,81,81,81,81,81,81,81,86,96,96],"issues_open":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],"prs_open":[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null]},"boards":{"weekly":55,"monthly":184,"daily_rate":125,"weekly_rate":16,"monthly_rate":77},"same_owner":[{"full_name":"FareedKhan-dev/train-llm-from-scratch","name":"train-llm-from-scratch","boards":{}}],"issues":[{"number":3,"title":"Support for Quantized / Storage-Efficient Kimi K3 Checkpoints","comments":6,"created_at":"2026-08-03T04:50:16Z"},{"number":23,"title":"I have a 5070TI with 16 GB Mem","comments":2,"created_at":"2026-08-10T10:11:35Z"}],"discussions":[],"latest_release":{"name":"v1.1.0","tag":"v1.1.0","published_at":"2026-10-02T03:35:37Z","body":"Chat, native Windows builds, an 8 GB class mode for the complete model, NEON kernels for ARM, and a round of hardening on everything that parses checkpoint files. Output is unchanged wherever it was already defined: every oracle gate still matches the PyTorch reference exactly.\n\n**One compatibility note:** `--save-state` files are now version 2, with a checksum. A version 1 file is refused rather than read; regenerate it with `--gen 0 --incremental --save-state`.\n\n## New\n\n- `--chat`: the official K3 XTML chat format, with JSONL history, `--no-think`, `--thinking-effort`, and opt-in sampling through `--temperature`, `--top-p`, `--top-k` and `--seed`. Greedy by default. Later turns reuse the previous turn's state instead of re-prefilling the transcript, behind a bit-exact gate.\n- `--preset ultra`: the complete 93-layer model in an 8 GB class memory budget, verified with four full runs on a Jetson Orin Nano Super.\n- Native Windows builds through MSYS2 MinGW-w64, with every gate passing, and NEON kernels on arm64.\n- Ctrl-C stops at a safe point, still writes the state file, the results and the reports, and exits 5. A second Ctrl-C kills.\n- `--stop-id`, `--gen 0` to warm a prefix once and resume it many times, `--threads`, and `--trunk-ring`.\n- `download-model.sh <dest> --layers N` fetches only the shards a layer prefix needs, about 7 GB for `N=1`, for trying the pipeline on a disk too small for the checkpoint.\n\n## Faster, output unchanged\n\n- The MXFP4 expert matmul is about 1.6x faster, with another 1.56x from a flat-row AVX2 path.\n- Trunk layers are read in parallel chunks and pinned largest-first; expert reads are split into 1 MiB chunks.\n- On Linux the thread count now defaults to physical cores, which measured about 1.3x faster per token on a 16-core SMT machine.\n\n## Fixed\n\n- `--preset auto` could not start on machines with roughly 100 GB or more available, and printed a negative shortfall.\n- macOS: reads of 2 GiB or more failed, and available memory read as 0, which switched off both memory checks.\n- Hardening across the safetensors reader, trunk.json and the JSON parser: integer overflow, out-of-bounds reads, overlapping or gapped tensor spans, and truncated documents. Checked against the released checkpoint's own files, including all 96 shard headers.\n- A batched prefill path summed uninitialized memory when an expert failed to load.\n- A partial download now names the missing file, using the checkpoint's real file names.\n- State files are checksummed and saved by atomic rename, so a failed save no longer destroys the previous one.\n\n## Verification\n\n`make test`, ctest, both sanitizer jobs and the strict-warnings build are green on Linux, macOS and Windows, and every oracle gate matches the PyTorch reference exactly.\n\nThanks to everyone whose work is in this release: @Avicennasis, @FermiHart, @cablepull, @Deobot2, @BlakeEvans22, @santhoshsathish94, @Irfanwani, @AuricTW, @arafatsolok, @genesisrevelationinc-debug, @TROY665, @openchat-ai, @ysgao, @ShaalanMarwan, @cwwjacobs, @biokraft and @Barba2k2.\n\nFull log in [CHANGELOG.md](https://github.com/FareedKhan-dev/kimi-k3-in-c/blob/v1.1.0/CHANGELOG.md). Diff: [v1.0.0...v1.1.0](https://github.com/FareedKhan-dev/kimi-k3-in-c/compare/v1.0.0...v1.1.0).\n"},"readme":true,"readme_zh":false,"news":[{"title":"Kimi K3 2.78T on One CPU with 8GB RAM","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","hn":"https://news.ycombinator.com/item?id=49181439","source":"Hacker News","lang":"en","date":"2026-08-05","points":4,"comments":0},{"title":"Kimi-k3-in-C: A 2.78T-parameter model. No GPU. 8 GB of RAM","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","hn":"https://news.ycombinator.com/item?id=50357277","source":"Hacker News","lang":"en","date":"2026-11-03","points":2,"comments":1},{"title":"Kimi-k3-in-C, 2.78T-parameter model, one CPU, 8GB RAM","url":"https://github.com/FareedKhan-dev/kimi-k3-in-c","hn":"https://news.ycombinator.com/item?id=49167975","source":"Hacker News","lang":"en","date":"2026-08-04","points":2,"comments":0},{"title":"kimi-k3-in-c","url":"https://www.53ai.com/AIWiki/pQX3kl.html","source":"53ai","lang":"en","date":"2026-10-09"},{"title":"kimi-k3-in-c 量化实测：为什么拒绝量化 Trunk？int8 损失1%、int4 损失17%的完整数据解析","url":"https://blog.csdn.net/gitblog_00359/article/details/155462282","source":"blog.csdn","lang":"zh","date":"2026-09-01"},{"title":"Kimi K3 开放权重全景解读:首个开源 3T 级多模态模型的架构与本地部署","url":"https://blog.csdn.net/Kartist139/article/details/163684634","source":"blog.csdn","lang":"zh","date":"2026-08-12"},{"title":"神了，8G 内存能本地跑 Kimi K3 了。。。","url":"https://news.google.com/rss/articles/CBMiU0FVX3lxTE8yQUtzajlFOXFUd1BhMGVidF90S3h1SGpRbDJXVDQ3Zmp6Ty02eHFVSmtWLU8weHFFNnU3RGpvNEFvRU4wVGlZTk1UOVg3VUpUNWpr?oc=5","source":"电子工程专辑EE Times China","lang":"zh","date":"2026-09-04"},{"title":"8GB 内存也能跑 Kimi K3？2026 本地部署大模型配置全指南","url":"https://news.google.com/rss/articles/CBMiQ0FVX3lxTE8tSzNWcUl5NXN0MHZtNmJ5V213UkdwX3JXS2g1dnJXYUI2QXZBRmljdVU4dS03eUg5ajZ0Y05ERXhGR1E?oc=5","source":"爱范儿","lang":"zh","date":"2026-08-05"},{"title":"Kimi K3: 2.78 Trillion Parameters on One CPU, 8 GB RAM","url":"https://news.google.com/rss/articles/CBMihAFBVV95cUxQc3ZndzlvNE9XRE9ENmp3NUZWalppR0RuSC1naTFTYW04bHV0YVROVE9hWHFJRi1KR3RoT3F0YzJ5eXkzR016a3h4UERheUxubHpoY0Ixd2dST3FfbWdmOEpOdHdBSmtMT3pjWmZLcEUyVHZTTmZ5M2d1N1Nxck5acHVWNHI?oc=5","source":"Pasquale Pillitteri","lang":"en","date":"2026-08-03"},{"title":"Kimi K3：单颗 CPU、8 GB 内存跑通 2.78 万亿参数","url":"https://news.google.com/rss/articles/CBMihgFBVV95cUxONjYtVE5paDJ6NzZ2Z3Y0eXR6UTh6Z05JVlRMdldTaVNzc1ZBUFRMbUFrLVJLR0xaOUhfZ1dTbXJ3TDhnV3BqbHNxN3VhXzBQYzZkYU8wbVJILWF5X2F6aUNRc2MyMVo1eG1idWprdGlqdmZQZXhJX2QtdVJPUmZlaDdfVnlLdw?oc=5","source":"Pasquale Pillitteri","lang":"zh","date":"2026-08-03"}],"stale":false,"snapshot_at":"2026-11-10T09:35:25Z","history_start":"2026-09-06"}