README

kimi-k3-in-c

A 2.78-trillion-parameter model. One CPU. 8 GB of RAM.

Kimi K3 inference in portable C99.No BLAS. No framework. No GPU.











2.78Tparameters
1.56 TBcheckpoint on disk
8.24 GBpeak RSS, measured
176 KBthe whole engine
0GPUs



The same 2.78-trillion-parameter model, the same answer, on whatever machine you own.More memory only buys speed:



the machine you have
RAM
time per token
what is going on


an ordinary laptop
8 GB
26.5 s
the whole model streams off the disk on every step


a high-end laptop
32 GB
24.2 s
some of the model now sits in memory


a desktop
64 GB
19.8 s
more of it sits in memory


a heavy workstation
128 GB+
5.6 s
the model fits entirely in memory, the disk wait is gone



Same short prompt at every size, and the output is byte-identical from the smallest machine to the largest; only the clock changes. One machine, 124 cores, fast NVMe drive: the first three rows still read the model from disk each step, so a slower drive is slower there, while the 128 GB+ row keeps everything in memory and no longer waits on the disk. On that same machine v1.0.0 made the math per token about 8× lighter, a follow-up question in a chat 3.9× faster, and long prompts about half as costly. (A token is roughly a short word-piece; the two runnable demos below are the original captures on a slower drive, so their clock reads a little higher.) Full data in docs/data/.




  
  I am open to AI research roles and PhD positions. CV.




The macOS, Windows and NEON ports, chat mode, the ultra preset, the faster MXFP4 kernels and a long list of parser fixes came from the people below. Who did what.





























console
$ ./bin/k3 ~/k3model --trunk ~/k3trunk --preset laptop \
           --tok ~/k3model --prompt "The capital of France is" --gen 8 --incremental

--- generated text ---
 Paris.",
+            "The Eiffel
----------------------
8 tokens in 261.5 s, 32.69 s/token average
PEAK RSS for the whole run: 8.24 GB


Slow, and answering correctly, in 8.24 GB, from a checkpoint of 1.56 TB. This particular
batch command deliberately asks for a raw continuation. The official Kimi K3 checkpoint
is also chat-capable; use the XTML REPL below when you want answers and multi-turn history.
Give the same batch request more memory and the answer does not change, only the clock:

console
$ ./bin/k3 ~/k3model --trunk ~/k3trunk --preset server \
           --tok ~/k3model --prompt "def fibonacci(n):" --gen 28 --incremental

--- generated text ---
    if n  [prompt] [memory] [generation] [diagnostics]


 is the directory holding the .safetensors shards. It is required for any
run, but --help, --version and --list-presets work without it:

bash
./bin/k3 --help
./bin/k3 --version
./bin/k3 --list-presets


 Prompt options

Outside --chat, exactly one of these is required. Passing none, or more than one, is a
usage error (exit 2).

| flag | argument | |
|---|---|---|
| --prompt | TEXT | tokenize TEXT and run it. Requires --tok.

Top issues

Latest release

Chat, native Windows builds, an 8 GB class mode for the complete model, NEON kernels for ARM, and a round of hardening on everything that parses checkpoint files. Output is unchanged wherever it was already defined: every oracle gate still matches the PyTorch reference exactly.

One compatibility note: --save-state files are now version 2, with a checksum. A version 1 file is refused rather than read; regenerate it with --gen 0 --incremental --save-state.

 New

- --chat: the official K3 XTML chat format, with JSONL history, --no-think, --thinking-effort, and opt-in sampling through --temperat