README

bitnet.cpp


!version









📰 News

07/23/2026: 📣 We released VibeASR.cpp — a real-time multilingual ASR inference engine on CPU using BitNet I2_S quantization, achieving RTF Code] [Models] [Report] !NEW

07/20/2026: 📣 We released BitNet-embedding-0.6B and BitNet-embedding-270M on Hugging Face — the first 1-bit embedding models that deliver competitive embedding quality with significantly faster inference on CPUs.
- 1.42x to 2.28x speedup over F16 on BitNet-embedding-0.6B prefill (8 threads)
- 1.32x to 1.74x speedup over F16 on BitNet-embedding-270M prefill (8 threads)
- Supports I2_S conversion with optimized kernels on x86 CPUs
- Lossless inference with 2 bits per weight

07/16/2026: 📣 Released BitNet Embeddings 0.6B/270M: I2_S Conversion and Inference Optimization — detailed guide for converting and running BitNet embedding models with optimized I2_S kernels.

01/15/2026: 📣 Released BitNet CPU Inference Optimization — parallel kernel implementations with configurable tiling and embedding quantization support, achieving 1.15x to 2.1x additional speedup over the original implementation.

05/20/2025: 📣 Released BitNet Official GPU inference kernel — extending 1-bit inference beyond CPUs.

04/14/2025: 📣 Released BitNet Official 2B Parameter Model on Hugging Face — the first official BitNet b1.58 model trained with 4T tokens.

02/18/2025: 📑 Bitnet.cpp: Efficient Edge Inference for Ternary LLMs — system-level paper on bitnet.cpp's architecture and design.

11/08/2024: 📑 BitNet a4.8: 4-bit Activations for 1-bit LLMs — enabling 4-bit activations for further efficiency gains.

10/21/2024: 📑 1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs — the technical report behind bitnet.cpp.

10/17/2024: 📣 bitnet.cpp 1.0 released.

03/21/2024: 📑 The-Era-of-1-bit-LLMs: Training Tips, Code, FAQ

02/27/2024: 📑 The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — the foundational paper introducing BitNet b1.58.

10/17/2023: 📑 BitNet: Scaling 1-bit Transformers for Large Language Models — the original BitNet paper.



 Overview

bitnet.cpp is the official inference framework for 1-bit LLMs (e.g., BitNet b1.58). It offers a suite of optimized kernels that support fast and lossless inference of 1.58-bit models on CPU and GPU (NPU support coming next).

Try it out via this online demo, or build and run it on your own CPU or GPU.

bitnet.cpp achieves speedups of 1.37x to 5.07x on ARM CPUs, with larger models experiencing greater performance gains. Additionally, it reduces energy consumption by 55.4% to 70.0%, further boosting overall efficiency. On x86 CPUs, speedups range from 2.37x to 6.17x with energy reductions between 71.9% to 82.2%. Furthermore, bitnet.cpp can run a 100B BitNet b1.58 model on a single CPU, achieving speeds comparable to human reading (5-7 tokens per second), significantly enhancing the potential for running LLMs on local devices. Please refer to the technical report for more details.



 Model Releases

 1.

Top issues

Discussions