README

exo: Run frontier AI locally. Maintained by exo labs.


  
  
  




---

exo connects all your devices into an AI cluster. Not only does exo enable running models larger than would fit on a single device, but with day-0 support for RDMA over Thunderbolt, makes models run faster as you add more devices.

 Features

- Automatic Device Discovery: Devices running exo automatically discover each other - no manual configuration.
- RDMA over Thunderbolt: exo ships with day-0 support for RDMA over Thunderbolt 5, enabling 99% reduction in latency between devices.
- Topology-Aware Auto Parallel: exo figures out the best way to split your model across all available devices based on a realtime view of your device topology. It takes into account device resources and network latency/bandwidth between each link.
- Tensor Parallelism: exo supports sharding models, for up to 1.8x speedup on 2 devices and 3.2x speedup on 4 devices.
- MLX Support: exo uses MLX as an inference backend and MLX distributed for distributed communication.
- Multiple API Compatibility: Compatible with OpenAI Chat Completions API, Claude Messages API, OpenAI Responses API, and Ollama API - use your existing tools and clients.
- Custom Model Support: Load custom models from HuggingFace hub to expand the range of available models.

 Dashboard

exo includes a built-in dashboard for managing your cluster and chatting with models.


  

4 × 512GB M3 Ultra Mac Studio running DeepSeek v3.1 (8-bit) and Kimi-K2-Thinking (4-bit)

 Benchmarks


  Qwen3-235B (8-bit) on 4 × M3 Ultra Mac Studio with Tensor Parallel RDMA
  
  
    Source: Jeff Geerling: 15 TB VRAM on Mac Studio – RDMA over Thunderbolt 5
  



  DeepSeek v3.1 671B (8-bit) on 4 × M3 Ultra Mac Studio with Tensor Parallel RDMA
  
  
    Source: Jeff Geerling: 15 TB VRAM on Mac Studio – RDMA over Thunderbolt 5
  



  Kimi K2 Thinking (native 4-bit) on 4 × M3 Ultra Mac Studio with Tensor Parallel RDMA
  
  
    Source: Jeff Geerling: 15 TB VRAM on Mac Studio – RDMA over Thunderbolt 5
  


---

 Quick Start

Devices running exo find each other automatically, without any manual configuration, when they are on the same network and run the same version of exo (see Adding another Mac). Each device provides an API and a dashboard for interacting with your cluster (runs at http://localhost:52415).

You can run exo with the macOS app, the easiest way on a Mac, or from source on macOS or Linux.

 macOS App

exo ships a macOS app that runs in the background, in your Mac's menu bar.



The app needs a Mac with Apple silicon and macOS Sequoia 15.1 or later. RDMA over Thunderbolt needs macOS Tahoe 26.2 or later.

Download the latest version: EXO-latest.dmg. Open it, drag EXO to your Applications folder, and open EXO from there. The first time, macOS asks whether you want to open an app downloaded from the internet: click Open.

You can also install it with Homebrew:

bash
brew install --cask exo


 What EXO asks for, and why

- Local Network access. mac

Top issues

Latest release

EXO v1.0.71 Release Notes

This is a small patch release, with better defaults for sampling and bug fixes for M5 series Macs and RDMA, along with support for the new Kimi K2.6.

 Models
- Add Kimi K2.6, including support for multimodality (1952)
- Add model cards for Qwen3.6 27B, GLM 5.1 and Kimi K2.6 quantisations (1970)


 Performance
- Add recommended sampling defaults (from model providers) to improve model output quality, including min_p and top_k (1947)
- Prevent JACCL all_sum from corrupting output (1952)

 Quality of Life
- Add a Pi integration tab! (1925)
- Handle multi