README

Deutsch |
Español |
français |
日本語 |
한국어 |
Português |
Русский |
中文


LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

A small core, not a bundle. Each backend wraps a best-in-class engine (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX...) in its own image, pulled only when a model needs it. You install nothing you don't use.

- Composable by design: backends are separate and pulled on demand, so you install only what your model needs
- Open and extensible: load any model, or build your own backend in any language against an open interface
- Drop-in API compatibility: OpenAI, Anthropic, and ElevenLabs APIs across every backend
- Any model, any modality: LLMs, vision, voice, image, and video behind one API
- Any hardware: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only
- Multi-user ready: API key auth, user quotas, role-based access
- Chats with saved agents: conversations the server keeps, with agents that use tools, MCP, skills and files, triggers, and a Telegram connector
- Add-ons: Studio, Talk, Voice, Face, Simple Office, Sandbox and more install from a signed catalog, and run beside the server
- Privacy-first: your data never leaves your infrastructure

!A small LocalAI core with backends (llama.cpp, vLLM, MLX, whisper.cpp, stable-diffusion, kokoro, parakeet.cpp...) plugged in as separate on-demand images

Created by Ettore Di Giacinto and maintained by the LocalAI team.

 :book: Documentation | :speech_balloon: Discord | 💻 Quickstart | 🖼️ Models | ❓FAQ

 Guided tour

https://github.com/user-attachments/assets/08cbb692-57da-48f7-963d-2e7b43883c18




Click to see more!


 User and auth

https://github.com/user-attachments/assets/228fa9ad-81a3-4d43-bfb9-31557e14a36c

 Usage metrics per user

https://github.com/user-attachments/assets/cbb03379-23b4-4e3d-bd26-d152f057007f

 Fine-tuning and Quantization

https://github.com/user-attachments/assets/5ba4ace9-d3df-4795-b7d4-b0b404ea71ee

 WebRTC

https://github.com/user-attachments/assets/ed88e34c-fed3-4b83-8a67-4716a9feeb7b



 Quickstart

 macOS


  


 Note: The DMG is not signed by Apple. After installing, run: sudo xattr -d com.apple.quarantine /Applications/LocalAI.app. See 6268 for details.

 Containers (Docker, podman, ...)

 Already ran LocalAI before? Use docker start -i local-ai to restart an existing container.

 CPU only:

bash
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest


 NVIDIA GPU:

bash
 CUDA 13
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13

 CUDA 12
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

 NVIDIA Jetson ARM64 (CUDA 12, for AGX Orin and similar)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64

 NVIDIA Jetson ARM64 (CUDA 13, for DGX Spark)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvi

Top issues

Discussions

Latest release

Patch release v4.12.1. Security fix only.

Security

- Hardened the external URL policy for user-supplied URLs (chat vision image/audio/video URLs, image and audio endpoints). Redirect targets are now validated on every hop, 100.64.0.0/10 and 0.0.0.0/8 are treated as non-public, and the connection goes to the address that was validated. See 12853. Credits to @spicysharkco for the discovery.

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.12.0...v4.12.1