README

!Banner image



PrivateGPT is the open-source API layer that turns local models into production AI applications.










---

Running a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks. PrivateGPT provides that layer as an open-source API following the Claude API model — so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.

Production-tested: PrivateGPT powers Zylon, the on-premise AI platform providing Private AI to enterprises across the globe.


text
Your app / agent / workflow / UI
              |
        PrivateGPT API
              |
OpenAI-compatible inference server (Ollama, llama.cpp, vLLM, …)              


 PrivateGPT does not run models itself. It connects to any OpenAI-compatible inference server via OPENAI_API_BASE. If it implements /v1/chat/completions and /v1/models, it works.

PrivateGPT ships a built-in workbench UI for testing and demos, available at /ui. The API is the actual product.

---

 What PrivateGPT gives you

- Standard messages API (streaming, async, token counting)
- File and artifact ingestion
- Retrieval with citations and agentic RAG
- Built-in tools mirroring the Claude API (web search, web fetch, code execution)
- Custom tools and MCP connectors
- Structured access to databases and CSVs
- Embeddings and orchestration

---

 Quickstart

 For Docker, full installation options, and model configuration see the full Quickstart guide.

Prerequisites: You need a running OpenAI-compatible LLM server. Ollama is the easiest starting point.

1. Install PrivateGPT

bash
 macOS
brew tap zylon-ai/tap
brew install private-gpt


bash
 Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

uv tool install --python 3.11 \
  --find-links https://wheels.privategpt.dev/packages/ \
  "private-gpt[core]"


powershell
 Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

uv tool install --python 3.11 
  --find-links https://wheels.privategpt.dev/packages/ 
  "private-gpt[core]"


2. Start your LLM server

bash
 Example with Ollama
ollama pull qwen3.5:35b          LLM (~24 GB)
ollama pull mxbai-embed-large    Embeddings (~670 MB)
ollama serve


3. Run PrivateGPT

bash
 macOS / Linux
OPENAI_API_BASE=http://localhost:/v1 \
  OPENAI_EMBEDDING_API_BASE=http://localhost:/v1 \
  private-gpt serve


powershell
 Windows (PowerShell)
$env:OPENAI_API_BASE = "http://localhost:/v1"
$env:OPENAI_EMBEDDING_API_BASE = "http://localhost:/v1"
private-gpt serve


4. Open the UI

Go to http://localhost:8080/ui. The API is at http://localhost:8080 and follows the Anthropic API spec.



The UI is useful for:

- Sending messages.
- Selecting models from /v1/models.
- Uploading documents.
- Testing retrieval with citations.
- Enabling tools per chat.
- Configuring databases, MCP connectors, skills, and custom tools.
- Inspecting requests and responses through the API De

Top issues

Discussions

Latest release

1.0.1 (2026-06-16)


 Bug Fixes

 add missing docker database deps (2264) (4302bd6)
 ensure that the publisher is ready to publish new messages (2266) (4c0c500)
 make embedding API key optional for local OpenAI-compatible engines (2261) (c120a90), closes 2260
 use PGPT_HOME for local data, caches, and cleanup paths (2267) (4021cf4)


 Performance Improvements

 prompts: use date-level current_date to preserve prompt-prefix caching (2271) (a64f1c9)