README

🚅 LiteLLM
    
    
        LiteLLM AI Gateway
        
        Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.
        
        
        
        
        
        
    
LiteLLM Proxy Server (AI Gateway) |  Hosted Proxy | Enterprise Tier | Website

    
        
    
    
        
    
    
        
    
    
        
    
    
        
    
    
        
    
    
        
    




---

 What is LiteLLM

LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.

Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization.

Jump to LiteLLM Proxy (LLM Gateway) Docs 
Jump to Supported LLM Providers

---

 Why LiteLLM

Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:

- Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling
- Drop-in OpenAI compatibility — swap providers without rewriting your code
- Production-ready gateway — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box
- 8ms P95 latency at 1k RPS (benchmarks)

 OSS Adopters


  
    
    
    
    
    
    Netflix
    
  


---

 Features


LLMs - Call 100+ LLMs (Python SDK + AI Gateway)

All Supported Endpoints - /chat/completions, /responses, /embeddings, /images, /audio, /batches, /rerank, /a2a, /messages and more.

 Python SDK

shell
uv add litellm


An independent litellm-core distribution provides the Python SDK with the same
import litellm API and runtime dependencies. It has no optional extras, CLI entry
points, or bundled dashboard. Install one SDK distribution per environment because
litellm and litellm-core own overlapping Python files. Use litellm for the
proxy, CLI, and optional extras

Build and install core from this checkout while its release integration is pending:

shell
python scripts/build_core_distribution.py --out-dir dist/core
python -m pip install dist/core/litellm_core-.whl


The builder requires Git, uv and the Rust build toolchain. It reads the release version
from the root pyproject.toml, leaves source files unchanged, and produces a wheel
and a self-contained sdist in dist/core. Run installation in a fresh environment
without litellm

python
from litellm import completion
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"

 OpenAI
response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])

 Anthropic  
response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}])


 AI Gateway (Proxy Server)

Getting Started - E2E Tutorial - Setup virtual keys, make your fi

Top issues

Discussions

Latest release

[!TIP]
 The lts image tag always points to the latest LTS release. Pull ghcr.io/berriai/litellm:lts to stay on the newest LTS patch

 Verify this release

bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.108.3


Same command for litellm-database, litellm-non_root and the docker.io/litellm/ images

 What's new in 1.108.3 (LTS)

11 PRs. 3 fixed. No action needed to upgrade

 Changes

- AWS Bedrock: Fixed every request failing on a Bedrock deployment with api_base: "", which answered 5