README

English | 中文 | 日本語


 [!IMPORTANT]
 :rocket: Gateway 2.0 (Pre-Release) Portkey's core enterprise gateway is merging into open-source with our 2.0 release. You can try the pre-release branch here.
 Read more about what's next for Portkey in our Series A announcement.




🆕 Portkey Models - Open-source LLM pricing for 2,300+ models across 40+ providers. Explore →


 AI Gateway
 Route to 250+ LLMs with 1 fast & friendly API



Docs | Enterprise | Hosted Gateway | Changelog | API Reference








 




The AI Gateway is designed for fast, reliable & secure routing to 1600+ language, vision, audio, and image models. It is a lightweight, open-source, and enterprise-ready solution that allows you to integrate with any language model in under 2 minutes.

- [x] Blazing fast (

 What can you do with the AI Gateway?
- Integrate with any LLM in under 2 minutes - Quickstart
- Prevent downtimes through automatic retries and fallbacks
- Scale AI apps with load balancing and conditional routing
- Protect your AI deployments with guardrails
- Go beyond text with multi-modal capabilities
- Explore agentic workflow integrations
- Manage MCP servers with enterprise auth & observability using MCP Gateway



 [!TIP]
 Starring this repo helps more developers discover the AI Gateway 🙏🏻

 !star-2
 





 Quickstart (2 mins)

 1. Setup your AI Gateway

bash
 Run the gateway locally (needs Node.js and npm)
npx @portkey-ai/gateway

 The Gateway is running on http://localhost:8787/v1
 
 The Gateway Console is running on http://localhost:8787/public/


Deployment guides:
   Portkey Cloud (Recommended)
   Docker
   Node.js
   Cloudflare
   Replit
   Others...



 2. Make your first request


Python Example --
python
 pip install -qU portkey-ai

from portkey_ai import Portkey

 OpenAI compatible client
client = Portkey(
    provider="openai",  or 'anthropic', 'bedrock', 'groq', etc
    Authorization="sk-"  the provider API key
)

 Make a request through your AI Gateway
client.chat.completions.create(
    messages=[{"role": "user", "content": "What's the weather like?"}],
    model="gpt-4o-mini"
)




Supported Libraries:
   JS
   Python
   REST
   OpenAI SDKs
   Langchain
  LlamaIndex
  Autogen
  CrewAI
  More..


On the Gateway Console (http://localhost:8787/public/) you can see all of your local logs in one place.




 3. Routing & Guardrails
Configs in the LLM gateway allow you to create routing rules, add reliability and setup guardrails.
python
config = {
  "retry": {"attempts": 5},

  "output_guardrails": [{
    "default.contains": {"operator": "none", "words": ["Apple"]},
    "deny": True
  }]
}

 Attach the config to the client
client = client.with_options(config=config)

client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Reply randomly with Apple or Bat"}]
)

 This would always response with "Bat" as the guardrail denies all replies containing "A