README

🗒️ Structured outputs for LLMs 🗒️

Made with ❤👷️ by the team at .txt
Trusted by NVIDIA, Cohere, HuggingFace, vLLM, etc.


[![PyPI Version][pypi-version-badge]][pypi]
[![Downloads][downloads-badge]][pypistats]
[![Stars][stars-badge]][stars]


[![Discord][discord-badge]][discord]
[![Blog][dottxt-blog-badge]][dottxt-blog]
[![Twitter][twitter-badge]][twitter]

The .txt API is currently in early access. Request access here →




 🚀 Building the future of structured generation

We're working with select partners to develop new interfaces to structured generation.

Need XML, FHIR, custom schemas or grammars? Let's talk.

Audit your schema: share one schema, we show you what breaks under generation, the constraints that fix it, and compliance rates before and after. Sign up here.

 Table of Contents

- Why Outlines?
- Quickstart
- Real-World Examples
  - 🙋‍♂️ Customer Support Triage
  - 📦 E-commerce Product Categorization
  - 📊 Parse Event Details with Incomplete Data
  - 🗂️ Categorize Documents into Predefined Types
  - 📅 Schedule a Meeting with Function Calling
  - 📝 Dynamically Generate Prompts with Re-usable Templates
- They Use Outlines
- Model Integrations
- Core Features
- Other Features
- About .txt
- Community



 Why Outlines?

LLMs are powerful but their outputs are unpredictable. Most solutions attempt to fix bad outputs after generation using parsing, regex, or fragile code that breaks easily.

Outlines guarantees structured outputs during generation — directly from any LLM.

- Works with any model - Same code runs across OpenAI, Ollama, vLLM, and more
- Simple integration - Just pass your desired output type: model(prompt, output_type)
- Guaranteed valid structure - No more parsing headaches or broken JSON
- Provider independence - Switch models without changing code


 The Outlines Philosophy



Outlines follows a simple pattern that mirrors Python's own type system. Simply specify the desired output type, and Outlines will ensure your data matches that structure exactly:

- For a yes/no response, use Literal["Yes", "No"]
- For numerical values, use int
- For complex objects, define a structure with a Pydantic model

 Quickstart

Getting started with outlines is simple:

 1. Install outlines

 shell
pip install outlines


 2. Connect to your preferred model

 python
import outlines
from transformers import AutoTokenizer, AutoModelForCausalLM


MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
    AutoTokenizer.from_pretrained(MODEL_NAME)
)


 3. Start with simple structured outputs

 python
from typing import Literal
from pydantic import BaseModel


 Simple classification
sentiment = model(
    "Analyze: 'This product completely changed my life!'",
    Literal["Positive", "Negative", "Neutral"]
)
print(sentiment)   "Positive"

 Extract specific types
temperature = model("What's the boiling point of water in Celsius?", int)
print(tempe