README

TextGen

A desktop app for local LLMs. Open source, no telemetry. Text, vision, tool-calling, web search. UI + API.





 Get started in 1 minute

Download, unzip, double-click textgen. A window opens.

https://github.com/oobabooga/textgen/releases

Portable builds for Linux, Windows, and macOS with CUDA, Vulkan, ROCm, and CPU-only options. All dependencies included. Compatible with GGUF (llama.cpp) models.

For additional backends (ExLlamaV3, Transformers), training, image generation, and extensions, see Installation.

 Features

 Chat & generation

- instruct mode for instruction-following (like ChatGPT), and chat-instruct/chat modes for talking to custom characters. Prompts are automatically formatted with Jinja2 templates.
- Vision (multimodal): Attach images to messages for visual understanding (tutorial).
- File attachments: Upload text files, PDF documents, and .docx documents to talk about their contents.
- Edit messages, navigate between message versions, and branch conversations at any point.
- Notebook tab for free-form text generation outside of chat turns.

 Backends & API

- Multiple backends: llama.cpp, ik_llama.cpp, Transformers, ExLlamaV3, and TensorRT-LLM. Switch between backends and models without restarting.
- OpenAI/Anthropic-compatible API: Chat, Completions, and Messages endpoints with tool-calling support. Use as a local drop-in replacement for the OpenAI/Anthropic APIs (examples).
- Tool-calling: Models can call custom functions during chat, including web search, page fetching, and math. Each tool is a single .py file. MCP servers are also supported (tutorial).

 Training & image generation

- Training: Fine-tune LoRAs on multi-turn chat or raw text datasets. Supports resuming interrupted runs (tutorial).
- Image generation: A dedicated tab for diffusers models like Z-Image-Turbo. Features 4-bit/8-bit quantization and a persistent gallery with image metadata (tutorial).

 Privacy & interface

- 100% offline and private, with zero telemetry, external resources, or remote update requests.
- Dark/light themes, syntax highlighting for code blocks, and LaTeX rendering for mathematical expressions.
- Built-in and community extensions including TTS, voice input, and translation. See the extensions directory for the full list.

 Downloading models

1. Download a GGUF model file from Hugging Face.
2. Place it in the user_data/models folder.

That's it. The UI will detect it automatically.

For recommended GGUF quants, check out LocalBench. To estimate how much memory a model will use, try the GGUF Memory Calculator.


Other model types (Transformers, EXL3)

Models that consist of multiple files (like 16-bit Transformers models and EXL3 models) should be placed in a subfolder inside user_data/models:


textgen
└── user_data
    └── models
        └── Qwen_Qwen3-8B
            ├── config.json
            ├── generation_config.json
            ├── model-00001-of-00004.safetensors
            ├── ...
            ├── tokenizer_config.j

Top issues

Discussions

Latest release

Changes

- MTP speculative decoding support: Add draft-mtp as a new --spec-type option. Auto-enabled when loading MTP GGUFs (e.g. Qwen 3.6 MoE MTP builds).
- Web search improvements:
  - Add snippet support to the web_search tool: results now include a short text excerpt that often answers the query directly, eliminating the need for a follow-up fetch_webpage call (7548).
  - Drop link URLs from fetch_webpage output (links now appear as plain text instead of text markdown), significantly reducing tokens used per page.
  - Prettier rendering of web_search results in the chat, with a spinn