README

AnythingLLM: The all-in-one AI app you were looking for.
    Chat with your docs, use AI Agents, hyper-configurable, multi-user, & no frustrating setup required.



  
      
   |
  
      
   |
  
      
   |
  
    Docs
   |
   
    Hosted Instance
  



  English · 简体中文 · 日本語



👉 AnythingLLM for desktop (Mac, Windows, & Linux)!  Download Now



📱 AnythingLLM Mobile (Android) is open source! View the repo


Chat with your docs. Automate complex workflows with AI Agents. Hyper-configurable, multi-user ready, battle-tested—and runs locally by default with zero setup friction.

!Chatting


Watch the demo!





 Product Overview

AnythingLLM is the all-in-one AI application that lets you build a private, fully-featured ChatGPT—without compromises. Connect your favorite local or cloud LLM, ingest your documents, and start chatting in minutes. Out of the box you get built-in agents, multi-user support, vector databases, and document pipelines — no extra configuration required.

AnythingLLM supports multiple users as well where you can control the access and experience per user without compromising the security or privacy of the instance or your intellectual property.

 Cool Features of AnythingLLM

- Built-in local LLM engine - Run local models on your CPU or GPU (NVIDIA via CUDA, AMD and Intel via Vulkan) with no extra setup, powered by Lemonade and llama.cpp.
- Dynamic Model Routing - Automatically route chats to the best provider & model for the conversation based on rules you define.
- Automatic & User Managed Memories - Have your LLM remember important information about you or your workspace.
- Scheduled Tasks - Run recurring tasks or prompts on a cron schedule with full agent capabilities.
- Intelligent Skill Selection Enable unlimited tools for your models while reducing token usage by up to 80% per query
- No-code AI Agent builder
- MCP-compatibility
- Multi-modal support (both closed and open-source LLMs!)
- Custom AI Agents
- 👤 Multi-user instance support and permissioning _Docker version only_
- 🎨 Custom color themes & white-labeling - Pick a built-in palette like True Dark or create your own from your brand colors, and set it for everyone on your instance. _Docker version only_
- 🦾 Agents inside your workspace (browse the web, etc)
- 💬 Custom Embeddable Chat widget for your website _Docker version only_
- 📖 Multiple document type support (PDF, TXT, DOCX, etc)
- Intuitive chat UI with drag-and-drop uploads and source citations.
- Production-ready for any cloud deployment.
- Works with all popular closed and open-source LLM providers.
- Built-in optimizations for large document sets—lower costs and faster responses than other chat UIs.
- Full Developer API for custom integrations!
- ...and much more—install in minutes and see for yourself.

 Supported LLMs, Embedder Models, Speech models, and Vector Databases

Large Language Models (LLMs):

- Any open-source llama.cpp compatible model
- OpenAI
- OpenAI (Generic)
- Azure OpenAI
- AWS Bedroc

Top issues

Discussions

Latest release

Built-in LLM for Docker


AnythingLLM Docker now has a default local LLM experience right out of the box. The same kind of built-in engine that ships with AnythingLLM Desktop can now run inside the container: Lemonade, the fully open-source local inference server from AMD, running your models with llama.cpp. You no longer need to run Ollama, LM Studio or any other inference server next to AnythingLLM.

Pass your GPU into the container and the engine is accelerated on whatever hardware you have. It picks the fastest backend automatically: CUDA for NVIDIA GPUs, Vulkan for AMD and Intel GPUs