README

🤗 Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, 🤗 Diffusers is a modular toolbox that supports both. Our library is designed with a focus on usability over performance, simple over easy, and customizability over abstractions.

🤗 Diffusers offers three core components:

- State-of-the-art diffusion pipelines that can be run in inference with just a few lines of code.
- Interchangeable noise schedulers for different diffusion speeds and output quality.
- Pretrained models that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.

 Installation

We recommend installing 🤗 Diffusers in a virtual environment from PyPI or Conda. For more details about installing PyTorch, please refer to their official documentation.

 PyTorch

With pip (official package):

bash
pip install --upgrade diffusers[torch]


With conda (maintained by the community):

sh
conda install -c conda-forge diffusers


 Apple Silicon (M1/M2) support

Please refer to the How to use Stable Diffusion in Apple Silicon guide.

 Quickstart

Generating outputs is super easy with 🤗 Diffusers. To generate an image from text, use the from_pretrained method to load any pretrained diffusion model (browse the Hub for 30,000+ checkpoints):

python
from diffusers import DiffusionPipeline
import torch

pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16)
pipeline.to("cuda")
pipeline("An image of a squirrel in Picasso style").images[0]


You can also dig into the models and schedulers toolbox to build your own diffusion system:

python
from diffusers import DDPMScheduler, UNet2DModel
from PIL import Image
import torch

scheduler = DDPMScheduler.from_pretrained("google/ddpm-cat-256")
model = UNet2DModel.from_pretrained("google/ddpm-cat-256").to("cuda")
scheduler.set_timesteps(50)

sample_size = model.config.sample_size
noise = torch.randn((1, 3, sample_size, sample_size), device="cuda")
input = noise

for t in scheduler.timesteps:
    with torch.no_grad():
        noisy_residual = model(input, t).sample
        prev_noisy_sample = scheduler.step(noisy_residual, t, input).prev_sample
        input = prev_noisy_sample

image = (input / 2 + 0.5).clamp(0, 1)
image = image.cpu().permute(0, 2, 3, 1).numpy()[0]
image = Image.fromarray((image  255).round().astype("uint8"))
image


Check out the Quickstart to launch your diffusion journey today!

 How to navigate the documentation

| Documentation                                                   | What can I learn?                                                                                                                                                                           |
|-------------------------------------------------------------------

Top issues

Discussions

Latest release

This is a security release. See GHSA-4prv-5hj7-c2xr for details.

This patch release fixes how a component's declared library name is resolved when loading with from_pretrained.

Pipelines and models declare each component as a ["", ""] pair in model_index.json / config.json. This release changes how that library name is resolved across the DiffusionPipeline, AutoModel, and ModularPipeline loading paths, and adds a clearer error when the declared class is not found in the resolved module.

All users on 0.42.0 are encouraged to upgrade:


pip install diffusers==0.42.1