README

Stable Cascade

    


This is the official codebase for Stable Cascade. We provide training & inference scripts, as well as a variety of different models you can use.

This model is built upon the Würstchen architecture and its main 
difference to other models, like Stable Diffusion, is that it is working at a much smaller latent space. Why is this 
important? The smaller the latent space, the faster you can run inference and the cheaper the training becomes. 
How small is the latent space? Stable Diffusion uses a compression factor of 8, resulting in a 1024x1024 image being 
encoded to 128x128. Stable Cascade achieves a compression factor of 42, meaning that it is possible to encode a 
1024x1024 image to 24x24, while maintaining crisp reconstructions. The text-conditional model is then trained in the 
highly compressed latent space. Previous versions of this architecture, achieved a 16x cost reduction over Stable 
Diffusion 1.5.  
Therefore, this kind of model is well suited for usages where efficiency is important. Furthermore, all known extensions
like finetuning, LoRA, ControlNet, IP-Adapter, LCM etc. are possible with this method as well. A few of those are
already provided (finetuning, ControlNet, LoRA) in the training and inference sections.

Moreover, Stable Cascade achieves impressive results, both visually and evaluation wise. According to our evaluation, 
Stable Cascade performs best in both prompt alignment and aesthetic quality in almost all comparisons. The above picture
shows the results from a human evaluation using a mix of parti-prompts (link) and aesthetic prompts. Specifically, 
Stable Cascade (30 inference steps) was compared against Playground v2 (50 inference steps), SDXL (50 inference steps), 
SDXL Turbo (1 inference step) and Würstchen v2 (30 inference steps).


    


Stable Cascade´s focus on efficiency is evidenced through its architecture and a higher compressed latent space. 
Despite the largest model containing 1.4 billion parameters more than Stable Diffusion XL, it still features faster 
inference times, as can be seen in the figure below.


    




    


 Model Overview
Stable Cascade consists of three models: Stage A, Stage B and Stage C, representing a cascade for generating images,
hence the name "Stable Cascade".
Stage A & B are used to compress images, similarly to what the job of the VAE is in Stable Diffusion. 
However, as mentioned before, with this setup a much higher compression of images can be achieved. Furthermore, Stage C 
is responsible for generating the small 24 x 24 latents given a text prompt. The following picture shows this visually.
Note that Stage A is a VAE and both Stage B & C are diffusion models.


    


For this release, we are providing two checkpoints for Stage C, two for Stage B and one for Stage A. Stage C comes with 
a 1 billion and 3.6 billion parameter version, but we highly recommend using the 3.6 billion version, as most work was 
put into its finetuning. The two versions