Icon

Updated weekly

OW

OW

Icon

Updated weekly

OW

Output Weekly

Output Weekly

Tools, workflows and honest rankings for people who make things with AI

Tools, workflows and honest rankings for people who make things with AI

Images

What is Stable Diffusion? A plain explainer

Ines Cardoso

A soft grainy gradient of dark and light tones

Key takeaways

  • Stable Diffusion is a family of text-to-image models that build a picture in a compressed "latent" space and then decode it into pixels. The first one came out in August 2022.

  • Every release has three parts. A text encoder reads the prompt, a denoising network (a U-Net in older models, a transformer in SD3 and later) does the generating, and a VAE converts between pixels and latents.

  • The lineage runs from SD 1.5 to SDXL (July 2023) to SD3 (2024) and SD 3.5 (October 2024). Stability AI's news page lists no newer Stable Diffusion image model in 2026.

  • SD 3.5 uses the Stability AI Community License, which is free for research and for commercial use under $1M in annual revenue. Older models use CreativeML OpenRAIL licenses.

  • Most people run it through a local interface such as ComfyUI or on a hosted service, and most of the community's styles and characters ship as LoRAs on top of a base checkpoint.

What is Stable Diffusion?

Stable Diffusion is a text-to-image model that makes pictures by starting from noise and removing it step by step, guided by your prompt. It's also the name of a whole family of models from Stability AI and its research partners, so "Stable Diffusion" can mean SD 1.5, SDXL, or SD 3.5 depending on who's talking.

What set it apart in 2022 was that the weights were public. The CompVis repository said the model could run on a GPU with at least 10GB of VRAM, so anyone with a gaming card could download it, change it, and run it at home. That openness is why a large ecosystem of interfaces, fine-tunes, and add-ons grew around it.

How Stable Diffusion works

The idea comes from the latent diffusion paper, by Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Earlier diffusion models worked on raw pixels, which is expensive. The authors ran the diffusion process in the latent space of a pretrained autoencoder instead, which cut the compute cost while keeping detail, and added cross-attention layers so the model could take text as a condition.

That gives three working parts.

  • Text encoder: it turns your prompt into numbers the image network can use. SD 1.5 uses a single CLIP ViT-L/14 encoder. SDXL uses two, OpenCLIP-ViT/G and CLIP-ViT/L. SD3 and SD 3.5 use three, adding T5-xxl to the two CLIP models.

  • Denoising network: it's the part that does the generating. In SD 1.5 it's a U-Net with about 860 million parameters, per the CompVis repository. SD3 and later swapped the U-Net for a Multimodal Diffusion Transformer, or MMDiT.

  • VAE: the variational autoencoder compresses images into latents and expands them back. On SD 1.x it shrinks each side by a factor of 8, so the network works on a much smaller grid than the final picture.

When you press generate, the model draws random noise in latent space, removes some of it at each step while steering toward your prompt, and hands the finished latent to the VAE to decode. The random starting point is the seed, which our seed explainer covers.

The Stable Diffusion family, release by release

The first public release was in August 2022. It was built by researchers at CompVis and Runway with compute from Stability AI, and trained on image and caption pairs from LAION-5B, according to Wikipedia's history of the model.

Model

Released

Notes

SD 1.x

August 2022 (1.5 in October 2022)

U-Net, one CLIP encoder, 512 by 512 training

SD 2.0 and 2.1

November and December 2022

2.0 retrained from scratch on a filtered dataset

SDXL 1.0

July 2023

Two text encoders, a base model plus an optional refiner

SD3 Medium

2024

MMDiT, three text encoders

SD 3.5 Large, Large Turbo, Medium

October 2024

8.1B, distilled 4-step, and 2.5B variants

SD 1.5 is still the version most older LoRAs and tutorials assume. SDXL is where the big community fine-tunes landed, and our Pony vs. Illustrious vs. SDXL breakdown covers two of them. The SDXL model card describes an ensemble-of-experts pipeline in which the base model makes the latents and a separate refiner handles the last denoising steps, though the base model can also run alone.

Where Stable Diffusion stands in November 2026

SD 3.5 is the newest Stable Diffusion image family. Stability AI's announcement lists Large at 8.1B parameters, Large Turbo as a 4-step distilled version, and Medium at 2.5B, which it says needs 9.9GB of VRAM excluding text encoders.

Stability's news page shows its 2026 announcements have gone elsewhere. They include Brand Studio in April, the Stable Audio 3.0 family in May, and a Series B funding round in August. None is a new Stable Diffusion image model. Several sites have reported a "Stable Diffusion 4," but no Stability page backs that up, so treat those claims as unconfirmed.

Licenses for each generation

Licenses changed as the family grew, and they decide whether you can use the output in paid work.

  • SD 1.5 and SDXL: SD 1.5 is under the CreativeML OpenRAIL M license and SDXL under CreativeML Open RAIL++-M. Both are open licenses with use-based restrictions.

  • SD3 Medium and SD 3.5: these fall under the Stability AI Community License. After SD3 Medium launched, Stability's license post said the earlier commercial terms had "caused some confusion and concern in the community," and replaced them in July 2024.

The Community License is free for research, non-commercial use, and commercial use by individuals and organizations under $1M in annual revenue. Above that, Stability's license page says to contact the company for an Enterprise License. The same page says you own the outputs you generate, and it asks researchers who use the models commercially to register with Stability.

Read the full license text before building a business on it. For the separate question of copyright in the pictures themselves, see our piece on whether you can copyright AI-generated images.

How people run Stable Diffusion

There are two routes, local and hosted.

Locally, you download a checkpoint and load it in an interface. ComfyUI is a node-based tool under the GPL-3.0 license, with a desktop app for Windows and macOS. Its README lists support for SD 1.5, SDXL, and SD 3.5, along with FLUX and other families. AUTOMATIC1111's web UI is a Gradio-based interface under the AGPL-3.0 license, and its repository lists thousands of forks. Our ComfyUI beginner's guide walks through a first setup on Windows.

Hosted means someone else's GPU. Stability AI's API documentation describes generating images with Stable Diffusion 3.5 and its Stable Image services, and the SD 3.5 announcement says the models are also offered through platforms such as Replicate. Hosted use trades control and privacy for not needing hardware, and what you can generate then follows that service's own rules.

Checkpoints, LoRAs, and embeddings

A checkpoint is the full set of model weights in one file. Download SDXL base and you have the model Stability trained. A fine-tuned checkpoint is the same architecture retrained on a narrower set of images, which is how community models for anime, photorealism, or a particular style appeared.

A LoRA is much smaller. It follows the method in the LoRA paper, which freezes the pretrained weights and injects trainable low-rank matrices, so a LoRA stores only the changes. It's trained for one base family, so an SD 1.5 LoRA doesn't load on SDXL. Our LoRA explainer covers training and stacking, and an embedding goes smaller still by teaching the text encoder a new word and leaving the image weights alone.

Stable Diffusion next to FLUX and other models

Black Forest Labs, whose founding team says it includes pioneers of Latent Diffusion and Stable Diffusion, now makes the models many people compare Stable Diffusion with. FLUX.1 [dev] is a 12 billion parameter rectified flow transformer, and its model card puts it under a non-commercial license that still allows commercial use of its outputs. BFL's documentation calls FLUX 3 its latest model family, and our FLUX 3 explainer covers the lineup.

The practical split is about ecosystem more than raw quality. Stable Diffusion has the deepest library of fine-tunes and LoRAs, built over four years, so older families keep a following. The FLUX line is newer, so its own library of fine-tunes is younger. Which suits a project depends on the style you need and whether a LoRA for it already exists.

Stable Diffusion FAQ

Is Stable Diffusion free?

The weights are free to download, and the SD 3.5 models are free for research and for commercial use under $1M in annual revenue. Hosted services charge for compute, and companies above the threshold need an Enterprise License from Stability AI.

Who makes Stable Diffusion?

The first release came from researchers at CompVis and Runway, with compute from Stability AI. Stability AI has released the later versions, including SDXL and SD 3.5.

What hardware do I need to run it?

The original CompVis release asked for a GPU with at least 10GB of VRAM. SD 3.5 Medium needs 9.9GB excluding text encoders, according to Stability. Smaller or optimized builds can run on less, and hosted services need no GPU at all.

What's the difference between SD 1.5, SDXL, and SD 3.5?

SD 1.5 uses a U-Net and one text encoder. SDXL keeps the U-Net but adds a second encoder and a larger base model. SD 3.5 switches to a transformer and three text encoders, including T5-xxl.

Is there a Stable Diffusion 4?

Stability AI's news page lists no Stable Diffusion 4 or other new Stable Diffusion image model in 2026. SD 3.5 is the newest family it has announced.

Can I use a LoRA from one version on another?

Not directly. A LoRA is sized for the layers of one model family, so an SD 1.5 LoRA won't load on SDXL, and neither will load on SD 3.5 or FLUX. Check the base model listed on the download page.