Images
What is a LoRA? A plain explainer
Key takeaways
A LoRA (Low-Rank Adaptation) is a small add-on file that pushes a base image or video model toward a style, character, object, or effect. The base model's own file stays untouched.
The SDXL and FLUX.1 LoRA files cited here range from 22.4 MB to 217.9 MB. The SDXL base checkpoint is 6.9 GB, and FLUX.1 [dev] is 23.8 GB.
You control a LoRA with a strength value, where 1.0 is the full trained effect, and many LoRAs also need a trigger word in the prompt.
A LoRA only loads on the model family it was trained for, and two LoRAs loaded together can pull the same layers in different directions.
The base model's license follows the LoRA. A LoRA trained on FLUX.1 [dev] falls under that model's non-commercial license.
What is a LoRA in AI image generation?
A LoRA is a small file of extra weights that sits on top of a base model and changes what it draws: an art style, a recurring character, a product, or a visual effect. You load the base model, load the LoRA next to it, and generate as usual.
The name stands for Low-Rank Adaptation, a fine-tuning method described in a 2021 paper by Edward Hu and seven co-authors. It was written about large language models such as GPT-3, and image tools picked it up afterward. Civitai and Hugging Face host LoRAs for image models like SDXL and FLUX.1, and Hugging Face also carries them for video models like Wan and LTX.
How a LoRA changes a model
A trained model is mostly large grids of numbers called weight matrices. Ordinary fine-tuning adjusts every number in every grid and saves a whole new copy of the model.
The LoRA paper took a shortcut. In its words, LoRA "freezes the pre-trained model weights and injects trainable rank decomposition matrices" into each layer. The original grid stays frozen, and training learns only the change to it, stored as two thin matrices that multiply out to the grid's full size. The "rank" is how thin they are: at rank 16, each change takes 16 columns instead of the layer's full width.
The paper's own figures show how big the saving can get. Compared with fully fine-tuning GPT-3 175B, the authors wrote, "LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times." The saved file shrank by roughly the same factor, from 350 GB to 35 MB. That was a language model, but the same arithmetic is why image LoRAs come in megabytes.
At generation time, the LoRA's change is added on top of the base weights. The paper points out that the change can be merged into those weights ahead of time, so unlike older adapter methods, it adds no inference latency.
LoRA vs. checkpoint vs. embedding
File type | What it changes | Example and size |
|---|---|---|
Full checkpoint or fine-tune | Every weight in the model | SDXL base 1.0: 6.9 GB; FLUX.1 [dev]: 23.8 GB |
LoRA | A low-rank change to selected layers | Pixel Art XL for SDXL: 170.5 MB |
Embedding (textual inversion) | What one word means to the text encoder | The "cat-toy" concept on Hugging Face: about 4 KB |
A full fine-tune is a standalone model that replaces the base. A LoRA rides on a base you already have, so one copy of SDXL can serve dozens of small files. The paper's version of this: 100 adapted GPT-3 models take about 354 GB with LoRA, against roughly 35 TB as full copies.
An embedding is smaller still because it changes no model weights at all. The Automatic1111 web UI wiki describes textual inversion this way: "A method to fine tune weights for a token in CLIP." CLIP is Stable Diffusion's text encoder. An embedding teaches it a new word and leaves the image layers alone, which a LoRA edits directly.
How big LoRA files are
The diffusers LoRA docs say LoRA checkpoints "are typically only a couple hundred MBs." Real files show a wide spread:
SDXL: Pixel Art XL is 170.5 MB and the Ikea Instructions LoRA is 185.7 MB.
FLUX.1 [dev]: a Ghibli-style characters LoRA is 172 MB, and XLabs' Realism LoRA is 22.4 MB.
On Civitai, Detail Tweaker XL for SDXL is 217.9 MB, while the SD 1.5 version of the same idea is 36.1 MB.
Size depends mostly on the rank and how many layers were trained, so two LoRAs for one base can differ nearly eightfold. Video LoRAs run bigger. A Wan 2.1 image-to-video LoRA called Squish is 359.3 MB, and Lightricks' Cakeify LoRA for LTX-Video 0.9.5 is 469.8 MB. If you're collecting dozens, our guide to how much storage you need for local AI models covers the totals.
Trigger words
Many LoRAs are trained with captions that include a specific phrase, and you put it in your prompt to call up what the LoRA learned. The Pixel Art XL model card lists "pixel art" as its prompt, the Squish card gives "sq41sh squish effect," and Lightricks' Cakeify LoRA uses "CAKEIFY." Others don't need one: the Ikea Instructions card says, "No trigger word needed." Either way, the model card is where you'll find out.
Trainers often pick a made-up token on purpose. An AWS guide to training a FLUX.2 Klein LoRA suggests a word such as "ohwx," because it "doesn't conflict with existing model knowledge."
Strength and weight settings
In the Automatic1111 web UI, you add a LoRA to the prompt as `<lora:filename:multiplier>`, and the wiki says the multiplier is "generally from 0 to 1." The diffusers docs define the ends of that scale: a weight of 0 uses only the base model, and a weight of 1 applies the LoRA fully.
ComfyUI's Load LoRA node splits strength in two. `strength_model` sets how strongly the LoRA changes the image model, and `strength_clip` sets how strongly it changes the text encoder's reading of your prompt.
Model cards often suggest their own value, and some go outside 0 to 1:
LoRA | Base | Suggested weight |
|---|---|---|
Ikea Instructions | SDXL | 1.0 "works well" |
Pixel Art XL | SDXL | 1.2 when paired with an LCM LoRA |
Detail Tweaker XL | SDXL | Start at 1.5; works from -3 to 3, and negative values reduce detail |
Squish | Wan 2.1 I2V | 1.0 |
The diffusers docs' own two-LoRA example uses 0.7 and 0.8, so the examples here mostly land between 0.7 and 1.2. Start with the card's value, lower it if the LoRA overrides your prompt, and raise it if you can't see the effect.
Stacking several LoRAs and why they conflict
ComfyUI lets you chain Load LoRA nodes in series, and diffusers takes a list of LoRAs with a weight for each. The catch is that every LoRA adds its own change to the same base weights, and the changes simply add up. Two style LoRAs that touch the same layers can pull them in different directions, and the result can be a muddy blend of neither. Lowering each weight keeps the total in range; in the diffusers docs, weights of 0.5 and 0.5 average two LoRAs. For characters specifically, our roundup of AI image generators for keeping the same face covers LoRAs alongside reference-image methods.
Why a LoRA only works with its own base model
Those two thin matrices are sized to fit specific layers in a specific model. SDXL is built around a UNet, while FLUX.1 uses a transformer with different layer names and sizes (diffusers loads LoRAs into `pipeline.unet` for SDXL and `pipeline.transformer` for Flux). An SDXL LoRA has no matching slots in FLUX.1, so it won't load correctly there, and the same goes for SD 1.5 on SDXL.
Civitai shows this, since creators often publish a separate file for each family. One hands LoRA lists versions for SDXL 1.0, SD 1.5, FLUX.1 D, Pony, Illustrious, and two Z-Image models, each trained separately. Check the "Base Model" field on Civitai or the `base_model` line on a Hugging Face card before you download.
Training your own LoRA
Training means gathering images, captioning them, training, and testing the saved checkpoints at a few strengths. The AWS FLUX.2 Klein guide recommends 20 to 50 images with varied poses, lighting, and backgrounds, and defaults to rank 16, 1,500 training steps, and 512-pixel images. Captions go in a `.txt` file next to each image, and Ostris' ai-toolkit swaps a `[trigger]` placeholder in them for your trigger word.
The common open-source trainers and some of the models they cover:
kohya_ss, a graphical interface for Kohya's scripts: SD 1.5 and 2.x, SDXL, SD3, and Flux.1, among others.
ai-toolkit, with a graphical interface or command line: FLUX.1, FLUX.2, FLUX 3 Image Klein, Qwen-Image, and SDXL, plus Wan 2.1 and 2.2 and LTX-2 through LTX-2.5 for video.
Diffusers: Python training scripts for people who'd rather work in code.
Musubi Tuner, also from Kohya: HunyuanVideo, Wan 2.1 and 2.2, FLUX.2, and Qwen-Image, among others.
Training takes more graphics memory than generating. Ai-toolkit's example FLUX.1 config is named for 24 GB cards, and the AWS guide asks for a GPU with at least 13 GB for the smaller FLUX.2 Klein 4B. Our VRAM guide for AI image and video covers what each card size can handle.
Licensing: the base model's terms come with the LoRA
Base-model licenses tend to treat a LoRA as a derivative of the model, and some say so outright. The FLUX.1 [dev] Non-Commercial License defines a Derivative as including "any customized or fine-tuned version" of the model and says its restrictions "also apply to any Derivative you create." XLabs' Realism LoRA card states that the file "falls under the FLUX.1 [dev] Non-Commercial License." Images are treated differently: the same license says "You may use Output for any purpose (including for commercial purposes)," except to train a competing model.
Other bases carry other terms. SDXL ships under the CreativeML Open RAIL++-M License, which requires its use restrictions to pass along with any derivative. FLUX.2 [klein] base 4B and Wan 2.2 are released under Apache 2.0, while LTX-2.5 and HunyuanVideo each have their own community license. Read the base model's license before you sell a LoRA or build a paid product around one.
Video LoRAs: Wan, LTX, and Hunyuan
Video models take LoRAs the same way. The diffusers docs say their LoRA loading works the same across model families, naming HunyuanVideo and Wan. Video LoRAs are trained on clips, and many teach a motion or effect: Squish squishes an object in your starting image, and Cakeify shows a knife cutting into a cake shaped like your object.
Video models come in several variants each, so the base-model check gets narrower. Squish was trained on Wan 2.1's 14B image-to-video model at 480p, and Cakeify on LTX-Video 0.9.5. Check the exact version and mode on the card.
What is a LoRA FAQ
Is a LoRA a model?
Not on its own. A LoRA is a set of changes to a base model's weights, so it does nothing without that base model loaded.
Can I use an SDXL LoRA with FLUX?
No. A LoRA's weights are sized to the layers of the model it was trained on, and SDXL and FLUX.1 have different architectures. Look for a FLUX version or train one.
What strength should I use for a LoRA?
Start with the model card's value. Automatic1111's wiki gives 0 to 1 as the general range, and the examples in this piece mostly fall between 0.7 and 1.2. Go lower when stacking LoRAs.
How many images do I need to train a LoRA?
An AWS guide to training a FLUX.2 Klein LoRA recommends 20 to 50 images with varied poses, lighting, and backgrounds.
Can I use images from a FLUX.1 [dev] LoRA commercially?
The FLUX.1 [dev] license allows commercial use of outputs, with conditions, and bars using them to train a competing model. The LoRA file itself is a derivative under the non-commercial license, so selling it isn't allowed under those terms.
Do LoRAs work with AI video models?
Yes. Wan, LTX, and HunyuanVideo all take LoRAs, often for motion effects, and diffusers loads them with the same calls it uses for images. The two video examples above are 359.3 MB and 469.8 MB.