The short answer
A diffusion model is a type of generative AI model that creates an image by starting from random noise and removing that noise step by step until a picture matching the prompt appears. At each step the model predicts which part of the image is noise, and a scheduler uses that prediction to produce a slightly cleaner version. Stable Diffusion popularised the approach, and the team behind Black Forest Labs' FLUX models describe themselves as pioneers of Latent Diffusion and Stable Diffusion.
| Plain meaning | A model that builds an image by cleaning up random noise in many small steps |
|---|---|
| Main use | Text-to-image and image editing; also video, and experimentally text |
| Key settings | Number of steps, guidance (how strongly to follow the prompt), seed (the starting noise) |
| Open example | FLUX 3 Image Klein from Black Forest Labs: open weights since 15 October 2026 |
| Main catch | Makers rarely say exactly how their image models work, so do not assume every image tool is a diffusion model |
What a diffusion model is, in plain English
Imagine an old television showing nothing but static. Now imagine someone who has studied millions of photos and can look at that static and say, "if this were a photo of a red bicycle by a canal, this speck here would be noise, so remove it". Do that 25 or 50 times and the static turns into a picture of a red bicycle by a canal. That is a diffusion model.
During training, the model is shown real images with increasing amounts of noise added and learns to predict the noise. When you generate an image, it runs the process backwards. Hugging Face's Diffusers documentation describes the loop exactly: the pipeline takes "random noise the size of the desired output" and passes it through the model several times, and at each timestep the model predicts the noise while a scheduler uses that prediction to make a less noisy image.
Your prompt steers the cleaning. The text is turned into an embedding, and that embedding conditions every step, so the image drifts towards what you described rather than towards any random picture.
How a diffusion model works, step by step
- Read the prompt. A text encoder turns your words into numbers the image model can use.
- Start from noise. The system creates random noise. The random number that decides this noise is called the seed, and the same seed with the same settings gives the same starting point.
- Remove noise in steps. At each step the model predicts the noise and the scheduler subtracts some of it. The Diffusers examples use 25 or 50 steps. More steps usually means more detail and slower results.
- Weigh the prompt. A setting usually called guidance scale decides how much weight the prompt gets. Too low and the image ignores you; too high and it can look harsh or overcooked.
- Decode. In a latent diffusion model, which is what Stable Diffusion is, all of this happens on a compressed version of the image to save memory, and a decoder turns the result into full-size pixels at the end.
The same idea also works for editing. Inpainting adds noise only to the area you select and cleans it up to match a new instruction, while keeping the rest of the picture.
Why diffusion models matter at work
Most people meet diffusion through the image tools they use for slides, social posts, mock-ups and product shots. Knowing how it works makes you better at those jobs.
- It explains why every run looks different. A new seed means new starting noise, so you get a new image. When you find one you like, keep the seed or the image and edit from there rather than regenerating from scratch.
- It explains the cost and the wait. Each step is a full pass through a large model. That is why image and video generation use up plan limits faster than text on tools such as Gemini, and why hosted models such as FLUX are sold pay as you go.
- It explains editing. Because diffusion can rebuild part of an image, tools can swap a background, extend a canvas or change one object while leaving the rest alone. Ask for an edit before you start again.
- It opens up private options. Black Forest Labs released FLUX 3 Image Klein, a 5.6 billion parameter model, with open weights on 15 October 2026 and says it runs on consumer GPUs down to an RTX 3090. A company that cannot send product images to a cloud service can run that kind of model on its own hardware. See how to run AI locally and what a GPU does for AI.
One honest caution: image makers do not always publish how their models work. Midjourney, Google's Nano Banana models and Adobe's Firefly models produce excellent images, but this page does not claim which architecture each one uses. For choosing a tool, our best AI for images guide and the Midjourney and Adobe Firefly guides matter more than the architecture.
Diffusion for text
The idea is spreading beyond pictures. The company Inception sells Mercury diffusion language models and says that while "all other LLMs generate text one token at a time", its models "generate tokens in parallel". Its newest model is Mercury 3. Speed claims for these models come mostly from the vendor, so treat them as claims until you test them on your own work.
An example you can try in ten minutes
Use any image tool you have. Gemini includes image creation on the free plan, and Adobe Firefly gives free daily generations on a limited set of models.
- Write one detailed prompt for an image you actually need, such as a header for a team newsletter.
- Run it twice without changing a word. Note how different the two results are. In a diffusion model, this is the seed at work: each run starts from different noise.
- Pick the better image and ask for one small edit, such as "make the sky overcast, keep everything else". Compare that with regenerating from scratch. Editing usually keeps more of what you liked.
- Now change the prompt structure: subject first, then setting, then style, then what to leave out. Run it again and see whether the image follows you more closely.
The lesson: treat image generation as a series of controlled edits, not a lottery. Our Midjourney prompts and how to make social media posts with AI put this into practice.
Related terms worth knowing
- Generative AI: the wider family of models that create new content.
- Embedding: how your prompt is turned into numbers that steer the image.
- Multimodal AI: models that work across text, images, audio and video.
- Open-source model: models such as FLUX 3 Image Klein whose weights you can download.
- Inference: running a trained model, which for diffusion means running many denoising steps.
- GPU: the chip that does the heavy lifting at every step.
Common misconceptions about diffusion models
- "It stitches together pieces of existing images." It does not cut and paste. It generates new pixels from noise using patterns learned in training. That said, models can reproduce styles, logos or likenesses closely, so check rights before commercial use.
- "More steps always means a better image." After a point, extra steps add time and little quality. Tools set sensible defaults for a reason.
- "Every AI image tool is a diffusion model." Many are, but makers often do not say, and some use other designs. Judge tools on results, editing controls, licence and privacy.
- "Diffusion is only for images." It is also used for video and is being tried for text, as Inception's Mercury models show.
- "Open weights means free for any use." FLUX 3 Image Klein comes under the FLUX Kommunity License, so read the licence before using it in a product.
Prompts to copy
Write a structured image prompt
Turn this idea into an image prompt for a diffusion image model: [idea]. Use this order: main subject, setting, lighting, camera angle or composition, style, colour palette ([brand colours]), and what to leave out. Keep it under 70 words and give me two variations with different compositions.
You get: Prompts that give the model clear, ordered guidance
Edit instead of regenerating
I like this image except for [problem]. Change only [element] to [new version]. Keep the subject, composition, lighting and colours exactly as they are.
You get: A targeted edit that keeps what already works