Skip to content
Quantoros

How do diffusion models work?

Computer VisionLevel: IntermediateGenerative Models

How do diffusion models work?

Hints
  • Identify the prediction unit: image, object, pixel or sequence.
  • How do occlusion, lighting or domain changes affect results and evaluation?

Answer

Diffusion models (DDPM, Stable Diffusion) generate images by learning to reverse a noise-adding process:

Forward process (training data creation):

Original image x₀ → add Gaussian noise step by step → x₁ → x₂ → ... → xT ≈ pure noise

Reverse process (what the model learns):

Pure noise xT → predict and remove noise step by step → ... → x₁ → x₀ (generated image)

The model (U-Net or Transformer) is trained to predict the noise added at each step, conditioned on a text prompt (via cross-attention with CLIP text embeddings).

Stable Diffusion innovation: latent diffusion — run the diffusion process in a compressed latent space (VAE encoder/decoder), not pixel space. 8–16× faster than pixel-space diffusion.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions