Skip to content
Quantoros

What is a Vision Transformer (ViT)?

Computer VisionLevel: IntermediateVision-Language Models

What is a Vision Transformer (ViT)?

Hints
  • Track Q, K and V shapes and which positions can interact.
  • Check masking, weight normalization and training versus autoregressive inference.

Answer

ViT (Dosovitskiy et al., 2020) applies the Transformer architecture to images:

  1. Patch embedding: split image (e.g., 224×224) into fixed patches (e.g., 16×16 = 196 patches)
  2. Each patch is flattened and linearly projected to an embedding
  3. Add positional embeddings to patch embeddings
  4. Pass through standard Transformer encoder layers (multi-head self-attention)
  5. Classification token ([CLS]) pools global image representation

Why ViT won: Transformers scale better than CNNs with more data. At large scale, ViT outperforms CNNs on image classification. Foundation for all modern vision encoders used in VLMs.

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions