What is a Vision Transformer (ViT)?
Computer VisionLevel: IntermediateVision-Language Models
What is a Vision Transformer (ViT)?
Hints
- Track Q, K and V shapes and which positions can interact.
- Check masking, weight normalization and training versus autoregressive inference.
Answer
ViT (Dosovitskiy et al., 2020) applies the Transformer architecture to images:
- Patch embedding: split image (e.g., 224×224) into fixed patches (e.g., 16×16 = 196 patches)
- Each patch is flattened and linearly projected to an embedding
- Add positional embeddings to patch embeddings
- Pass through standard Transformer encoder layers (multi-head self-attention)
- Classification token ([CLS]) pools global image representation
Why ViT won: Transformers scale better than CNNs with more data. At large scale, ViT outperforms CNNs on image classification. Foundation for all modern vision encoders used in VLMs.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0