Skip to content
Quantoros

What does in-context learning mean, and what makes it different from fine-tuning?

LLM & AILevel: FoundationPrompting & In-Context Learning

What does in-context learning mean, and what makes it different from fine-tuning?

Hints
  • Which parameters change and which stay frozen?
  • Separate memory for weights, gradients, optimizer states and activations.

Answer

In-context learning is a model's ability to do a new task purely from the instructions or examples in the prompt, with no weight updates at all. Fine-tuning permanently changes the weights on a labelled dataset.

Source and licence: Skillumen — Awesome AI Engineer Interview Questions · CC0-1.0

Related questions