Skip to content
Quantoros

What is transfer learning, and when would you fine-tune vs. use a model as a fixed feature extractor?

Deep LearningLevel: FoundationNeural networks

What is transfer learning, and when would you fine-tune vs. use a model as a fixed feature extractor?

Hints
  • Which parameters change and which stay frozen?
  • Separate memory for weights, gradients, optimizer states and activations.

Answer

Transfer learning reuses a model pretrained on a large dataset (e.g., ImageNet, or a large text corpus) as a starting point for a new, often smaller-data task, since early/general layers tend to learn transferable features (edges, textures, or general language patterns). Use it as a fixed feature extractor (freeze pretrained layers, train only a new head) when your target dataset is small and similar to the source domain. Fine-tune (unfreeze some/all layers with a low learning rate) when you have more target data or the target domain differs meaningfully from the source, allowing the model to adapt its learned representations.

Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT

Related questions