Skip to content
Quantoros

Explain what Dropout does and why it helps prevent overfitting.

Deep LearningLevel: IntermediateNeural networks

Explain what Dropout does and why it helps prevent overfitting.

Hints
  • Which activations change randomly during training?
  • Compare training with inference; how can their signal scales remain comparable?

Answer

Dropout randomly "drops" (zeroes out) a fraction of neurons during each training forward pass, forcing the network to not rely too heavily on any single neuron or co-adapted group of neurons. This acts like training an ensemble of many thinned sub-networks that share weights, and at test time the full network is used (with weights scaled, or using "inverted dropout" during training) to approximate averaging over that ensemble — reducing overfitting, especially in large fully-connected layers.

Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT

Related questions