Skip to content
Quantoros

What is dropout? Why is it useful? How does it work?

Deep LearningLevel: IntermediateNeural networks

What is dropout? Why is it useful? How does it work?

Hints
  • Which activations change randomly during training?
  • Compare training with inference; how can their signal scales remain comparable?

Answer

Dropout is a technique that at each training step turns off each neuron with a certain probability of p. This way at each iteration we train only 1-p of neurons, which forces the network not to rely only on the subset of neurons for feature representation. This leads to regularizing effects that are controlled by the hyperparameter p.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions