Skip to content
Quantoros

Which optimization techniques for training neural nets do you know?

Deep LearningLevel: IntermediateOptimization in neural networks

Which optimization techniques for training neural nets do you know?

Hints
  • Trace data and gradients through the model, noting shapes.
  • Separate training from inference and identify a necessary assumption.

Answer

  • Gradient Descent
  • Stochastic Gradient Descent
  • Mini-Batch Gradient Descent(best among gradient descents)
  • Nesterov Accelerated Gradient
  • Momentum
  • Adagrad
  • AdaDelta
  • Adam(best one. less time, more efficient)

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions