Skip to content
Quantoros

Why do we need activation functions?

Deep LearningLevel: FoundationNeural networks

Why do we need activation functions?

Hints
  • Trace data and gradients through the model, noting shapes.
  • Separate training from inference and identify a necessary assumption.

Answer

The main idea of using neural networks is to learn complex nonlinear functions. If we are not using an activation function in between different layers of a neural network, we are just stacking up multiple linear layers one on top of another and this leads to learning a linear function. The Nonlinearity comes only with the activation function, this is the reason we need activation functions.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions