Skip to content
Quantoros

What’s the difference between L2 and L1 regularization?

Classic MLLevel: IntermediateRegularization

What’s the difference between L2 and L1 regularization?

Hints
  • Start from the loss: what changes when a parameter penalty is added?
  • Compare the constraint geometry and behavior near zero. Separately consider feature scaling.

Answer

  • Penalty terms: L1 regularization uses the sum of the absolute values of the weights, while L2 regularization uses the sum of the weights squared.
  • Feature selection: L1 performs feature selection by reducing the coefficients of some predictors to 0, while L2 does not.
  • Computational efficiency: L2 has an analytical solution, while L1 does not.
  • Multicollinearity: L2 addresses multicollinearity by constraining the coefficient norm.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions