Skip to content
Quantoros

What's the difference between L1 (Lasso) and L2 (Ridge) regularization?

Classic MLLevel: IntermediateClassical machine learning

What's the difference between L1 (Lasso) and L2 (Ridge) regularization?

Hints
  • Start from the loss: what changes when a parameter penalty is added?
  • Compare the constraint geometry and behavior near zero. Separately consider feature scaling.

Answer

Both add a penalty to the loss to shrink coefficients and reduce overfitting. L1 (Lasso) adds the sum of absolute values of coefficients (|w|), which tends to push some coefficients exactly to zero, producing sparse models useful for feature selection. L2 (Ridge) adds the sum of squared coefficients (w²), shrinking all coefficients smoothly toward zero but rarely to exactly zero. Elastic Net combines both when you want sparsity plus the stability L2 provides with correlated features.

Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT

Related questions