What's the difference between L1 (Lasso) and L2 (Ridge) regularization?
Classic MLLevel: IntermediateClassical machine learning
What's the difference between L1 (Lasso) and L2 (Ridge) regularization?
Hints
- Start from the loss: what changes when a parameter penalty is added?
- Compare the constraint geometry and behavior near zero. Separately consider feature scaling.
Answer
Both add a penalty to the loss to shrink coefficients and reduce overfitting. L1 (Lasso) adds the sum of absolute values of coefficients (|w|), which tends to push some coefficients exactly to zero, producing sparse models useful for feature selection. L2 (Ridge) adds the sum of squared coefficients (w²), shrinking all coefficients smoothly toward zero but rarely to exactly zero. Elastic Net combines both when you want sparsity plus the stability L2 provides with correlated features.
Source and licence: Ishan Dutta — Awesome Data Science Interview Q&A · MIT