How does L2 regularization look like in a linear model?
Classic MLLevel: IntermediateRegularization
How does L2 regularization look like in a linear model?
Hints
- Start from the loss: what changes when a parameter penalty is added?
- Compare the constraint geometry and behavior near zero. Separately consider feature scaling.
Answer
L2 regularization adds a penalty term to our cost function which is equal to the sum of squares of models coefficients multiplied by a lambda hyperparameter. This technique makes sure that the coefficients are close to zero and is widely used in cases when we have a lot of features that might correlate with each other.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0