Which regularization techniques do you know?
Classic MLLevel: IntermediateRegularization
Which regularization techniques do you know?
Hints
- Start from the loss: what changes when a parameter penalty is added?
- Compare the constraint geometry and behavior near zero. Separately consider feature scaling.
Answer
There are mainly two types of regularization,
- L1 Regularization (Lasso regularization) - Adds the sum of absolute values of the coefficients to the cost function.
- L2 Regularization (Ridge regularization) - Adds the sum of squares of coefficients to the cost function.
- Where determines the amount of regularization.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0