Skip to content
Quantoros

Which regularization techniques do you know?

Classic MLLevel: IntermediateRegularization

Which regularization techniques do you know?

Hints
  • Start from the loss: what changes when a parameter penalty is added?
  • Compare the constraint geometry and behavior near zero. Separately consider feature scaling.

Answer

There are mainly two types of regularization,

  1. L1 Regularization (Lasso regularization) - Adds the sum of absolute values of the coefficients to the cost function. λ∑i=1n∣wi∣\lambda\sum_{i=1}^{n} \left | w_i \right |
  2. L2 Regularization (Ridge regularization) - Adds the sum of squares of coefficients to the cost function. λ∑i=1nwi2\lambda\sum_{i=1}^{n} {w_{i}}^{2}
  • Where λ\lambda determines the amount of regularization.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions