Skip to content
Quantoros

How do we select the right regularization parameters?

Classic MLLevel: FoundationRegularization

How do we select the right regularization parameters?

Hints
  • Start from the loss: what changes when a parameter penalty is added?
  • Compare the constraint geometry and behavior near zero. Separately consider feature scaling.

Answer

Regularization parameters can be chosen using a grid search, for example https://scikit-learn.org/stable/modules/linear_model.html has one formula for the implementing for regularization, alpha in the formula mentioned can be found by doing a RandomSearch or a GridSearch on a set of values and selecting the alpha which gives the least cross validation or validation error.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions