Skip to content
Quantoros

Why is bestscore not an unbiased final estimate after trying many hyperparameter configurations?

Classic MLLevel: AdvancedSearch-induced optimism

Why is best_score_ not an unbiased final estimate after trying many hyperparameter configurations?

Hints
  • Selection consumes validation information.
  • Evaluate the procedure, not only its winner.

Answer

This answer is part of mentoring

Advanced questions come with a full breakdown and a mock interview with a working quant researcher. The first call is free.

Source and licence: scikit-learn 1.7.2 — grid search · BSD-3-Clause · answer: Quantoros editorial

Related questions