Why does RFE refit the estimator after removing low-importance features rather than rank features once?
Classic MLLevel: IntermediateIterative elimination mechanism
Why does RFE refit the estimator after removing low-importance features rather than rank features once?
Hints
- Importance is conditional on the current feature set.
- Removal changes the next fitting problem.
Answer
Importance can change when competing features disappear. RFE updates the model on the remaining set at each step; this costs more fitting than a one-shot selector.
Source and licence: scikit-learn 1.7.2 — feature selection · BSD-3-Clause · answer: Quantoros editorial