How do we choose K in K-fold cross-validation? What’s your favorite K?
Classic MLLevel: FoundationValidation
How do we choose K in K-fold cross-validation? What’s your favorite K?
Hints
- Which observations must remain independent during evaluation?
- Compare random, grouped and temporal splits. Where are hyperparameters selected?
Answer
There are two things to consider while deciding K: the number of models we get and the size of validation set. We do not want the number of models to be too less, like 2 or 3. At least 4 models give a less biased decision on the metrics. On the other hand, we would want the dataset to be at least 20-25% of the entire data. So that at least a ratio of 3:1 between training and validation set is maintained.
I tend to use 4 for small datasets and 5 for large ones as K.
Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0