Skip to content
Quantoros

How do we choose K in K-fold cross-validation? What’s your favorite K?

Classic MLLevel: FoundationValidation

How do we choose K in K-fold cross-validation? What’s your favorite K?

Hints
  • Which observations must remain independent during evaluation?
  • Compare random, grouped and temporal splits. Where are hyperparameters selected?

Answer

There are two things to consider while deciding K: the number of models we get and the size of validation set. We do not want the number of models to be too less, like 2 or 3. At least 4 models give a less biased decision on the metrics. On the other hand, we would want the dataset to be at least 20-25% of the entire data. So that at least a ratio of 3:1 between training and validation set is maintained.

I tend to use 4 for small datasets and 5 for large ones as K.

Source and licence: Alexey Grigorev & contributors — Data Science Interviews · CC-BY-4.0

Related questions