How the folds work
Average the fold scores for a summary and inspect their spread. The standard deviation describes variation across these folds; it is not a confidence interval, because fold scores are not independent samples. Cross-validation estimates how the training procedure performs on similar data, not a guarantee for future populations.
Follow one evaluation
Set Folds (k) to 5 and press Run Next Fold until each fold has been held out. The visualizer shows which block is used for training and which for evaluation. Press Auto-Run All to compare the scores. A low-scoring fold may reveal a subgroup or an unstable split; do not hide it by reporting only the mean.
Choosing a split strategy
Five or ten folds are common practical choices, but the best value depends on data size, computation, and the task. Larger (k) means each model trains on a larger share of the data but requires more fits. Very large (k), including leave-one-out, can be computationally expensive and does not automatically produce a more reliable estimate.
- Classification: stratified folds can preserve class proportions when random splitting is appropriate.
- Repeated people, devices, or documents: use group-aware folds so one entity cannot occur on both sides.
- Time series: use forward-chaining or time-series splits so training precedes evaluation.
Random k-fold is not suitable for every dataset. A split that leaks groups or future information gives a precise-looking answer to the wrong question.
Model selection and the final test
If you use the same cross-validation scores to choose among many models or settings, the best score tends to be optimistic because selection favored a lucky result. Use nested cross-validation when you need an estimate of the full tuning process, or keep an untouched test set for a final evaluation after choices are complete. Fit preprocessing separately inside each training fold, typically with a pipeline.
Common mistakes with K-Fold Cross Validation
- Treating fold standard deviation as a confidence interval: the folds share much of their training data.
- Using random folds for temporal or grouped data: the split must reflect how new examples arrive.
- Tuning on the final test set: repeated choices turn it into validation data.
- Comparing different folds for competing models: use the same splits to make comparisons more informative.