Visualize how datasets are partitioned for model training and evaluation.
Ratios
70%
15%
15%
Total100%
Dataset (n=100)
Train
Validation
Test
Distribution
Training Set
70
Model Learning
Validation Set
15
Hyperparameter Tuning
Test Set
15
Final Evaluation
A Guide to Splitting Data: Train, Validation, and Test Sets
Understand the most critical step in building a reliable machine learning model.
Why Split Data? The Core Principle
Imagine you're studying for an exam. If you study using the exact same questions that will be on the final test, you might get a perfect score. But does that mean you've truly learned the material? Not necessarily. You've just memorized the answers. Machine learning models can do the same thing—a phenomenon called overfitting.
To build a model that performs well on new, unseen data, we must split our dataset into distinct sets, each with a specific purpose.
The Training Set
This is the largest part of your dataset, typically 60-80%. The model uses this data to learn the underlying patterns, relationships, and features. It's the "study material" for your model. The model sees both the input features and the correct answers (labels) and adjusts its internal parameters to make accurate predictions.
The Validation Set
This is a smaller portion of the data (around 10-20%) that the model does not learn from directly. Instead, we use it to tune the model's hyperparameters. Hyperparameters are the model's configuration settings, like the learning rate in a neural network or the number of trees in a random forest. We train the model on the training set and then evaluate its performance on the validation set. We repeat this process with different hyperparameter settings and choose the combination that performs best on the validation data. This is like taking practice quizzes to see which study techniques are working best.
The Test Set
This final piece of the dataset (around 10-20%) is the "final exam." The model has never seen this data before—not during training and not during hyperparameter tuning. We use the test set only once, at the very end, to get an unbiased estimate of how our final, tuned model will perform in the real world. This score tells us how well the model is expected to generalize to new, unseen data.
Experiment with the Visualization
The interactive panel demonstrates these concepts visually:
Adjust the Ratios: Use the sliders to change the percentages of the train, validation, and test sets. Notice how the counts and the colors in the data grid update instantly. A common split is 70% train, 15% validation, and 15% test.
The Importance of Shuffling: Click the "Shuffle" button. The data points are randomly reassigned to the different sets. This is a crucial step in practice. If your data is ordered (e.g., by date), failing to shuffle could mean your test set contains only the most recent data, leading to a biased and inaccurate evaluation of your model's performance.
Two-Way vs. Three-Way Split: Set the validation slider to its minimum. This simulates a simple train-test split, which is common for basic models that don't require hyperparameter tuning. However, for most modern machine learning tasks, the three-way split is the gold standard.
The Golden Rule: No Data Leakage
The most important rule in data splitting is to prevent data leakage. This happens when information from outside the training set is used to create the model. The most common form of leakage is "peeking" at the test set. If you use the test set to make any decisions—like choosing which model to use or which hyperparameters to tune—you have contaminated it. Your final evaluation will be overly optimistic, and your model will likely fail in the real world. The test set must remain a sacred, untouched resource until the very end.
Cheat sheet
Train-Test Split Method
Imagine you're studying for an exam. If you study using the exact same questions that will be on the final test, you might get a perfect score. But does that mean you've truly learned the material? Not necessarily. You've just memorized the answers. Machine learning models can do the same thing—a phenomenon called overfitting.