Evaluation Metrics for Regression

In regression, our goal is to predict a continuous value, like a price or a temperature. But how do we know if our model's predictions are any good? Evaluation metrics are the tools we use to measure a model's performance and quantify its error. This lab visualizes four of the most common regression metrics, allowing you to see how they respond to changes in data and model fit in real-time.

The Four Key Metrics

Each metric tells a slightly different story about your model's errors. The interactive plot above shows the true data points (green dots), the model's prediction line (orange), and the errors, or residuals (dashed red lines), which are the distances between each true point and the line.

Guided Experiments

Use the controls and the interactive plot to build a strong intuition for these metrics.

1. The Impact of a Single Outlier: Start with the "Good Linear Fit" dataset. Drag one of the green data points far away from the prediction line. Watch the metrics change. You'll notice that MSE and RMSE increase dramatically compared to MAE. This is because they square the error, heavily penalizing this large deviation (outlier).
2. Good Fit vs. Poor Fit: Use the "Initial Dataset" dropdown. Switch between "Good Linear Fit" and "Poor Fit (High Bias)". Observe how all error metrics (MAE, MSE, RMSE) are much higher for the poor fit. Now look at the $R^2$ score. For the good fit, it should be high (e.g., > 0.8), while for the poor fit, it will be very low, possibly even negative, indicating the model is useless.
3. Manually Improving the Fit: Select the "Poor Fit" dataset. The orange line will be far from the data. Now, click and drag the handles on the ends of the orange line to move it closer to the green data points. As you manually improve the fit, watch all the error metrics decrease and the $R^2$ score increase towards 1.0.
4. The Effect of Noise: Select the "Good Linear Fit" and use the "Noise Level" slider. As you increase the noise, the data points become more scattered. Notice that even with the best possible line, the error metrics will rise, and the maximum achievable $R^2$ score will fall. This shows that the inherent randomness in data puts a limit on how well any model can perform.

Which Metric Should You Use?

• Use MAE when you want a simple, interpretable metric that is not sensitive to outliers. For example, forecasting sales where a few large errors are acceptable.
• Use RMSE when large errors are particularly undesirable and you want to penalize them more. It's the most common metric for regression tasks. For example, predicting house prices where a large error could be very costly.
• Use $R^2$ when you want to understand the proportion of variance your model explains. It's great for communicating the overall "goodness of fit" of your model to a non-technical audience.