vizlearn
Home / Machine Learning

Linear Regression (OLS)

Machine LearningBy Updated

Fit a line to a dataset by minimizing the Mean Squared Error (MSE). Drag the regression line manually to see how the residuals (red lines) change, or compute the exact best fit.

Controls

Slope (m)
Intercept (c)
Current Loss 0.00
Optimal Loss 0.00
Distance to Optimal 0.00
Tip: You can click and drag individual blue data points, or drag the line handles to manually adjust the fit. Drag the background to pan the view.

Visualization

Drag to pan & edit points

Overview

Ordinary least squares (OLS) fits a linear model by minimizing the sum of squared residuals, where each residual is observed value minus prediction. For one predictor, its fitted line is ŷ = mx + c; the slope and intercept depend on the sample.

The line and its residuals

The visualizer draws each vertical residual. Set Hypothesis Line manually and watch Live Loss (MSE) change; Snap to Best Fit displays the OLS solution. Change Data Points or use Generate New Data to compare another sample.

How the coefficients are found

For a simple line, the slope is the covariance between x and y divided by the variance of x; the intercept makes the line pass through the sample means:

m = sum((xᵢ - x̄)(yᵢ - ȳ)) / sum((xᵢ - x̄)²)
c = ȳ - m x̄

For points (1, 2), (2, 4), and (3, 5), the means are x̄=2 and ȳ=11/3. The slope is 3/2, and the intercept is 2/3, giving ŷ=1.5x+0.67. The line passes through the mean point (2, 11/3), though it does not need to pass through every observation.

For multiple predictors, OLS chooses a coefficient for each feature while minimizing the same squared-error objective. A coefficient describes the fitted change associated with one feature while the others are held fixed; that association is not automatically causal.

Why square the residuals?

Squaring prevents positive and negative errors from cancelling and gives larger residuals more weight. A residual twice as large contributes four times as much squared error. This is useful when large misses should be penalized, but it makes the fit sensitive to outliers; inspect the data and residuals when a point pulls the line strongly.

Check the fit, not just the line

Residuals should not show a clear pattern if a linear model is adequate for the task. A curve can suggest nonlinearity; a funnel can suggest changing error variance; a time pattern can indicate dependent errors. These diagnostics matter for interpretation and uncertainty estimates. OLS can still calculate coefficients when assumptions fail, but the fit or standard inferential claims may be unreliable.

OLS does not require normally distributed errors to compute its least-squares coefficients. Normality is sometimes assumed for particular small-sample confidence intervals and tests, under other model assumptions. Avoid interpreting a straight line outside the observed predictor range without additional evidence: extrapolation can be far less reliable than interpolation.

Sources and next steps

Predict, then reveal

About to run: double Slope (m) to 2. Before it does — what happens to the readout?

Committing to an answer first is the point — the reveal runs the experiment on the visualisation above and reads the real value back, so nothing here is scripted.

Check yourself

0 of 3

Answer without scrolling back up.

  1. Ordinary least squares minimises the sum of:

  2. Why does OLS react so strongly to a single far-off point?

  3. R-squared of 0.0 means:

Cheat sheet

Linear Regression with OLS

Drag a regression line by hand and watch the residuals and mean squared error react, then let least squares find the optimum.

MACHINE LEARNING · vizlearn.in/machine_learning/linear_regression_with_ols.html

More in Machine Learning

Machine LearningEvaluation Metrics for RegressionMachine LearningLogistic RegressionMachine LearningK-Nearest NeighborsMachine LearningConfusion Matrix Analysis

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.

About VizLearnGitHubLinkedInContact