The simplest useful model, and the one whose fitted parameters you can actually read - with one caveat about reading them.
Overview
What it is fitting
Linear regression is the first model most people try. It is fast, interpretable and often good enough for problems where the relationship between features and target is roughly proportional.
The model assigns one coefficient to each feature. Each coefficient says how much the target changes when that feature increases by one unit, holding everything else constant. That directness is why linear regression remains popular even when fancier models score higher. You can read the fitted model and explain it to someone who has never heard of machine learning.
It also serves as a baseline. If a complex model barely beats a linear one, the extra complexity is probably not worth it. Running linear regression first gives you a floor to compare against.
Linear regression finds one number per feature, plus an intercept, such that adding them up predicts the target as closely as possible.
For one feature that is a straight line through a scatter of points. For three features it's a plane in four dims. This No one can picture and which behaves just the same way arithmetically. Times each feature by its coeff, add the intercept, and that is the guess.
"As closely as possible" has a precise meaning. It cuts the sum of the *squared* differences between guesses and actuals. Squaring is what makes the problem solvable in one step rather than by search. There's a formula. Scikit-learn uses it. And it's also why the model is sensitive to outliers. A residual of 10 adds a hundred times more than a residual of 1. One badly wrong point pulls the line further than a dozen slightly wrong ones.
Worth knowing
coef_ holds one number per feature, in column order; intercept_ is the value when every feature is zero.
The fit minimises the sum of the squared residuals - which is why a single outlier moves the line further than several small errors.
score() returns R²: 1.0 is perfect, 0.0 is no better than predicting the mean, and negative is worse than that.
A coefficient's size depends on the feature's units, so it is not a measure of importance unless the features were scaled first.
"Linear" describes the coefficients, not the data - add x**2 as a feature and a linear model fits a curve.
With more features than samples, or perfectly correlated columns, the solution is not unique and the coefficients become unstable.
Linear Regression: A Practical Guide
The model whose params you can read, whose failures are visible. Whose coeffs are misread more often than any other number in the library.
Fitting a line, and reading it back
Two learned numbers, and together they are the whole model.
example_01.pyscikit-learn
from sklearn.linear_model import LinearRegression
import numpy as np
# price = 40 + 25 * rooms, with a little noise
rooms = np.array([[1], [2], [3], [4], [5], [6]])
price = np.array([66, 89, 115, 140, 168, 190])
model = LinearRegression().fit(rooms, price)
print("slope (coef_) : %.2f" % model.coef_[0])
print("intercept (intercept_): %.2f" % model.intercept_)
print()
print("so the line it found is: price = %.1f + %.1f * rooms"
% (model.intercept_, model.coef_[0]))
Output
More features, more coefficients
One coefficient per column, in the same order as the columns.
example_02.pyscikit-learn
from sklearn.linear_model import LinearRegression
import numpy as np
# two features now: rooms and area
X = np.array([[1, 40], [2, 55], [3, 75], [4, 90], [5, 110], [6, 130]])
y = np.array([66, 89, 115, 140, 168, 190])
model = LinearRegression().fit(X, y)
for name, c in zip(["rooms", "area"], model.coef_):
print("%-6s coefficient: %8.3f" % (name, c))
print("intercept : %8.3f" % model.intercept_)
print()
print("one coefficient per column of X, in the same order")
Output
Predictions and residuals
What the model says, what the data said, and the gap - which is the thing the fit was minimising.
example_03.pyscikit-learn
from sklearn.linear_model import LinearRegression
import numpy as np
rooms = np.array([[1], [2], [3], [4], [5], [6]])
price = np.array([66, 89, 115, 140, 168, 190])
model = LinearRegression().fit(rooms, price)
pred = model.predict(rooms)
print(" rooms actual predicted residual")
for r, a, p in zip(rooms.ravel(), price, pred):
print("%6d %7d %10.1f %9.1f" % (r, a, p, a - p))
print()
print("residuals sum to about zero: %.10f" % (price - pred).sum())
Output
score() on a regressor is R-squared
And it is worth computing once by hand, because the definition explains what a negative value would mean.
example_04.pyscikit-learn
from sklearn.linear_model import LinearRegression
import numpy as np
rooms = np.array([[1], [2], [3], [4], [5], [6]])
price = np.array([66, 89, 115, 140, 168, 190])
model = LinearRegression().fit(rooms, price)
print("score() on a regressor is R-squared:", round(model.score(rooms, price), 4))
# R-squared compares the model against always predicting the mean.
mean_only = np.full_like(price, price.mean(), dtype=float)
ss_res = ((price - model.predict(rooms)) ** 2).sum()
ss_tot = ((price - mean_only) ** 2).sum()
print("computed by hand :", round(1 - ss_res / ss_tot, 4))
Output
A big coefficient does not mean an important feature
The same model twice, with one feature's units changed. Watch the coefficient move and the fit stay identical.
example_05.pyscikit-learn
from sklearn.linear_model import LinearRegression
import numpy as np
# Same information, different units: area in square metres, then in hectares.
X_m2 = np.array([[40.0], [55.0], [75.0], [90.0], [110.0], [130.0]])
y = np.array([66, 89, 115, 140, 168, 190])
X_ha = X_m2 / 10000.0
for label, X in [("area in m2", X_m2), ("area in hectares", X_ha)]:
m = LinearRegression().fit(X, y)
print("%-18s coef_ = %12.4f R2 = %.4f" % (label, m.coef_[0], m.score(X, y)))
print()
print("identical model, coefficient 10000x apart - size means nothing")
print("until the features are on the same scale.")
Output
Linear in the coefficients, not in the data
A straight line fits a curve badly, and the same linear model fits it exactly once you give it the right feature.
example_06.pyscikit-learn
from sklearn.linear_model import LinearRegression
import numpy as np
x = np.arange(1, 11).reshape(-1, 1)
y = (x.ravel() ** 2).astype(float) # a curve, not a line
lin = LinearRegression().fit(x, y)
print("straight line on a curve, R2 =", round(lin.score(x, y), 3))
# Give it x squared as a feature and the same linear model fits exactly.
X2 = np.hstack([x, x ** 2])
poly = LinearRegression().fit(X2, y)
print("with x**2 as a second feature, R2 =", round(poly.score(X2, y), 3))
print()
print("'linear' means linear in the coefficients, not in the data.")
Output
Reading the fitted model
coef_ is an array with one entry per column of X, in the same order as the columns. intercept_ is a single number: the guess when each feature is zero.
Together they're the entire model. There's nothing else stored, no data kept, no lookup table. Which is why a fitted linear regression is a few numbers even so of whether it was trained on a hundred rows or a hundred million.
That readability is the model's main hands-on edge. A coeff of 25.2 on rooms says. Holding the other features fixed, one more room is associated with 25.2 more units of price. The phrase "holding the other features fixed" is doing real work in that sentence and is the part people drop. A coeff isn't the effect of a feature on its own, it's the effect after the other features in the model have accounted for what they can.
The intercept is often meaningless in alone. A house with zero rooms and zero area doesn't exist. The intercept is where the line happens to cross rather than a guess anyone would make. It matters for the math and rarely for the reading.
The coefficient trap
This is the most often repeated mistake about linear models. It's worth being blunt: the size of a coeff tells you nothing about the importance of a feature.
A coeff is expressed in units of target per unit of feature. Measure area in square metres and the coeff is some number. Measure the same area in hectares and the coeff is ten thousand times larger. The model is the same, the guesses are the same, the R² is the same. Only the units changed.
So checking coeffs across features only means a thing if the features are on the same scale. Standardising them first. Subtracting the mean and cutting by the standard deviation. Makes the coeffs close. They then answer "how much does the guess move per standard deviation of this feature". This Is a question worth asking.
Even then, "importance" is slippery. Two linked features share the credit between them in a way that depends on the noise. A feature can have a small coeff because a linked one absorbed the signal rather than because it doesn't matter. Permutation importance answers the question more honestly. This track's tree modules cover the options.
R-squared, and what a negative value means
score() on a regressor returns R². Its meaning is a check rather than an abs measure. How much of the variation in the target the model accounts for, scaled to a baseline that always predicts the mean.
1.0 means the guesses are exact. 0.0 means the model does just and always guessing the mean. Which is to say it has learned nothing handy. Values in between are the usual case.
A minus R² surprises people and is fully possible. It means the model is worse than predicting the mean. This Happens routinely on a test set when a model has overfitted. Always shows a thing is wrong rather than merely weak.
The number's weakness is that it's unitless and so not right real. An R² of 0.85 doesn't tell you whether the guesses are wrong by pounds or by thousands of pounds. That is often the question that matters. The next module covers the metrics that answer it.
What "linear" actually restricts
The name misleads. Linear regression is linear in its *coeffs*, not in the data. The gap is what makes it far more flexible than it appears.
Adding x**2 as an extra column lets a linear model fit a parabola just. Adding x1 * x2 lets it capture an interplay between two features. Adding log(x) lets it fit a curve that flattens. In each case the model is still linear. It's still timesing each column by a coeff and adding. Because the non-linearity is in the *feature*, not in the fitting.
PolynomialFeatures automates this by making all the powers and products up to a degree you choose. It's truly handy and it grows quickly. Degree 3 on ten features makes 286 columns, most of them useless. The model will happily fit noise with them. Which is the standard trade this track keeps returning to.
When it is the wrong model
Three situations where linear regression struggles, and knowing them saves fitting it and being puzzled.
truly non-linear ties, unless you make the features to capture them. A tree-based model finds them without being told what shape to look for. This Is why grad boosting is the usual default on tabular data.
More features than samples. The answer is no longer unique. Infinitely many coeff sets fit the training data just right. And plain linear regression has no principle for choosing between them. Ridge and Lasso add one. This Is just what smoothing is for.
Highly linked features. Two columns carrying almost the same info make the coeffs unstable. Small changes in the data give large swings in how the credit is cut. The single numbers stop being readable though the guesses remain fine.
For the second and third, Ridge is the drop-in swap and is a better default than plain LinearRegression for any of it with many features. It uses the same interface, adds one setting. Gives up a little training fit for coeffs that don't move around.
Ridge and Lasso, in one paragraph each
Two variants solve the problems at the end of the previous section, and both are one-line swaps.
Ridge adds a fine proportional to the sum of the squared coeffs. The effect is to shrink them all towards zero, more so as alpha grows. This Stops any one of them growing large to chase noise. It handles linked features gracefully. Rather than one coeff swinging plus and another minus, both end up moderate and stable. Ridge(alpha=1.0) is a sensible default and a better starting point than plain LinearRegression when there are more than a handful of features.
Lasso punishes the sum of the *abs* coeffs instead. That change has a striking consequence. It drives some coeffs to just zero rather than merely small. So it selects features and fitting them. The fitted model names the columns it decided to skip. The cost is that among linked features it tends to pick one arbitrarily and zero the rest. This Makes the pick unstable even when the guesses are fine.
ElasticNet mixes the two fines for when you want both properties. All three take alpha, all three need the features scaled first. A fine on coeff size is meaningless when the coeffs are in incomparable units. And all three are tuned with the cross-validated search covered later in this track.
The assumptions, and which ones matter in practice
Textbooks list four or five assumptions behind linear regression. For guess. This Is what this library is for, they matter less than the textbooks imply. It's worth knowing which is which.
Linearity is the one that truly matters. If the tie curves and you haven't given the model a curved feature, the guesses will be systematically wrong in a pattern. Too low at the ends and too high in the middle, or the reverse. Plotting residuals against guesses makes it obvious. A good fit shows a shapeless cloud. A missed curve shows an arc.
Free errors matters for time series and grouped data. Is the reason those need different splitting. Nothing about the fit detects it.
Constant error variance and most times spread errors are assumptions of the *inference*. The p-values and trust intervals that scikit-learn doesn't find. If you only want guesses, violating them costs you fit in some regions rather than validity. If you want to make a claim about whether a coeff is much different from zero, you want statsmodels, and then they matter.
The hands-on version: look at the residuals for a pattern. A pattern means a missing feature or the wrong shape of model. No amount of smoothing fixes either.
Should I use LinearRegression or Ridge by default? Ridge, for any of it with more than a few features. It costs one setting and drops the instability that plain least squares has when columns are linked.
Does it need scaled features? Plain linear regression doesn't - the fit is the same either way. Ridge, Lasso and ElasticNet do. Their fine is on coeff size.
Why does fitting take no iterations? plain least squares has a closed-form answer. The answer is found right rather than searched for. That is why there's no max_iter and no settling warning on this estimator.
What is plus=True for? It constrains each coeff to be non-minus. This Is at times what physics or accounting needs - mixing proportions, say. Here A minus part is meaningless.
Can it handle missing values? No. It raises on NaN. This Is why imputation comes earlier in a pipeline than the model does.
Things to try with Linear Regression
Add an outlier. In the first editor, change one price to 400 and watch the slope move. Squared errors mean one bad point has a large vote.
Read the units. The fifth editor makes the coeff trap concrete. Fit both, then look at the R² column and note it doesn't move.
Fit the curve. In the last editor, try degree three by adding x ** 3 as a third column and check whether R² raises on data that is truly quadratic.
Break it on purpose. Give the model two the same columns and look at the two coeffs. The sum is stable. The split between them isn't.
Questions people ask about Linear Regression
Does linear regression need scaled features? Plain least squares does not, since the predictions are unchanged. Scaling matters for Ridge and Lasso, and for comparing coefficients across features.
Can R-squared be negative? Yes, on held-out data. It means the model predicts worse than always guessing the mean of the target.
Can a linear model fit a curve? Yes, if you add features such as squares with PolynomialFeatures. The model is still linear in its coefficients.
What is the difference between Ridge and Lasso? Both shrink coefficients to reduce overfitting. Lasso can push some exactly to zero, which drops features, while Ridge only makes them small.
Do outliers affect it? Strongly. Squared error gives a far point a large pull on the line, so check for outliers or try HuberRegressor.
Can I read coefficients as importance? Only when features share a scale and are not strongly correlated. Otherwise the same effect can be split across features in misleading ways.
Where this leaves you
Two learned arrays, one formula, and a score that checks the model against guessing the mean. The coeffs are readable. This Is the model's great edge, and readable in a way that needs care about units before any of them can be checked.
Check yourself
0 of 4
Answer without scrolling back up.
What does coef_ contain?
One number per column of X, in the same order as the columns, plus intercept_ as a single separate number.
A feature's coefficient is ten times larger than another's. What does that tell you?
A coefficient is in units of target per unit of feature. Change the feature's units and the coefficient changes while the model stays identical.
What does a negative R-squared mean?
R-squared compares the model against the mean baseline. Below zero means the baseline would have done better, which is common on a test set after overfitting.
Can a linear model fit a curved relationship?
"Linear" refers to the coefficients. Adding x**2 as a column lets the same model fit a parabola exactly.
Cheat sheet
Linear Regression
Fit a linear regression, read its coefficients and intercept, and learn why the numbers are misread more often than any other model's.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.