Maths for Machine Learning
The vectors, matrices, derivatives and probability that every machine learning course assumes you already have.
About this track
Almost every machine learning explanation stops to say "recall that the gradient points uphill" or "this is just a dot product" and moves on. This track is where you go to actually see those things, one at a time, with something you can drag.
It starts at the equation of a line and works up through vectors, matrix multiplication, derivatives and probability distributions. Nothing here assumes a maths degree, and nothing is left as an exercise for the reader.
The order matters more here than anywhere else on the site. Each page is built only from ideas introduced before it, so the track can be read straight through without ever hitting a symbol that has not been explained.
What you will be able to do
- Read a vector as both a list of numbers and a direction, and say what a dot product measures.
- Multiply matrices by hand, and explain why the inner dimensions have to agree.
- Interpret a derivative as a slope and a gradient as the direction of steepest increase.
- Tell a probability distribution from a likelihood, and read what a standard deviation claims.
- Recognise these objects when they appear unannounced in a machine learning paper.
How the track is ordered
Start at the equation of a line, because every later idea is a generalisation of it: a slope becomes a gradient, a line becomes a plane, and a coefficient becomes a weight. From there the track splits into three strands that meet at the end - linear algebra (vectors, dot products, matrices as transformations), calculus (derivatives, partial derivatives, the chain rule), and probability (distributions, expectation, Bayes). Nothing assumes a maths degree. If you can rearrange an equation and read a graph, you have the prerequisites.
Where this leads
Once the gradient and the dot product are familiar, the machine learning and deep learning tracks stop being notation and start being mechanisms. Gradient descent is a derivative followed downhill; a neuron is a dot product with a bias. Most people find those tracks considerably easier after this one, which is why it is first on the learning path.
All 47 modules, in teaching order
- 01Equation of a Line (y = mx + c)Drag slope and intercept and watch y = mx + c respond. See slope-intercept, point-slope and standard form describe the same line.
- 02LogarithmsTurn a product of tiny probabilities into a sum of manageable numbers — the reason loss functions and learning-rate plots use logs.
- 03ExponentialsSee why multiplying by a constant at each step starts slow then turns vertical — the shape behind softmax and vanishing gradients.
- 04Vectors and the Dot ProductMultiply matching components and sum to get a number that measures how much two vectors point the same way — behind every forward pass.
- 05Vector NormsThree answers to how big a vector is. See why the L1 diamond's corners make Lasso produce exact zeros while the L2 circle only shrinks.
- 06Distance MetricsFour answers to how far apart two points are. Watch cosine ignore length entirely while Euclidean, Manhattan and Chebyshev disagree by shape.
- 07Matrix MultiplicationCompute each entry as a row-times-column dot product and see why the shape rule and GPU parallelism make this the engine of deep learning.
- 08A Matrix is a TransformationDrag four numbers and watch space rotate, stretch and shear. The columns are where the basis vectors land, and the determinant is the area they span.
- 09ProjectionsDrag one vector onto another and watch its shadow, plus the perpendicular residual left behind. The geometric root of least squares.
- 10DeterminantThe determinant is the area the unit square becomes. Sweep it through zero and watch the plane flatten, then turn itself inside out.
- 11Identity, Inverse and TransposeApply a transformation, then undo it. Watch the undo fail the moment the determinant hits zero, and see why the transpose is not an undo at all.
- 12Rank and Linear IndependenceDrag two columns until one is a multiple of the other and watch the reachable plane collapse onto a line. That collapse is what rank counts.
- 13Eigenvalues and EigenvectorsSweep a vector around the circle and find the two directions a matrix does not rotate. Trace, determinant and the guarantee PCA is built on.
- 14Derivatives and SlopeShrink the gap between two points until the secant becomes the tangent — the single idea every gradient-based optimiser is built on.
- 15The Chain RuleMultiply local derivatives of nested functions link by link — the exact arithmetic backpropagation performs through every layer.
- 16Partial Derivatives and the GradientTake one derivative per variable, bundle them into the gradient vector, and see why it points straight uphill on any loss surface.
- 17Mean, Mode and MedianCompare three measures of typical: the mean that outliers drag, the median that resists them, and the mode that works on categories.
- 18Mean, Variance and Standard DeviationMeasure centre with the mean and spread with variance and standard deviation — the two numbers every normalisation layer uses.
- 19Covariance and CorrelationStretch and tilt a point cloud. Covariance moves with the units, correlation does not, and neither notices a curve.
- 20Probability BasicsCount outcomes, combine events with addition and multiplication rules, and see why a fair coin owes you nothing after ten heads.
- 21Conditional ProbabilityConditioning does not change the outcomes, it discards them. Watch the sample space shrink, and see why P(A given B) is not P(B given A).
- 22Bayes' TheoremUpdate a prior belief with new evidence and see why a 99%-accurate test still gives mostly false positives when the condition is rare.
- 23The Normal DistributionTwo numbers — mean and standard deviation — define the bell curve. The 68-95-99.7 rule and why the CLT makes this shape unavoidable.
- 24Maximum Likelihood EstimationSlide a candidate distribution over fixed data and watch the likelihood peak. Where squared error, log loss and cross-entropy all come from.
- 25Entropy and InformationDrag a distribution and watch uncertainty rise and fall. Entropy is the average surprise, and the quantity every classification loss is built from.
- 26Cross-Entropy and KL DivergenceDrag one distribution towards another and watch the gap close. Cross-entropy is the cost of being wrong; KL divergence is how much of it is your fault.
- 27Information GainMove a split through a dataset and watch entropy fall. The threshold that drops it most is exactly the question a decision tree decides to ask.
- 28Basis, Span and OrthogonalityAdd vectors to a set and watch the span grow. See why orthogonal bases make projections simple and what happens when vectors depend.
- 29Singular Value DecompositionFactor any matrix into U, Sigma and V-transpose. Truncate to k components and watch the low-rank reconstruction improve as k increases.
- 30The Jacobian and the HessianCompute the Jacobian and Hessian for small functions. See how optimisers use first and second derivatives to choose direction and step size.
- 31Convexity and Optimisation LandscapesSee why a convex loss surface has one minimum and gradient descent always finds it. Then watch what happens when convexity disappears.
- 32Taylor SeriesExpand a function into a polynomial term by term. Watch the approximation improve near the centre and fail past the radius of convergence.
- 33Expectation and VarianceCalculate expected value and variance from a probability distribution. Apply the linearity and independence rules that simplify real problems.
- 34Bernoulli, Binomial and PoissonStart from a single coin flip and build the binomial and Poisson distributions step by step. See the limit that connects each to the next.
- 35The Central Limit TheoremSample from skewed, uniform and bimodal distributions. Watch the sample mean become normal as the sample size grows, and see where it fails.
- 36Quantiles and PercentilesCompute quantiles, percentiles and the interquartile range. See why rank-based summaries describe skewed data better than the mean.
- 37Sampling Distributions and Standard ErrorSimulate thousands of samples and plot the distribution of the sample mean. Watch the standard error shrink as the sample size grows.
- 38Confidence IntervalsDraw a thousand confidence intervals and count how many capture the true mean. See what the ninety-five percent actually refers to.
- 39Hypothesis Testing and p-valuesRun a hypothesis test step by step. See what the p-value actually measures and why a small one does not mean the effect is large.
- 40Type I and Type II ErrorsAdjust significance level and sample size and watch Type I and Type II error rates trade off. See what statistical power actually measures.
- 41QR Decomposition and Gram-SchmidtOrthogonalise a set of vectors with Gram-Schmidt and factor the matrix into Q times R. See why this makes least squares numerically stable.
- 42Cholesky and Positive-DefinitenessFactor a positive-definite matrix into a lower triangle times its transpose. Watch the decomposition break when definiteness fails.
- 43Matrix CalculusWork through the matrix derivative identities used in backpropagation. Use dimension analysis to verify each gradient before coding it.
- 44Lagrange MultipliersOptimise a function under a constraint by introducing a multiplier. See how the multiplier itself tells you the cost of the constraint.
- 45Jensen's InequalityApply a convex function before and after averaging and see why the results differ. Trace how this gap appears in KL divergence and the ELBO.
- 46Markov ChainsStep through a Markov chain and watch the state distribution converge to steady state. See why MCMC sampling and PageRank are both examples.
- 47Combinatorics: Permutations and CombinationsCount arrangements with and without order, with and without replacement. One decision tree picks the right formula every time.