Quick Context
In y = mx + c the slope m is one number for the whole line. A curve has no single slope — it changes everywhere. The derivative gives you the slope at each individual point.
The Trick: Two Points, Then Squeeze
Slope needs two points, so start by cheating — take your point x and another a distance h away, and find the slope of the line through both. That line is the secant:
secant slope = [ f(x + h) − f(x) ] / h
Now shrink h. The second point slides toward the first, and the secant pivots until it just grazes the curve. That limiting line is the tangent, and its slope is the derivative:
f′(x) = lim (h→0) [ f(x + h) − f(x) ] / h
Press Shrink h → 0 and watch the amber secant rotate onto the green tangent while the error collapses toward zero. That animation is the definition of a derivative.
Reading the Sign
- f′(x) > 0 — the function is rising here.
- f′(x) = 0 — flat: a peak, a valley, or a plateau. Optimisation lives and dies at these points.
- f′(x) < 0 — the function is falling.
Tick plot the derivative and slide x across the curve: wherever the original function turns around, its derivative crosses zero.
Why Machine Learning Cares
Training a model means minimising a loss. The derivative answers the only question that matters at each step: which way is downhill, and how steep is it? Gradient descent then takes a step against the slope:
w ← w − learning_rate × f′(w)
A large derivative means a steep slope and a big correction; a derivative near zero means you have arrived somewhere flat. Every optimiser on this site — SGD, Adam, the lot — is a refinement of that one line.
Interactive Exploration Guide
- Start on f(x) = x² at x = 2 with h large. The secant is visibly wrong; shrink h and the error falls to almost nothing.
- Slide to x = 0. The tangent goes flat — the derivative is exactly 0 at the bottom of the parabola, which is what an optimiser is hunting for.
- Switch to f(x) = 2x + 1. The secant equals the tangent at every h, because a straight line has the same slope everywhere. There is nothing to shrink.
- Try 3 sin(x) and slide across. The derivative swings positive and negative as the wave rises and falls, hitting zero at every crest and trough.
- Try e^(x/2) — its derivative grows in proportion to the function itself, which is the defining property of exponentials.
Key Takeaway
A derivative is the slope of the tangent, obtained by taking the slope between two points and letting the gap shrink to nothing. Positive means rising, negative means falling, zero means flat — and that sign is the compass every learning algorithm follows.