Quick Context
A logarithm asks: what exponent turns the base into this number?
log_b(x) = y ⇔ b^y = x
log₂(8) = 3 because 2³ = 8
log₁₀(1000) = 3 because 10³ = 1000
Logs and exponentials are the same relationship read in opposite directions. Tick show b^x and you will see the two curves mirror each other across the diagonal.
The Property That Matters Most
log(p × q) = log(p) + log(q)
A logarithm converts multiplication into addition. Change the two numbers in the product demo and watch both sides stay equal. This is not a curiosity — it is why logs are everywhere in machine learning.
Probabilities multiply. Multiply a thousand numbers below 1 and you get something so small the computer rounds it to zero — numerical underflow. Take logs and that product becomes a sum of manageable negative numbers. Every implementation of maximum likelihood, cross-entropy loss and Naive Bayes relies on this exact substitution.
The Three Bases You Will Meet
- Base 2 — information theory. log₂ counts bits: how many yes/no questions to pin something down. Entropy is measured this way.
- Base e ≈ 2.71828 — the natural log, written ln. It appears everywhere in calculus because its derivative is simply 1/x. This is the default in ML loss functions.
- Base 10 — human-readable scales: decibels, pH, the Richter scale.
The base only rescales the curve — every log is a constant multiple of every other. Watch the three readouts stay in fixed proportion as you slide x.
Log Scales Tame Huge Ranges
Notice how brutally the curve flattens. Going from 1 to 10 climbs as much as going from 10 to 100, or 100 to 1000 — each multiplication by 10 adds a constant amount. Tick log-scale the x axis and those equal jumps become evenly spaced.
That compression is why loss curves are so often plotted on a log axis: it makes an improvement from 0.001 to 0.0001 as visible as one from 1.0 to 0.1.
The Domain Restriction
Logs are only defined for x > 0. As x approaches 0 the curve dives to negative infinity; at 0 and below there is no answer at all, because no power of a positive base ever produces zero or a negative number.
This causes real bugs. Cross-entropy computes −log(p), so a model that confidently predicts probability 0 for the correct answer produces infinite loss and NaNs everywhere. Every framework quietly clamps p away from zero for exactly this reason. Slide x below about 0.1 and watch the value plunge.
Interactive Exploration Guide
- Set base 2 and x = 8. The answer is exactly 3 — three doublings from 1 reaches 8.
- Keep the base and try x = 16, then 32. Each doubling of x adds exactly 1 to the log.
- Slide x toward 0. The curve dives without limit — the underflow trap in one picture.
- Change p and q in the product demo and confirm log(p×q) = log(p) + log(q) every time.
- Raise the base toward 10. The curve flattens: a larger base needs fewer powers to reach the same number.
Key Takeaway
A log is an exponent in disguise. It converts products into sums, compresses vast ranges into readable ones, and is undefined at zero — three facts that between them explain log-likelihood, cross-entropy loss, entropy in bits, and most NaN bugs in training code.