Networks train best when inputs live on a small, consistent scale. Compare raw, Min-Max, and Z-Score views of the same series — then throw in an outlier.
Technique
Replaces one sensor reading with a huge spike (420) and re-normalizes — watch what each technique does to the rest of the curve.
Series Statistics
Min
-
Max
-
Mean μ
-
Std σ
-
Formula
x' = x
Sensor Readings Over Time
RANGE: -
Raw values — note the y-axis scale. Large, drifting magnitudes make gradient updates uneven across features and slow training down.
Normalizing Sequential Data: Min-Max vs Z-Score
Why scale matters for gradient-based learning, and how to choose between the two classic techniques.
Why Networks Care About Scale
Neural networks learn by nudging weights along gradients. When one input ranges over [0, 1] and another over [100, 10000], the loss surface becomes a stretched valley: gradients explode along one axis and vanish along the other, forcing tiny learning rates and slow, unstable training. Normalization reshapes that valley into something closer to a bowl — the same step size works in every direction.
Min-Max Scaling
x' = (x - min) / (max - min)
Squashes every value into [0, 1] — predictable, bounded, great for sigmoid/tanh activations.
Fragile to outliers: a single extreme value becomes the new max, crushing every other point into a tiny sliver of the range.
For sequences, min and max must come from the training portion only — computing them over the full series leaks future information into the past.
Z-Score Standardization
x' = (x - μ) / σ
Centers the series at mean 0 with standard deviation 1 — unbounded, but statistically well-behaved.
More robust to outliers than Min-Max: a spike shifts μ and σ somewhat, but doesn't single-handedly define the scale.
The default choice for most deep learning pipelines, especially when inputs can occasionally exceed historical extremes.
Interactive Exploration Guide
Cycle through the three views. The curve's shape never changes — only the y-axis. Normalization is a re-labeling of the ruler, not a distortion of the data.
Enable the outlier with Min-Max selected. The spike becomes 1.0 and the entire rest of the series collapses toward 0 — the network would see an almost-flat line.
Switch to Z-Score with the outlier still on. The spike stands out at several σ, but the remaining points keep usable spread. That resilience is why Z-Score is usually the safer default.
Key Takeaway
Normalize sequential inputs before feeding them to a network: Min-Max when you need a bounded range and trust your extremes, Z-Score when outliers are possible. And always fit the scaler on training data only — for time series, the future must never leak into the past.
Cheat sheet
Normalization Techniques for Sequential Data
Networks train best when inputs live on a small, consistent scale. Compare raw, Min-Max, and Z-Score views of the same series — then throw in an outlier.