Deep Learning, One Layer at a Time
From a single perceptron to backpropagation, optimizers, regularisation and the shape of a training curve.
About this track
Deep learning is a small number of ideas repeated at scale. This track introduces them in the order they build: one weighted sum, then an activation, then a layer, then a loss, then the gradient that moves the weights, then everything people add to stop it going wrong.
Because each page animates a single step, you can see what a learning rate actually does to a descent path, why a training curve separates from the validation curve, and what dropout removes on each forward pass.
All 27 modules, in teaching order
- 01Perceptron ClassifierAdjust the weights and bias of a single perceptron by hand and watch its decision boundary swing across the data.
- 02Weights & BiasesSee what the numbers a network learns are actually for: weights scale each input, bias shifts where the neuron fires.
- 03Activation Functions in DLCompare ReLU, sigmoid, tanh and others on the same input, and see how each one bends the signal passing through a layer.
- 04Neural Network VisualizerBuild a neural network layer by layer and watch data flow forward through weights, biases and activations into a prediction.
- 05How Loss is CalculatedFollow a prediction through to a single loss value, and see how the error signal that drives all learning is actually computed.
- 06Gradient DescentWatch gradient descent step downhill across a loss surface, and see how the learning rate decides between converging and diverging.
- 07Linear Regression with Gradient DescentTake gradient descent one step at a time and watch the best-fit line and its mean squared error converge together.
- 08Optimizers in Neural NetworksRun several optimizers in parallel on one loss surface and see how momentum and adaptive learning rates change the route.
- 09Optimizers in 3DRace SGD, Momentum, RMSprop and Adam across a 3D loss landscape and watch which of them escape local minima and which get stuck.
- 10Learning Rate SchedulingCompare a fixed learning rate against a decaying schedule, and watch scheduling rescue a model that would otherwise overshoot.
- 11Batch Processing in NetworksCompare stochastic, mini-batch and full-batch training, and see how batch size trades gradient noise against speed.
- 12Gradient Descent Batch ProcessingChange one control at a time and watch how batch size reshapes the path gradient descent takes toward the minimum.
- 13Feature Scaling & Weight BiasSee what happens when one feature is measured in thousands and another in decimals, and how scaling rebalances the gradients.
- 14Weight Initialization MethodsCompare He, Xavier, random and zero initialisation, and watch poor starting weights kill a deep network before it learns.
- 15Vanishing & Exploding GradientsWatch gradients shrink to nothing or blow up as they multiply back through layers, and see what keeps them in range.
- 16Batch Normalization in Deep NetworksToggle batch normalisation on a deep network and watch it rescale layer outputs to keep gradients stable as they flow back.
- 17Overfitting vs UnderfittingMove model complexity from too simple to too flexible and watch the gap open between training and validation performance.
- 18Model Training CurvesRead training and validation curves like a practitioner: spot underfitting, a healthy fit and overfitting from their shape alone.
- 19Dropout RegularizationSwitch neurons off at random during training and watch dropout stop the network leaning on any single pathway.
- 20Regularization TechniquesCompare L1 and L2 penalties, and watch Lasso zero out useless weights while Ridge shrinks all of them smoothly.
- 21Early StoppingWatch validation loss bottom out then start climbing, and see early stopping halt training at the moment generalisation peaks.
- 22Hyperparameter TuningRun grid search and random search side by side on the same model and see which finds strong hyperparameters in fewer trials.
- 23Data SparsityFeed mostly-zero inputs through a network and watch entire pathways go dormant, because anything multiplied by zero stays zero.
- 24Neural Network for RegressionAdapt a neural network to predict continuous values, and see how its output layer and loss differ from a classifier's.
- 25Neural Network for Unsupervised LearningTrain a network with no labels at all and watch it uncover structure by learning to reconstruct its own input.
- 26Model ReproducibilityFix the random seed and see exactly what becomes repeatable: initial weights and the order training data is shuffled in.
- 27Model Training on CPU vs GPUInteractive visualization of CPU vs GPU in deep learning. Learn how batch size and parallelism affect training speed on VizLearn.