Natural Language Processing
How text becomes numbers: characters, tokens, embeddings, recurrence and attention.
About this track
Everything a language model does rests on one uncomfortable fact: the model never sees words. This track follows a piece of text all the way down - characters to codes, codes to tokens, tokens to vectors - and then back up through the architectures that read those vectors in order.
It covers the recurrent family in full: what a recurrent cell holds, why backpropagation through time makes long sequences hard, and how LSTM's four gates were designed to fix exactly that. It ends on GRUs and bidirectional layers.
All 36 modules, in teaching order
- 01ASCII Character CodesType any character and see the ASCII number a computer actually stores, the very first step from text into numbers.
- 02NLP TokenizerSplit text into words, characters or sentences and see how the tokenizer you choose changes what a model actually reads.
- 03Text Normalization PipelineRun raw text through lowercasing, punctuation stripping and stop-word removal, and watch each stage clean it further.
- 04Stemming vs LemmatizationCompare a stemmer chopping suffixes against a lemmatizer resolving proper dictionary forms of the very same words.
- 05N-gram ExplainerSlide an n-gram window across text and see how counting short sequences lets a model guess which word comes next.
- 06Word Cloud GeneratorPaste any text and watch a word cloud size each term by how often it appears, exposing what a document is really about.
- 07Why Text Encoding is Needed in NLPFeed raw text into a neuron and watch the math break — then encode it and watch it work. An interactive first-principles demo on VizLearn.
- 08Text Encoding Techniques in NLPCompare Label Encoding, One-Hot, Bag-of-Words and TF-IDF on a live corpus you can edit, side by side, on VizLearn.
- 09How Words are Represented in Neural NetworksFollow a word through the full representation pipeline — token ID, one-hot vector, dense embedding — interactively on VizLearn.
- 10What are Embeddings?Explore a 2D map of word embeddings: click words to measure similarity and run the famous king − man + woman analogy, on VizLearn.
- 11How are Embeddings Generated?Train word embeddings live in your browser: watch random vectors organize themselves into semantic clusters as co-occurring words pull together. On VizLearn.
- 12How Neural Networks Process TextFollow a sentence from raw characters through tokens and vectors into something a neural network can actually multiply.
- 13What is a Sequence?Understand what sequential data is and why order matters, with an interactive shuffle experiment on VizLearn.
- 14Sequential Data Preparation with Sliding WindowSee how a sliding window turns a raw sequence into supervised training pairs (X, y), step by step, on VizLearn.
- 15Normalization Techniques for Sequential DataCompare Min-Max scaling and Z-Score standardization on a live time series, and see how outliers distort each, on VizLearn.
- 16Limitations of ANN with Sequential DataDiscover why plain feed-forward neural networks struggle with sequences: fixed input size, order blindness, and no memory. Interactive demos on VizLearn.
- 17What is a Recurrent Cell?Step a sentence through a recurrent cell one word at a time and watch the hidden state — the network's memory — evolve. On VizLearn.
- 18How RNN Processes TextFeed a sentence word by word into a recurrent network and watch the hidden state carry context from each step to the next.
- 19RNN ArchitectureLearn what a standard RNN cell is with a beginner-friendly interactive visualization of its memory states and data flow.
- 20Interactive 3D RNN VisualizerSee how a recurrent neural network processes a sequence step by step, carrying hidden state from one time step to the next.
- 21Backpropagation Through TimeUnroll an RNN, run the forward pass, then watch gradients flow backwards through every timestep — with the chain-rule product computed live. On VizLearn.
- 22Vanishing Gradient Problem in RNNPlay with the recurrent weight and sequence length to see gradients vanish or explode — with live decay curves and a log-scale view. On VizLearn.
- 23What is LSTM?Learn what an LSTM cell is with a beginner-friendly interactive visualization of its internal gates, memory states, and data flow.
- 24Forget Gate in LSTMInteractive walkthrough of the Forget Gate in an LSTM cell - see how a sigmoid between 0 and 1 erases or preserves the cell state, and how forgetting compounds over time.
- 25Input Gate in LSTMInteractive walkthrough of the Input Gate in an LSTM cell - see how the sigmoid gate controls which candidate values are written into the cell state.
- 26Candidate Memory in LSTMInteractive walkthrough of the candidate memory layer in an LSTM - see how tanh proposes new signed content for the cell state and why it saturates.
- 27Output Gate in LSTMInteractive walkthrough of the Output Gate in an LSTM cell - see how it filters the cell state through tanh to produce the hidden state that the rest of the network sees.
- 28How LSTM Processes TextWatch an LSTM carry a cell state for long-term memory and a hidden state for short-term as it reads through a sequence.
- 29What is a GRU?Step a sentence through a GRU and watch the update and reset gates blend old memory with new input. Compare parameter counts with LSTM. On VizLearn.
- 30What is a Bidirectional Layer?Learn what a Bidirectional layer is with a beginner-friendly interactive visualization of forward and backward passes.
- 31The Attention MechanismSee why a fixed context vector forgets, and how attention lets the decoder re-weigh every input word at each output step. Full alignment matrix.
- 32Query, Key and ValueFollow one attention lookup end to end - score, scale, softmax, blend - and watch softmax saturate when the 1/sqrt(d_k) scaling is removed.
- 33Self-AttentionThe full n x n attention matrix over one sentence. Watch a pronoun resolve at layer 2, apply a causal mask, and see why word order needs encoding.
- 34Multi-Head AttentionRun several attention patterns at once, each tracking a different relationship. See why more heads cost no extra parameters - the dimension is split, not added.
- 35Positional EncodingSelf-attention is blind to word order. See sinusoidal encoding give every position a bounded unique fingerprint, and watch the naive schemes fail.
- 36The Transformer ArchitectureAttention, add, normalise, feed-forward, add, normalise - stacked. Switch the residual connections off and watch a 24-layer stack stop being trainable.