A recurrent cell is a neuron with memory: it reads one token at a time and carries a hidden state forward. Step through a sentence and watch the memory change.
hₜ = tanh(W·xₜ + U·hₜ₋₁ + b)
xₜ — embedding of the current token
hₜ₋₁ — memory carried from the previous step
W, U, b — the same weights, reused at every step
Hidden state hₜ — the network's memory (4 dims)
How a single loop turns a stateless network into a sequence processor.
A feed-forward layer computes its output from the current input and forgets it immediately. A recurrent cell adds a single wire: its own previous output, hₜ₋₁, feeds back in alongside the new input. That loop gives the network a hidden state — a running summary of everything it has seen so far.
hₜ = tanh(W·xₜ + U·hₜ₋₁ + b)
The recurrent cell fixes all three ANN limitations at once: it consumes sequences of any length (one token at a time), it respects order (each step builds on the last), and it has memory (the hidden state). Training it, however, requires backpropagating through that loop — the subject of the next module.
A recurrent cell is a neuron with memory: it reads one token at a time and carries a hidden state forward. Step through a sentence and watch the memory change.