A neuron computes w·x + b. Try handing it the word “cat” and see exactly where — and why — the math breaks.
y = w · x + b
w = 0.5 b = 0.1
Multiplication and addition. That's all a neuron can do — and you cannot multiply a word.
The most fundamental fact in NLP: neural networks are arithmetic machines, and words are not numbers.
Every neuron in every network — from a 1958 perceptron to GPT — computes some flavour of w · x + b. Multiplication is only defined for numbers. Therefore, before any text can enter any network, it must be converted into numbers. That conversion is text encoding, and it isn't an optimization — it's a precondition.
Try to compute 0.5 × "cat" + 0.1 in any language and you get an error or NaN — the arithmetic is simply undefined. In practice a framework like PyTorch refuses at the door: tensors hold floats, not strings. The demo above makes this failure visible instead of hiding it in a stack trace.
Insert one stage — a tokenizer plus a lookup table assigning each word an ID — and the identical neuron suddenly works: 0.5 × 2 + 0.1 = 1.1. Every downstream marvel (embeddings, attention, generation) rests on this humble dictionary lookup. The quality of the numbers matters enormously too, which is why the next modules explore encoding techniques and embeddings.
Text encoding isn't a preprocessing nicety — it is the bridge without which no NLP is possible. A neural network can no more process the raw word "cat" than a calculator can. First we turn language into numbers; everything else in NLP is about turning it into good numbers.
Every neuron in every network — from a 1958 perceptron to GPT — computes some flavour of w · x + b. Multiplication is only defined for numbers. Therefore, before any text can enter any network, it must be converted into numbers. That conversion is text encoding, and it isn't an optimization — it's a precondition.