Generative AI & LLMs
Tokenizers, next-token prediction, and the techniques that make large models trainable and servable.
About this track
A large language model predicts one token at a time. Everything else - the tokenizer in front of it, the fine-tuning that specialises it, the quantization that fits it on a smaller card - exists to serve that loop.
This track starts with how text is split and encoded, moves through the two training objectives that produced most current models, and ends on the practical techniques: LoRA, quantization and distillation.
All 8 modules, in teaching order
- 01How LLMs Process Text?Interactive walkthrough of the LLM text pipeline - tokenization, token IDs, embeddings, positional encoding, self-attention and logits, with live computed values.
- 02Byte Pair Encoding TokenizerInteractive Byte Pair Encoding lab - train real BPE merges on your own corpus and see how any word gets segmented into subword tokens.
- 03How LLMs Predict the Next Word?Interactive lab on LLM next-word prediction - see logits become probabilities through softmax, and how temperature, top-k and top-p sampling reshape the distribution.
- 04Causal Language ModelingSee how predicting the next token, over and over, is all it takes for a model like GPT to produce fluent text.
- 05What is Masked Language Modeling?Interactive Masked Language Modeling lab - mask tokens in a sentence and see how a bidirectional model like BERT predicts them from left and right context.
- 06Quantization in LLMsInteractive quantization lab for LLMs - map FP32 weights to INT8 or INT4, see quantization error, outlier effects and real model memory savings.
- 07LoRA in LLMsInteractive LoRA lab - see how low-rank adapters replace full fine-tuning, how rank controls trainable parameters, and what a rank-limited update can represent.
- 08Knowledge Distillation in LLMsInteractive knowledge distillation lab - soften teacher logits with temperature, compare hard labels to soft targets, and watch KL divergence guide a student model.