Generative AI & LLMs
Tokenizers, next-token prediction, and the techniques that make large models trainable and servable.
About this track
A large language model predicts one token at a time. Everything else - the tokenizer in front of it, the fine-tuning that specialises it, the quantization that fits it on a smaller card - exists to serve that loop.
This track starts with how text is split and encoded, moves through the two training objectives that produced most current models, and ends on the practical techniques: LoRA, quantization and distillation.
A large language model predicts one token at a time, and everything around it exists to make that prediction useful: the tokenizer in front, the training objective behind, the retrieval that grounds it, and the quantisation that makes it servable.
What you will be able to do
- Explain how text is split into tokens and why byte-pair encoding handles unseen words.
- Distinguish causal from masked language modelling, and say which models each produced.
- Describe what a context window costs and what a KV cache is caching.
- Build a retrieval pipeline and evaluate it with the metric that matches the task.
- Say what LoRA, quantisation and distillation each trade away in exchange for what.
How the track is ordered
Tokenization first, because it determines what the model can represent. Then the two training objectives - causal and masked - that account for most current models, followed by the mechanics of generation: next-token prediction, context windows and KV caching. Retrieval comes next and takes up much of the track, from embeddings and vector search through chunking, hybrid retrieval, reranking and the retriever variants, ending on how to evaluate any of it. Adaptation and efficiency - fine-tuning, RLHF, LoRA, quantisation, distillation - close the track. The NLP track is the prerequisite.
Where this leads
Retrieval quality is mostly an embeddings-and-evaluation problem, so the NLP and machine learning tracks both feed back into it. If you are building rather than studying, the retrieval modules are the ones that change day-to-day results the most.
All 52 modules, in teaching order
- 01How LLMs Process Text?Follow a prompt through tokenisation, embedding, positional encoding, transformer layers, softmax and sampling — one token out.
- 02Byte Pair Encoding TokenizerMerge the most frequent character pair, repeat, and build the subword vocabulary GPT-style models use to tokenise any text.
- 03How LLMs Predict the Next Word?See logits become probabilities through softmax, and watch temperature, top-k and top-p sampling reshape which word the model picks.
- 04What are Queries, Keys and Values in an LLM?Trace a single attention step: see how a query matches a key, why the dot products are scaled, and what the weighted sum of values produces.
- 05Causal Language ModelingSee how predicting the next token, over and over, is all it takes for a model like GPT to produce fluent text.
- 06What is Masked Language Modeling?Hide 15 percent of tokens and train the model to reconstruct them using both sides of the sentence — BERT's pretraining objective.
- 07Quantization in LLMsSnap 32-bit weights to 8-bit or 4-bit levels and fit a 70B-parameter model on a single consumer GPU, with the accuracy cost shown.
- 08LoRA in LLMsFreeze the base model and train two thin matrices whose product approximates the full weight update — 10,000x fewer parameters.
- 09Knowledge Distillation in LLMsTrain a small student on the teacher's soft probability distribution and recover most of the accuracy at a fraction of the size.
- 10Embeddings and Vector SearchMeaning becomes geometry. Run an exact nearest-neighbour search, then a partitioned one, and watch comparisons collapse while recall quietly slips.
- 11Dot Product vs Cosine Similarity for RetrievalStretch a document vector and watch dot-product ranking promote it for being longer, while cosine ranking stays put.
- 12TF-IDFCompute TF-IDF scores by hand, watch common words shrink and rare words rise, and see how a simple formula still powers keyword-based search.
- 13BM25 and Sparse Lexical RetrievalThe keyword-matching half of search, computed live. Drag length normalization to zero and watch a verbose repetitive document dominate purely on term count.
- 14Indexing in vector databasesSee why brute-force comparison breaks at scale, survey the index families vector databases use, and understand what each one trades for speed.
- 15ANN indexing: HNSW and IVFBuild an HNSW graph and an IVF index from the same vectors, run a nearest-neighbour query on each, and compare the recall-speed tradeoff.
- 16Flat IndexWalk through a brute-force flat index, measure its exact-match guarantee, and learn when exhaustive search still beats every approximate method.
- 17IVF-FlatCluster vectors into Voronoi cells, search only the nearest few, and watch nprobe trade recall for speed inside an inverted file index.
- 18HNSWBuild an HNSW graph layer by layer, trace a search from the top level down, and see how ef and M shape the speed-vs-recall trade-off.
- 19AnnoyBuild a forest of random projection trees, query a point, and see how Annoy trades tree count for recall when finding nearest neighbours.
- 20Product QuantizationSplit a high-dimensional vector into subvectors, quantize each one separately, and watch product quantization shrink a billion vectors into RAM.
- 21IVF-PQCombine inverted file partitioning with product quantization, trace a query through both stages, and see why IVF-PQ scales to a billion vectors.
- 22DiskANNBuild a DiskANN graph on a billion vectors, serve queries from SSD with a small memory footprint, and see how the alpha parameter controls recall.
- 23ScaNNSee how anisotropic vector quantization preserves inner-product rankings better than standard methods, and why ScaNN wins on large-scale benchmarks.
- 24Chunking Strategies for RAGSlide the chunk size down and watch fixed-size splitting cut sentences in half, while semantic chunking respects paragraph boundaries and never does.
- 25Recursive chunkingSplit a document with recursive character chunking, watch it try each separator in turn, and see why this simple strategy is the default splitter.
- 26Structure-aware chunkingParse headings, lists and tables before splitting. See how structure-aware chunking keeps logical sections intact and beats character-based splitters.
- 27Semantic chunkingSplit text where the meaning shifts instead of where a character count runs out. Compare semantic chunking to fixed-size splits on real paragraphs.
- 28Context-aware chunkingChunk a document by headings and semantic boundaries instead of fixed token counts and see the retrieval accuracy gained by preserving context.
- 29Parent Document RetrieverSearch small chunks so the match is precise, then hand the model the whole parent document so the answer has enough context to be worth reading.
- 30Retrieval-Augmented Generation (RAG)Ask about a product no model has heard of. Watch retrieval score every chunk, paste the winners into the prompt, and turn a guess into a cited answer.
- 31Retrieval Evaluation MetricsThe same five relevant documents out of ten retrieved, in three different orders. Precision and recall cannot tell them apart - MRR and nDCG can.
- 32What is Hit Rate@k?Calculate Hit Rate at k step by step, compare it to Recall at k, and see when a binary hit-or-miss metric is the right choice for retrieval.
- 33What is Recall@k?Calculate Recall at k for a retriever, compare it to Precision at k, and understand why recall is usually the metric that gates RAG quality.
- 34What is Precision@k?Calculate Precision at k for a ranked result list, compare it to Recall at k, and see why the fraction of relevant hits matters more in RAG.
- 35What is MRR (Mean Reciprocal Rank)?Compute MRR by hand on ranked lists, compare it to Precision at k, and see why the rank of the first correct result is often the right metric.
- 36Hybrid Search: Dense + Sparse (Reciprocal Rank Fusion)Run vector search and BM25 side by side and watch them disagree, then fuse the two rankings and see a document neither method alone ranked first win on consensus.
- 37Re-ranking: Bi-Encoders vs Cross-EncodersRetrieve six candidates fast with a bi-encoder, then re-score them with a slower cross-encoder that reads query and document together and catches what was missed.
- 38Maximal Marginal Relevance (MMR)Pick the next document by relevance minus similarity to what you already picked, and watch three near-duplicate chunks stop crowding out everything else.
- 39Query Rewriting and HyDEA terse question shares few words with the passage that answers it. Expand the query or embed a hypothetical answer and watch it climb.
- 40Multi-Query RetrieverOne question asked several ways. Each phrasing retrieves separately and the union catches the document a single wording would have missed.
- 41Self-Query RetrieverSplit a question into a semantic query and a metadata filter, so "after 2020" becomes a rule that is enforced rather than a phrase that is matched.
- 42Corrective RAG (CRAG)Add a relevance filter between retrieval and generation, discard off-topic documents, and see how Corrective RAG stops answers built on noise.
- 43Context Windows and the KV CacheFill a context window and watch what gets pushed out, then switch off the KV cache and watch the work per token go quadratic.
- 44Fine-Tuning vs RLHF and DPOThree stages, three jobs. Watch a model's answers move under supervised fine-tuning, then under preference optimisation, and watch reward hacking happen.
- 45Hallucination and GroundingA model always has probability mass somewhere, even when it knows nothing. Watch it move as evidence, temperature, retrieval and an abstain option are added.
- 46Groundedness in LLM evaluationMeasure whether an LLM answer stays within the retrieved context. Compare groundedness to correctness and see how evaluators score each claim.
- 47Correctness in LLM evaluationEvaluate a model answer for factual correctness against a reference, compare automated and human scoring, and see why this dimension costs most.
- 48Relevance in LLM evaluationScore whether an LLM answer actually addresses the question asked. Compare answer relevance to context relevance and see how evaluators separate them.
- 49Completeness in LLM evaluationScore a model answer for completeness against a reference, see why a correct response can still miss required points, and measure the gap.
- 50Query, embedding and prompt cachingAdd query, embedding and prompt caches to a RAG pipeline, measure the latency and cost each one removes, and see where staleness matters.
- 51Permission filtering in RAG retrievalStop a RAG system from surfacing documents the user cannot see. Compare pre-filter and post-filter approaches and weigh speed against safety.
- 52Distributed retrieval and shardingScale vector search across machines. Compare sharding strategies, see how routing works, and spot the latency-throughput trade-off at every split.