← All courses

Transformers & LLMs

Build a real understanding of the transformer architecture, then train and fine-tune LLMs — theory and practice.

Chapter 01

The Transformer Architecture

  1. The Big Picture: What a Transformer DoesComing soon
  2. Tokenization: Text → IntegersComing soon
  3. Embeddings: Tokens → VectorsComing soon
  4. Positional Information: Order MattersComing soon
  5. Attention I: Query, Key, ValueComing soon
  6. Attention II: Scale, Mask, Softmax → WeightsComing soon
  7. Multi-Head AttentionComing soon
  8. The Feed-Forward Network (MLP)Coming soon
  9. Residual Connections & NormalizationComing soon
  10. The Full Transformer BlockComing soon
  11. The Output Head: Logits → Next TokenComing soon
  12. Putting It Together: A Forward Pass, End to EndComing soon
  13. Chapter examComing soon
Chapter 02

Building & Training a Small LLM

  1. Next-Token Prediction as ClassificationComing soon
  2. The Data Pipeline: Text → BatchesComing soon
  3. Backprop Through the TransformerComing soon
  4. The Optimizer: AdamW in PracticeComing soon
  5. Initialization & Numerical StabilityComing soon
  6. The Training LoopComing soon
  7. Watching a Model LearnComing soon
  8. Sampling & Generation RevisitedComing soon
  9. Scaling Laws: Predicting Loss Before You TrainComing soon
  10. Capstone: Your Tiny LLMComing soon
  11. Chapter examComing soon