← All courses

Mathematics

The math foundation for understanding the transformer architecture and reading modern AI papers — intuition first.

Chapter 01

Linear Algebra: The Language of Transformers

  1. Vectors: Lists of Numbers You Can PictureContinue
  2. Vector Norms: How Long Is That Arrow?Coming soon
  3. Dot Product & Cosine SimilarityComing soon
  4. Matrices: A Grid (and a Stack of Vectors)Coming soon
  5. Matrix Multiplication: A Grid of Dot ProductsComing soon
  6. Matrix–Vector Products as Linear MapsComing soon
  7. Transpose: Flipping a Matrix Across Its DiagonalComing soon
  8. Identity & Inverse: Undo Buttons for MatricesComing soon
  9. Special Structure: Diagonal, Symmetric, and OrthogonalComing soon
  10. Trigonometry & Rotation (RoPE-prep)Coming soon
  11. Projections & OrthogonalityComing soon
  12. Rank & Low-Rank IntuitionComing soon
  13. Eigen (brief) & SVD: Every Matrix = Rotate → Scale → RotateComing soon
  14. Tensors, Shapes & BroadcastingComing soon
  15. einsum: One Notation to Rule Them AllComing soon
  16. Chapter examComing soon
Chapter 02

Calculus & Backprop

  1. Functions, Graphs & the Story a Curve TellsComing soon
  2. Slope: The Rate of Change You Already KnowComing soon
  3. The DerivativeComing soon
  4. Differentiation Rules I: Power, Constant & SumComing soon
  5. Exponentials & LogsComing soon
  6. Integration PrimerComing soon
  7. The Chain RuleComing soon
  8. Product & Quotient RulesComing soon
  9. Partial DerivativesComing soon
  10. The GradientComing soon
  11. Softmax & Cross-Entropy DerivativeComing soon
  12. The JacobianComing soon
  13. Computation Graphs & BackpropagationComing soon
  14. Chapter examComing soon
Chapter 03

Probability & Information Theory

  1. What Is a Probability?Coming soon
  2. Random VariablesComing soon
  3. Bernoulli & CategoricalComing soon
  4. The GaussianComing soon
  5. Expectation & VarianceComing soon
  6. Joint, Marginal & Conditional DistributionsComing soon
  7. Independence & Light BayesComing soon
  8. Maximum Likelihood EstimationComing soon
  9. Softmax as a Probability DistributionComing soon
  10. EntropyComing soon
  11. Cross-Entropy: THE LossComing soon
  12. KL DivergenceComing soon
  13. RLHF / DPO / GRPO Math (Assembly)Coming soon
  14. Sampling & DecodingComing soon
  15. Chapter examComing soon
Chapter 04

Optimization & Numerical Foundations

  1. The Loss LandscapeComing soon
  2. Gradient DescentComing soon
  3. The Learning RateComing soon
  4. SGD & Mini-BatchesComing soon
  5. MomentumComing soon
  6. AdamComing soon
  7. Convexity (Light) + CurvatureComing soon
  8. Numerical Stability I: Log-Sum-ExpComing soon
  9. Online / Streaming SoftmaxComing soon
  10. Normalization Statistics: LayerNorm / RMSNormComing soon
  11. Floating Point & Precision (Light)Coming soon
  12. Chapter examComing soon