Table of contents
What this series promises
Understand how LLMs (ChatGPT, Claude, etc.) work without needing a PhD in mathematics. Each episode starts from a simple question, explains the "why" before the "how", and uses Rust examples where they help.
Why "from scratch"?
Because reading articles is not enough. It's like cooking: watching a recipe doesn't teach you how to cook. Coding the 4 operations of attention yourself will teach you more than hours of reading.
What you will learn
- How a word "looks at" the other words (attention)
- Why Q, K and V are kept separate (specialization)
- How meaning is turned into geometry (embeddings)
- How to reuse an existing model (transfer learning)
- How it all comes together to build a RAG
Who is this series for?
Anyone curious who can read simple code and wants to understand AI without getting lost in jargon. You don't need to be a mathematician, just patient.
The common thread
A single idea runs through the whole series: meaning can be represented by vectors, and those vectors can be compared by how close they are. Everything else follows from that.
Read next
- Step 2 of 7•5 minEpisode 1: Attention Is Just 4 Mathematical Operations
Behind Transformers and the attention mechanism there are only four simple operations: matrix multiplication, transposition, scaling and softmax. Here they are, in Rust.
- Step 3 of 7•2 minEpisode 2: Q, K, V — Why 3 Matrices Instead of One?
- Step 4 of 7•3 minEpisode 3: Embeddings — Absolute at the Start, Contextual at the End