#attention

3 articles about attention

Filter by topic

All #advanced #ai #allocation #associated-types #attention #azure #basics #beginners #blanket-implementations #borrow-checker #borrowing #bounds #box #c #cache #closures #code-duplication #collections #concurrency #cosmos-db #dangling-pointer #data-races #deadlock #debugging #desktop #developer-tools #dioxus #dispatch #drain #drop #dynamic-dispatch #embedded #embeddings #event-grid #fine-tuning #fnmut #gc #generics #git #heap #higher-order-functions #integration #io-drivers #iterators #javascript #kql #library-design #lifetimes #llm #log-analytics #logic-apps #lora #machine-learning #memory #monomorphization #object-safety #observability #operators #optimization #ownership #performance #plugins #rag #retain #rust #service-bus #simd #sized #stack #standard-library #str #string #supertraits #trait-bounds #trait-coherence #traits #transfer-learning #transformers #type-safety #typescript #unsized #vec #vector-database

llmSeptember 27, 2026

Episode 1: Attention Is Just 4 Mathematical Operations

Behind Transformers and the attention mechanism there are only four simple operations: matrix multiplication, transposition, scaling and softmax. Here they are, in Rust.

llmSeptember 27, 2026

Episode 2: Q, K, V — Why 3 Matrices Instead of One?

Query, Key, Value: why real models project each word into three different spaces, and what that separation of roles changes for attention.

llmSeptember 27, 2026

Episode 3: Embeddings — Absolute at the Start, Contextual at the End

Is an embedding the word on its own or the word in context? Both, at different moments. Plus the three engineering details around attention: multi-head, masking and position encoding.