The Transformer Architecture
A transformer block has two sublayers — attention and FFN — each followed by a residual connection. Attention handles communication between tokens. FFN handles computation within each token. Stack 32 of these and you get hierarchical abstraction: syntax at the bottom, reasoning at the top.
One block, stacked
Every chatbot you’ve used is one kind of block, copied dozens of times and stacked.
Your sentence goes in at the bottom and comes out at the top. Each block does two jobs.
Job one: talk
Job one is attention: words share information with each other.
You’ve seen this. It’s the only place in the block where words can see each other.
Job two: think
Job two: each word thinks on its own, using what attention just gathered.
A small network works on every word separately. It’s called the feed-forward layer.
Which job?
Which job does this? Tap the one you think handles it.
Ask yourself: does it need to see other words?
A running total
Neither job replaces the word. Each adds its result on top, like notes in a margin.
That running total is called the residual stream. Every block reads it and adds to it.
Low and high
Stacked blocks don’t repeat the same work. Later blocks build on what earlier ones found.
The whole stack
Your sentence climbs every block. At the top, the running total for the last word becomes the next-word scores.
That’s lesson 5’s scoreboard. The whole stack is called a transformer.
Blocks on a total
A transformer is one block, attention then feed-forward, stacked many times on a running total.
The two jobs are attention and the feed-forward layer. The running total is the residual stream.