How AI Understands Meaning
Token IDs are just addresses. The real meaning lives in embeddings — long lists of numbers where similar words sit close together, and relationships between words become directions you can calculate.
The raspberry answer
This time it gets it right.
But to the AI, strawberry is only the number 73700. How can a number mean “fruit”?
Numbers mean nothing
It can’t. Look at the numbers next door.
Neighbouring numbers, nothing in common. The number is just a name tag.
A map of meaning
So the AI also gives every piece a spot on a map.
Neighbours
Words that are used the same way end up as neighbours.
Nobody drew this map. The AI moved words around while reading huge amounts of text, until words used alike sat close together.
Place the word
Where would this word live?
Tap the neighbourhood where you think it lives.
Back to raspberry
That’s part of how it answered. On its map, raspberry lives right next to strawberry.
A map of meaning
Every piece gets a spot on a map of meaning. Neighbours mean similar things.
That spot is called an embedding.