Embeddings (Deep)
Static embeddings give every word one fixed vector — which breaks for words like "bank" that mean different things in different sentences. Transformers fix this with contextual embeddings: attention reshapes each word's vector based on its neighbours, resolving ambiguity automatically.
Found by meaning
That note never says “car” or “trouble”. So how was it found?
Not by matching words. By matching meaning, using the map you met in lesson 3.
A spot is numbers
A spot on the map is really a list of numbers, one per direction.
Our map has two directions. Real models use hundreds or thousands, so the list is long.
Closeness is measurable
With numbers, “similar meaning” becomes a distance you can measure.
Compare two lists and you get one score for how close they are. No dictionary needed.
Place the word
Where would this word live?
Tap the neighbourhood where you think it lives.
The bank problem
But one fixed spot fails for “bank”. It lands halfway, close to neither meaning.
Older embeddings worked exactly like this: one spot per word, whatever the sentence.
The spot moves
Inside the model, the spot moves. Attention pulls it toward the meaning the sentence needs.
Back to the search
Search tools give each whole note one spot, and your question one too.
They return the notes whose spots sit closest to your question. That’s why matching words wasn’t needed.
Spots that move
An embedding is a list of numbers: a spot on the map. Inside the model, context moves it.
A spot that depends on the sentence is a contextual embedding. Finding text by closeness is semantic search.