How can “dog” be close to “puppy”?
A text system cannot use the feeling of relatedness directly. It needs numbers it can compare, combine and use as input for the next task.
Short version: words become places in a learned space.Click a point, change the lens.
00 Overview / start here
Give meaning a place to live.
Computers do not begin with a human sense of “cat” or “queen.” They need a numerical way to carry meaning and compare one item with another.
A small address for a token.
An embedding turns a token into a dense list of numbers, placing related items near each other in a learned vector space.
the basic storyinput → operation → output
- 01 / inputA token goes in.
“puppy” is represented by an ID: a label that tells the model which row to look up.
- 02 / operationA vector gets looked up.
The ID selects a dense list of learned values. Nearby values can capture shared context.
- 03 / outputA usable position comes out.
The model can measure distance, blend vectors or pass the numbers into another prediction.
puppy → [0.82, 0.78, 0.27]
These three visible numbers are a toy vector. In this illustration, the first two help put “puppy” near “dog”; the last helps keep it in the pet neighborhood.
02 Unpack one lookup
From token to vector.
The ID is only a label. The vector is the useful payload: a learned numeric signature that can move as context changes.
Make the lookup visible.
Choose a token and a context lens. Watch the short vector, map position, neighbors and explanation change together.
An ID points to a row. The vector carries the relationship.
03 Combine directions
Numbers can hold a relationship.
Once words are vectors, simple operations can expose a direction. This famous analogy is useful as a toy idea, not a universal rule.
Walk through a toy analogy.
Use the stepper to reveal each operation. The values are invented for teaching, but the arithmetic is real.
STEP 01 / 04kingStart with a toy vector for “king”.
king[0.90, 0.80, 0.70]
−man[0.70, 0.60, 0.50]
+woman[0.60, 0.70, 0.50]
≈queen[0.80, 0.90, 0.70]
current result[0.90, 0.80, 0.70]king − man + woman is still hidden
Labels for lookup.
An ID such as 0412 does not mean “cat” by itself. It only points to the right row.
Numbers for relationships.
The dense vector is the usable representation: compare it, combine it or send it onward.
Context shapes the row.
Embeddings are learned from context, so words appearing in similar surroundings can become close.
The map is learned from usage. Context is what gives the coordinates meaning.
FAQ Embeddings / quick answers
Give words
an address.
Short answers for the moment when a vector starts to feel less abstract.
01 / definitionWhat is an embedding?
An embedding is a learned numeric vector used to represent a token. Its coordinates are not a dictionary definition; they give later calculations a way to compare and combine patterns.
02 / distanceWhy can related words be close together?
Embeddings are learned from context. Tokens that appear in similar surroundings can receive vectors that are close in the model’s space, which makes similarity useful to downstream math.
03 / limitsAre close vectors always truly similar?
No. Distance depends on the model, data, dimensions and context. A 2D map or a word analogy is a teaching lens, not a universal measure of meaning.