Embedding Explorer
An AI model turns a word, a picture, a sound or a video into a list of 768 numbers, called an embedding. Things with similar meanings get similar lists — even when one is a word and the other is a photo.
1. Load the model
The model (Google's EmbeddingGemma 2) runs inside this browser, so what you add stays on this computer. It downloads once and is then remembered.
Timings and details
2. Add things
Type a word or sentence, or add a picture, sound or video. You can also drop files onto this page.
How the words are given to the model
The model works best when text starts with a short instruction. Each word or sentence you add is actually sent as:
The first works well for matching words to pictures and sounds; the second for comparing sentences with each other. Try switching and watch the numbers change.
3. Star view
Each of the numbers is drawn as a ray from the center: green for positive, red for negative, longer for bigger. Pick a second thing to see the difference between the two.
Why do all the stars look alike?
Every embedding this model makes contains a big shared part that is almost the same for a word, a photo or a sound. In the example set, that shared part is about two thirds of each list, and just six positions (such as numbers 111 and 115) carry almost half of it. That is why every star has the same few long rays, and why even “idea” and “horse” come out 0.70 alike.
Tick Take away what everything shares at the top. It subtracts the average of everything you have added, leaving what makes each thing different. The stars stop looking alike, unrelated things score near 0, and related things such as “unicorn” and “horse” stay well above 0. The average depends on what you have added, so adding more things changes it.
Even then, 768 rays are too many to tell apart by eye. Use the first 128 numbers as well and the differences are easy to see.
A few rays may still reach the edge: the rays are scaled to fit 99% of the numbers, and the few bigger ones are cut off.
Why can I use fewer numbers?
This model was trained so that the first numbers carry the most important information. Keeping only the first 256 of the 768 loses very little; keeping 128 loses more, especially for pictures and sounds. The setting at the top changes every section.
4. Who is close to whom?
Each square compares two things: 1 means the same direction, 0 means unrelated. Darker means closer.
Something to notice
Compare a word with a picture of the same thing, and two words with each other. Words tend to be closer to other words, and pictures to other pictures, even when the meanings match. Researchers call this the modality gap. So look at which picture is closest to a word, not just at the size of the number.
Most squares are between 0.5 and 0.8 because of the part every embedding shares (see “Why do all the stars look alike?” above). Tick Take away what everything shares at the top and the numbers spread out around 0.
The number is the cosine similarity: multiply the two lists number by number and add everything up (each list has been scaled to length 1 first).
5. Embedding arithmetic
Lists of numbers can be added and subtracted. What is closest to the result?
6. Your own 3D space
Choose three directions, each from one word to another. Everything you added is placed along them. Drag to turn the space.
How the places are worked out
The direction from “small” to “big” is the embedding of “big” minus the embedding of “small”. A thing's place along that direction is how much its own embedding points the same way. The places are then centered and stretched to fill the box, so only where things are compared with each other matters.