What Are Embeddings? How Meaning Becomes Math

A computer cannot read the word "cat." It has no idea what a cat is. It can do exactly one thing well, which is arithmetic, and the entire modern AI stack is a series of increasingly clever tricks for turning things that are not numbers into numbers a computer can do arithmetic on.

Embeddings are the most important of those tricks, and the most quietly useful. They are how meaning becomes math. Once you understand them, a large part of AI that looked like magic, semantic search, recommendations, and the retrieval half of every RAG system, turns into something you can actually reason about. So let me give you the whole idea, cleanly.

The core idea: meaning as a location

Picture a map. On a good map, things that are near each other in the world are near each other on the paper. An embedding is that idea, applied to meaning instead of geography, and in far more than two dimensions.

An embedding model takes a piece of text and outputs a vector: a long list of numbers, often hundreds or thousands of them. You should not try to read those numbers; individually they mean nothing. What matters is where the vector sits relative to others. The model is trained so that texts with similar meaning end up as points close together, and texts with different meaning end up far apart.

"The cat sat on the mat" and "a feline rested on the rug" share almost no words. To a keyword search they are strangers. As embeddings they are neighbours, because the model learned they mean nearly the same thing. That is the whole trick, and it is a genuinely deep one: meaning becomes a position, and similarity becomes distance.

The demonstration that made people believe it

The moment this stopped being abstract for most people was a result from 2013. Mikolov and colleagues at Google built word embeddings (the method was called word2vec) and showed something startling: the relationships between words had become geometry you could do arithmetic on.

Take the vector for "king," subtract the vector for "man," add the vector for "woman," and the point you land on sits closest to "queen." The model was never taught that. It fell out of the geometry, because the model had arranged words so that the direction encoding "royalty" and the direction encoding "gender" were consistent across the space.

Modern embedding models are far more powerful than word2vec and work on whole sentences and documents, not single words. But the founding intuition is identical: arrange things in space so that meaning is a location.

Why this is so useful in practice

Here is the payoff, and it is bigger than it first sounds. Once meaning is a location, the question "what is similar to this?" becomes "what is nearest to this point?", and that is a question a computer answers instantly.

That single capability is the engine under a surprising amount of what you use:

Semantic search. Search by meaning instead of keywords. A user asks "how do I cancel my subscription" and you find the article titled "ending your membership," even with no words in common.

Retrieval-augmented generation (RAG). This is the big one. Before a language model answers a question about your documents, you embed the question, find the nearest chunks of your documents, and hand those chunks to the model as context. The retrieval step, the part that decides what the model gets to see, is embeddings doing nearest-neighbour search. Get that wrong and the smartest model gives you a confident wrong answer.

Recommendations, clustering, deduplication, classification. Similar users, similar products, near-duplicate documents, grouping tickets by theme. All of them are distance in embedding space.

How you actually use it

The workflow is refreshingly simple, which is part of why embeddings are the workhorse.

You embed your documents once and store the vectors, usually in a vector database built to find nearest neighbours quickly. When a query comes in, you embed the query, measure its distance (cosine similarity is the usual metric) to the stored vectors, and return the closest matches. No model training. No GPU cluster. It runs cheaply, and it runs fast.

That last point connects to the companion piece in this series on reasoning models. Reasoning is the expensive, slow tool you reach for on hard problems. Embeddings are the opposite: cheap, fast, and used constantly in the background. Knowing which is which is most of practical AI engineering.

Why this matters if you build on a budget

Embeddings are the most accessible power tool in AI, and that is exactly why they matter most for practitioners without a hyperscaler's resources.

You do not need to train anything. You do not need frontier compute. You need your own data, an embedding model, and somewhere to store vectors, and you can build semantic search or retrieval over a corpus nobody else has: a ministry's documents, a clinic's records, a local-language archive. That is the moat this publication keeps returning to, made concrete. Your data, embedded, is a system a global lab cannot replicate, because it does not have your data.

What would make me wrong

Embeddings capture correlation, not truth. The model learned what tends to appear near what, in the data it was trained on, and that is not the same as what is correct or fair. Three cautions follow, and they are not small.

Similar is not the same as relevant. The nearest vector is sometimes the wrong answer, and a retrieval system that returns "close" is not returning "right." Embeddings inherit every bias and blind spot of the model that produced them, which is why the same word2vec geometry that gives you king to queen also encodes uglier analogies nobody wants. And coverage is uneven: most embedding models are far weaker on low-resource languages than on English, which matters enormously if you are working in Krio or Swahili or Wolof, and is a limitation to test for rather than assume away. The tool is powerful. It is not neutral, and it is not always right.

Key takeaways

1. An embedding turns meaning into a location, a vector, so that similar things sit close and different things sit far apart.
2. Similarity becomes distance, which lets a computer answer "what is like this?" with fast arithmetic.
3. This is the engine under semantic search, recommendations, and the retrieval half of RAG.
4. It is cheap and needs no training, which makes it the most accessible power tool in applied AI.
5. It captures correlation, not truth, inherits the model's biases, and is weaker on low-resource languages. Verify relevance, do not assume it.

What is a corpus you have access to that nobody else does? That is the one to embed first.

---

Source for the word2vec analogy: Mikolov et al., "Efficient Estimation of Word Representations in Vector Space" and related 2013 work introducing word2vec, which demonstrated that analogies such as king minus man plus woman land nearest to queen via cosine similarity (with input words excluded from the nearest-neighbour search).