What Is an Embedding?

🧩 What the numbers are, and what distance means

An embedding model maps an input to a vector of fixed dimension — several hundred to several thousand numbers, the same count for every input regardless of its length. Two vectors are compared with a similarity measure, most often cosine similarity, which reads the angle between them and ignores their magnitude.

"Similar" means whatever the embedding model was trained to make similar, which is a stronger caveat than it sounds. A space trained on question-and-answer pairs places a question near its answer. A space trained on paraphrase pairs places a sentence near its restatements. These are different geometries with different neighbourhoods, and a vector produced by one is not comparable with a vector produced by the other.

Two consequences of the fixed dimension are worth drawing out, because both are usually discovered after an index has been built. The first is that length is not preserved: a single clause and a long passage arrive at vectors of the same size, so the longer one has compressed far more material into the same number of values, and what it compressed away is not inspectable from the vector. The second is arithmetic housekeeping — many embedding models return vectors already scaled to unit length, and where that holds, cosine similarity and the plain dot product produce the same ordering. It matters only when a store is configured for one measure and filled with vectors built for the other, at which point the ranking is wrong and nothing about it looks wrong.

The last row causes most production incidents involving embeddings. An index quietly describes the documents as they were on the day it was built, and nothing about a stale vector looks stale.

🗃️ How the Registry classifies an embedding task

Computing vectors for a catalogue of product descriptions and storing them in an index is filed as data processing here: an existing body of text is transformed into derived quantities, and those quantities are what every downstream stage reads.

The Registry's canonical brief for this entry is filed as: Compute embedding vectors for a catalogue of product descriptions and store them in a vector index. Submitted for assessment it is classified as Data processing, and its wording is hashed once — to 26f694dcac542123…, the first sixteen of sixty-four hexadecimal characters — with the wording itself never stored. The hash is what the derivation reads. That class's own page is /tasks/analysis.

Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.

❓ Questions about embedding

What is an embedding?

An embedding is a fixed-length vector of numbers representing a piece of text or another input, produced so that inputs with similar meaning are close together in the vector space. The vector supports comparison with other vectors and nothing else.

What is cosine similarity?

Cosine similarity measures the angle between two vectors, ignoring their lengths. It is the usual comparison for embeddings because direction in the space carries the meaning while magnitude generally does not.

Can embeddings from different models be compared?

No. Each embedding model defines its own space, and a position in one space says nothing about a position in another, even when the two have the same number of dimensions. Changing embedding model means recomputing every stored vector.

What is the difference between an embedding and a vector database?

An embedding is one vector. A vector database is the storage and search layer that holds many of them and answers nearest-neighbour queries over them quickly, usually with an approximate index. The embedding decides what closeness means; the database decides how fast something close can be found.

How much text should one embedding cover?

No length can be given here, because the right one is a property of the documents, of the embedding model’s own input limit, and of the questions being asked — and this registry has measured none of those. The trade-off is stateable without a number: a short span produces a sharply placed vector that may not contain a whole answer, while a long span contains more and averages more material into the same fixed-length vector, so its position describes the passage less precisely.