Cosine Similarity on AIBussin — AI applications, systems and books

Meaning Becomes Geometry

Mon, 07 Sep 2026 09:10:00 +0000

Part I — A Vector Is Not Meaning

A space you can draw

Give six words two coordinates each, by hand:

              x = monarchy / institutional power        y = gendered association (-1 … +1)
king       ( 0.9,  0.6)
queen      ( 0.9, -0.6)
duke       ( 0.7,  0.5)
duchess    ( 0.7, -0.5)
apple      (-0.7,  0.1)
orange     (-0.7, -0.1)

Now several semantic questions have a geometric form:

Which words form a group? → apple and orange sit almost on top of each other, about 0.2 apart, and far from the four titles. The titles form a looser group of their own near x ≈ 0.8. Grouping shows up as density.
What distinguishes king from queen? → the vector king − queen ≈ (0, 1.2). It points almost purely along y. The difference between them, in this space, is one attribute. (duke − duchess ≈ (0, 1.0) points the same way, but only because we placed the points that way; a learned space owes us no such consistency.)
Is apple oriented like king? → the angle between them is obtuse and their cosine is negative. Fruit and titles point into different half-planes. Orientation separates the two categories cleanly, even though, as we are about to see, it is noisier within a category.
Are king and queen related? → obviously: they are the two monarchs. But their Euclidean distance is 1.2, while king sits 0.22 from duke and about 1.12 from duchess. Rank king’s neighbors by distance and queen comes third, behind duke and duchess. The gender axis — which a search for “royal titles” need not care about at all — has quietly reordered the neighborhood.

That last point is the one to carry forward.

Similarity Is a Decision

Mon, 07 Sep 2026 09:30:00 +0000

Part I — A Vector Is Not Meaning

Two vectors, four answers

Here are three document vectors, kept to three dimensions so every number is checkable:

x = ( 2.0, 0.0, 0.0 )      a short doc, one strong topic
y = ( 6.0, 0.1, 0.0 )      a long doc, same topic, much more of it
z = ( 0.0, 2.0, 0.0 )      a short doc, a different topic

Before computing anything, look at the geometry. x and y point in almost exactly the same direction but have very different lengths (norm 2 versus about 6). x and z have the same length but point ninety degrees apart. So “is x more like y or z?” comes down to a prior question: which kind of difference should count — a difference in direction, or a difference in size?

Feature Space: What Does a Linear Model Actually See?

Fri, 28 Aug 2026 15:30:00 +0100

Here is a sequence classifier. Eight examples, 128 positions each, 768 features per position, and a linear head that produces one score.

x = torch.randn(8, 128, 768)
classifier = nn.Linear(768, 1)

scores = classifier(x)

input           (8, 128, 768)
Linear(768,1)  -> (8, 128, 1)

One thousand and twenty-four scores where eight were expected. Nothing raised, nothing is non-finite, and the shape is entirely predictable once you know the rule. The layer did exactly what the tensor asked of it: nn.Linear transforms the last axis and preserves every axis before it, so it produced one score for every (example, position) pair — 128 scores per example, computed independently.

RELATE: Searching Embeddings by Relation, Not Just Similarity

Wed, 05 Aug 2026 23:24:45 +0100

Embeddings are everywhere in modern AI.

They power semantic search, retrieval-augmented generation, recommendations, clustering, duplicate detection, code search, memory systems, and many of the mechanisms through which an AI system decides what information is relevant.

Yet most systems interrogate embeddings in essentially the same way:

Take two vectors and calculate cosine similarity.

That is useful. But it also makes a strong assumption.

It assumes that the information we care about is expressed directly through the default geometry of the embedding space.