AI Search Scores: Check Whether Vector Length Counts

An AI-assisted search returns a document with a similarity score of 0.8. It is tempting to read that as an 80 per cent chance that the document answers the question. Before making that translation, ask what calculation produced the number. A score can compare mathematical representations without estimating the probability of a correct answer.

One useful distinction is whether the calculation counts a vector’s length as well as its direction. A small, checked example shows how that choice can reverse a ranking. For business owners and independent creators, the lesson is to keep a search score connected to its definition and to the task it is meant to support.

A vector is a representation with a scoring rule

For this example, a vector is simply a list of numbers. We will use two-number lists so every calculation is visible. The values are invented; they are not embeddings generated by an AI model, and their coordinates have no hidden linguistic meaning.

The scikit-learn project’s primary documentation on pairwise metrics defines cosine similarity as a dot product divided by the lengths of the two vectors. Its linear-kernel section describes the unnormalised dot product. These definitions give us two clear rules to compare. The documentation was checked on 30 October 2026; this article explains established mathematics rather than announcing a new research result.

To calculate a dot product, multiply corresponding coordinates and add the results. To calculate a vector’s Euclidean length in our two-dimensional example, take the square root of the sum of its squared coordinates. Cosine similarity divides the dot product by the product of the two lengths. We will use only nonzero vectors, so the divisor is nonzero.

Two rules, two different rankings

Let the query vector be Q = (1, 0). Candidate A is (4, 3), and candidate B is (6, 8). Suppose we rank higher scores first under each rule.

The dot products are:

• Q with A: 1 × 4 + 0 × 3 = 4.
• Q with B: 1 × 6 + 0 × 8 = 6.

By that rule, B ranks above A. Now calculate their lengths. Q has length 1. A has length √(4² + 3²) = 5. B has length √(6² + 8²) = 10. The cosine scores are therefore:

• Q with A: 4 ÷ (1 × 5) = 0.8.
• Q with B: 6 ÷ (1 × 10) = 0.6.

By cosine similarity, A ranks above B. The candidates and query have not changed. The scoring rule has. These calculations were independently checked in a small JavaScript calculation for this article; no search engine, embedding model or commercial service was tested.

What happens when only the scale changes

Multiply both coordinates of A by ten to obtain (40, 30). This preserves its direction while increasing its length from 5 to 50. The dot product with Q becomes 40. Its cosine similarity with Q remains 40 ÷ (1 × 50) = 0.8.

In this worked example, the unnormalised score changes with that scaling, while cosine removes the positive scale change. Neither outcome, by itself, tells us which candidate would be the more useful document. We have supplied no documents and no definition of usefulness.

That limitation matters. It would be unjustified to conclude that cosine is always the better rule, or that a larger vector is necessarily a longer document. Those interpretations would require evidence about how an actual system constructs its representations and what its scores are intended to capture.

A score of 0.8 has not established an 80 per cent success rate

The 0.8 above comes entirely from the supplied coordinates and arithmetic. We did not observe a collection of successful and unsuccessful answers, or estimate the likelihood that a document contains the required information. Calling it an 80 per cent probability would add a claim that the example has not established.

For a real product displaying a percentage, ask the supplier to explain what the percentage represents. Is it a transformed similarity score, a separately estimated probability or another measure? Ask what evidence supports that interpretation. A familiar-looking scale should not have to carry an unexplained meaning.

The same discipline applies to a threshold. A proposed cutoff of 0.8 needs a reason connected to the actual scoring system and task. Our A happens to meet that number under one calculation, but this arithmetic example cannot establish a useful acceptance threshold for someone else’s document collection.

Turn the technical question into an acceptance check

Imagine a fictional course provider considering an AI-assisted handbook search. Its practical requirement is to locate the instructions that apply to the learner’s cohort. The owner does not need to select a mathematical rule by intuition. The implementer should explain the configuration and show whether the intended cases work.

A compact request could ask for:

• The representation method and version used for the query and documents.
• The scoring rule, including whether vectors are normalised and how scores are ordered.
• Examples with expected relevant documents identified before reviewing the returned ranking.
• The retrieved document, the displayed score and the eventual answer for each example.
• An explanation of any threshold or percentage shown to users.

These are proposed evaluation questions, not evidence that a particular product offers those controls. If a tool hides the scoring details, record that limitation and test the observable behaviour. Do not invent an explanation for an undocumented score.

Similarity is only one part of selecting a source

In the fictional course service, a document can resemble the question while applying to a different cohort. That is a separate requirement from numerical similarity. MaryChuks’ guide to deciding which knowledge-base document applies explores that operational distinction.

For this example, define the eligible handbook before treating its rank as sufficient evidence. If the learner’s cohort is missing, the intended response may be a clarifying question. If the returned source is wrong, a fluent answer should not end the investigation.

The owner’s useful question is whether the system can explain and demonstrate its selection. A similarity score becomes more informative when the representation, calculation and task remain visible together. Our two-vector example gives a modest starting point: even before considering language, changing the mathematical rule can change which candidate comes first.

Sources and example basis

• scikit-learn: Pairwise metrics, Affinities and Kernels, project-maintained documentation, especially cosine similarity and the linear kernel; checked directly in the public source repository on 30 October 2026.
• MaryChuks: AI Knowledge Bases — Decide Which Document Applies, related operational guidance; published link verified 30 October 2026.
• The vectors and course-provider scenario are original illustrative examples. The arithmetic check did not run scikit-learn or test an AI retrieval system.