Changing the search metric can change the top result for the same vectors. That need not indicate a broken index. Inner product depends on direction and magnitude; cosine similarity divides out magnitude. Equivalence requires explicit normalization assumptions.

This is a useful configuration check before RAG hybrid retrieval. 中文版

A two-dimensional counterexample

Let the query be q=(1,0), candidate A=(1,0), and B=(80,60). These invented teaching vectors do not represent actual text. A points in exactly the query direction. B has length 100 and a different direction.

The inner products are q·A=1 and q·B=80, so B ranks first. Cosine similarity is cos(q,d)=(q·d)/(‖q‖‖d‖). A scores 1 and B scores 0.8, placing A first. This demonstrates different objectives; it does not establish which is better for real semantic search.

What normalization changes

For a nonzero vector, define x̂=x/‖x‖. Once both vectors have unit length, x̂·ŷ equals the original cosine similarity. The official Faiss guide maps cosine search to maximum inner product search by normalizing database and query vectors.

For a fixed nonzero query, changing only its magnitude leaves raw inner-product ordering unchanged: every score receives the same positive multiplier. Scores and thresholds still change. Candidate magnitudes vary, so this argument does not remove their influence. Normalize both sides consistently if returned values must represent cosine similarity.

L2 equivalence has its own condition

Expanding squared distance gives ‖x−y‖²=‖x‖²+‖y‖²−2x·y. Only when both vectors have unit length does it become 2−2x·y. Exact maximum-inner-product and minimum-squared-L2 ordering therefore agree on unit vectors; do not apply that conclusion to unnormalized vectors.

Faiss returns squared Euclidean distance for L2. Taking a square root preserves ordering, but thresholds and displayed distances need to use the intended quantity.

Reproduce the calculation without a model

The following standard-library Python example was executed. It checks the ranking change, the unit-vector identity and zero-vector handling. It performs neither embedding inference nor a Faiss index search and is not a retrieval benchmark.

from math import sqrt, isclose

def dot(a, b):
    if len(a) != len(b):
        raise ValueError('dimension mismatch')
    return sum(x*y for x,y in zip(a,b))

def unit(v):
    norm = sqrt(dot(v,v))
    if norm == 0:
        raise ValueError('zero vector has no cosine direction')
    return [x/norm for x in v]

q = [1., 0.]
docs = {'A': [1., 0.], 'B': [80., 60.]}
raw = {k: dot(q,v) for k,v in docs.items()}
cos = {k: dot(unit(q),unit(v)) for k,v in docs.items()}
assert raw == {'A': 1., 'B': 80.}
assert isclose(cos['A'], 1.) and isclose(cos['B'], .8)
for v in docs.values():
    x,y = unit(q),unit(v)
    squared_l2 = sum((a-b)**2 for a,b in zip(x,y))
    assert isclose(squared_l2, 2-2*dot(x,y), abs_tol=1e-12)
try:
    unit([0.,0.])
except ValueError:
    pass
else:
    raise AssertionError('zero vector must be rejected')
print('inner product:', raw)
print('cosine:', cos)
print('normalized squared L2:', {k:2-2*s for k,s in cos.items()})

Inner product produces A=1 and B=80; cosine produces A=1 and B=0.8. Normalized squared L2 is approximately 0 and 0.4. The printed 0.3999999999999999 is floating-point representation error, so the identity uses a tolerance. A zero vector has no defined cosine direction; this code rejects it explicitly. Production handling also needs an explicit policy.

Three checks before changing the metric

First, check the embedding model’s intended scoring method and whether its outputs are already normalized. Magnitude may carry information by design; normalization is not automatically a harmless transformation for every model.

Second, check that indexing and querying share model version, dimensions and preprocessing. Normalizing new queries while keeping non-unit document vectors does not generally produce cosine scores. A change needs to account for existing indices and live thresholds.

Third, build a small exact baseline before comparing approximate search. Mathematical ordering equivalence does not guarantee identical lists under different approximate settings, quantization errors or tie-breaking rules. Separate score semantics from search error to distinguish configuration mistakes from approximate recall loss.

References

Faiss: MetricType and distances（2024-12-03）

Publication note: published as a catch-up on October 9, 2026 (Beijing time), retaining the originally scheduled 09:00 article date. This original mathematical example makes no model-performance claim.