Why Does HNSW Index Recall Drop?

A drop in HNSW recall is almost always addressed from the wrong angle. The team sees poor results, raises ef_search, adds a few milliseconds, and declares the incident closed. A week later, recall drops again because the real cause was a new embedder, mixed vector versions, a post-search filter, or an incomplete collection load.

HNSW does not know that a document is outdated, that the text was split with a different chunker, or that half the records in a new segment received vectors without L2 normalization. It traverses the graph using the numbers it was given. That is why diagnosis starts by firmly separating two questions: does exact search over the current vectors still find the expected documents, and how well does HNSW reproduce that exact search?

The original work by Malkov and Yashunin describes HNSW as a hierarchical graph for approximate search. Its speed comes from limiting graph traversal, not from a promise to always return the exact top-k. Faiss documentation clearly distinguishes exhaustive IndexFlat from IndexHNSWFlat, while OpenSearch documentation links a higher ef_search to better recall and higher latency. This is a useful starting point, but it is not a diagnosis.

First prove that HNSW itself got worse

A drop in HNSW recall means that the approximate top-k overlaps less with the exact top-k on identical data. It does not mean that users have started receiving less useful answers.

In production, three different metrics are often mixed together. The first is a product metric: did retrieval find a document that helps the user? The second is a vector metric that measures embedding quality: does the relevant document appear in the exact top-k for the selected metric? The third is ANN recall, which shows how many elements from the exact vector top-k made it through HNSW.

If the product metric has dropped while ANN recall is stable, do not touch the graph parameters. Check the embedding model, document splitting, query language, access rules, and reranker. If ANN recall has dropped while exact vector search still looks meaningful, then it makes sense to investigate the index and the data update path.

There is an even more unpleasant case: the exact top-k itself has become poor. This can happen after changing the embedder, changing normalization, transforming the text, or adding a large document domain with different terminology. In that situation, HNSW may honestly reproduce a poor baseline with recall of 0.99.

Define the metric before running any experiments:

recall@k = |ANN_top_k ∩ exact_top_k| / k

Compare document or chunk IDs specifically. Do not replace recall with average similarity. Average similarity may barely change even when HNSW systematically misses one or two of the closest elements.

The baseline must use the same collection snapshot

Exact and approximate search must run on the same data snapshot. Otherwise, the measurement has no meaning.

For every control run, save at least the collection version, number of live points, embedding version, vector dimensionality, metric, normalization status, filter parameters, and list of query IDs. If exact search ran before the nightly load while HNSW was checked afterward, you measured a corpus change together with index quality.

Here is a minimal control calculation for cosine similarity. It works for exporting a control set to NumPy and deliberately does not depend on a particular vector database.

import numpy as np

# db: float32 [n_vectors, dim], q: float32 [n_queries, dim]
# ids: int64 [n_vectors], ann_ids: int64 [n_queries, k]

def normalize(x):
    norms = np.linalg.norm(x, axis=1, keepdims=True)
    return x / np.clip(norms, 1e-12, None)

def exact_topk_cosine(db, ids, q, k):
    db = normalize(db.astype(np.float32))
    q = normalize(q.astype(np.float32))
    scores = q @ db.T
    pos = np.argpartition(-scores, kth=k - 1, axis=1)[:, :k]
    row = np.arange(q.shape[0])[:, None]
    pos = pos[row, np.argsort(-scores[row, pos], axis=1)]
    return ids[pos], scores[row, pos]

def recall_at_k(exact_ids, ann_ids):
    hits = []
    for truth, found in zip(exact_ids, ann_ids):
        hits.append(len(set(truth.tolist()) & set(found.tolist())) / len(truth))
    return float(np.mean(hits))

exact_ids, exact_scores = exact_topk_cosine(db, ids, q, k=20)
print({
    "recall_at_20": recall_at_k(exact_ids, ann_ids),
    "queries": len(q),
    "exact_first_score_median": float(np.median(exact_scores[:, 0]))
})

The output format should be boring and consistent:

{'recall_at_20': 0.9475, 'queries': 400, 'exact_first_score_median': 0.7812}

Do not use a different SDK for the baseline if it silently rounds vectors, applies a different distance operator, or adds a post-filter. The exact calculation must receive the same arrays as the index. In particular, cosine similarity through a dot product is correct only when both the database and the queries have been normalized in the same way.

Data drift changes the geometry even when the model stays the same

Data drift means that the distribution of vectors, queries, or their relationship to one another has changed. It is not the same as a broken graph.

Imagine a database of banking product instructions where documents have spent years in relatively even topical clusters. Then large numbers of short transaction logs, templated notifications, and duplicate excerpts from policies are added. The embedding model has not changed. But dense islands of nearly identical vectors have grown, and old queries now receive dozens of candidates with almost identical distances.

Two events can happen at the same time in this database. User quality falls because the exact top-k is filled with near-duplicates. ANN recall by ID may also fall because the exact top-k contains many nearly equivalent neighbors, and HNSW selects a different element from the dense group. The second effect looks like an index problem, although the stability of the baseline should be checked first.

Do not look only at the average distance to the first neighbor. Save the following for every query:

• distance or similarity of the first exact neighbor;
• gap between the first and k-th neighbor;
• gap between the k-th and k+1-th neighbor;
• share of results from each data slice;
• share of near-duplicate chunks in the top-k.

For cosine similarity, a small score[k] - score[k+1] gap means that the top-k boundary is fragile. One document can leave the exact list because of a minimal numerical difference, and another can take its place. In this mode, it is also useful to calculate recall@50 with a user k of 10, or to measure whether the result falls within a wider candidate pool. This does not excuse poor HNSW. It protects you from drawing the wrong conclusion from an unstable boundary.

Compare distributions separately for the old and new corpus, as well as for old and recent queries. If exact-first similarity, gaps, and top composition have changed, the data is drifting. If exact-search distributions are unchanged while ANN increasingly misses obvious exact neighbors, suspicion shifts to the index.

An increase in ef_search separates traversal limits from other causes

If recall rises noticeably together with ef_search, HNSW is not exploring enough of the graph within the current traversal budget.

ef_search sets the number of vectors considered during HNSW traversal. OpenSearch states the trade-off directly: a higher value improves recall but increases latency. In Faiss, the same parameter is called the search depth. Choose it from a measured curve rather than copying a value from someone else’s example.

Run one experiment on a frozen snapshot. Do not change the model, replicas, compression, filter, and query mix at the same time.

1. Take a frozen query set containing ordinary, rare, and problematic queries.
2. Run exact search and save the top-20.
3. Run HNSW with several increasing ef_search values.
4. For each value, record recall@10, recall@20, p50 and p95 latency, and results by slice.
5. Repeat the run after restarting the client to rule out warm-up effects and an accidentally different request route.

The interpretation is usually straightforward. A sharp recall increase with an acceptable latency increase points to an insufficient search budget. A nearly flat curve points to another problem: the wrong metric, corrupted vectors, a filter error, quantization, or a mismatch between the export used for exact search and the real collection.

Do not draw conclusions from the average alone. One dense language segment or one document type may lose half its neighbors while overall recall still looks acceptable. This is especially visible in RAG systems with rare legal wording, tariff names, error codes, and queries containing several constraints.

Build parameters cannot be fixed with one query

A high ef_search can compensate for part of a weak graph, but it cannot restore connections that the index did not create during construction.

The parameters that most strongly affect HNSW structure are M and ef_construction or ef_construct, depending on the engine. M limits the number of connections, while ef_construction determines the amount of candidate search performed when a vector is added. OpenSearch documentation says that a higher ef_construction builds a more accurate graph at the cost of slower indexing, while Qdrant separately describes m, ef_construct, and the search ef as different controls.

This distinction is often blurred. ef_search controls how deeply you traverse an existing graph. ef_construction affects what that graph becomes. Raising the first after a bulk load with a low second value can be reasonable, but it pays for an old construction-quality decision with additional latency.

The check is simple: create a temporary index from the same exported snapshot using the current parameters, and a second index using changed construction parameters. Run the same queries on both with the same ef_search values. If the new graph wins with the same search budget, rebuilding is justified. If there is no difference, do not launch an expensive rebuild just to feel in control.

Do not copy parameters between collections without measuring. A corpus of short, similar announcements, a multilingual archive of support requests, and a technical documentation database have different cluster densities. A parameter that worked well for one may consume memory without a noticeable benefit on another.

Collection updates disrupt measurement more often than they damage the graph

After a collection update, first look for data inconsistency rather than a mysterious HNSW degradation.

The most common error looks mundane. A pipeline writes new payloads, then fails before writing the vector. Another worker retries the task but uses a new chunker. A third process deletes old IDs in only one shard. The document count looks almost right, but the exact export and the live collection actually contain different objects.

Check these five invariants before tuning the index:

• every live ID has exactly one vector with the required dimensionality;
• the embedding version is consistent within the measured collection;
• vectors went through the same normalization;
• documents and chunks do not change IDs on retries without a clear reason;
• filterable fields and access rights are present in the exact baseline and ANN request in the same way.

Pay particular attention to upserts. Replacing text under an old ID is safe only if you guarantee that the new payload and new vector belong to the same logical version. If a process reads a document, computes its vector, and writes them in separate operations without versioning, a concurrent update can easily leave text from one revision and an embedding from another.

Deletions and segmentation also require checking. Some engines may use a full scan in small segments, others build the graph later, and others optimize segments asynchronously. For example, Qdrant documents the threshold at which the planner uses a full scan instead of HNSW and separately describes when segments are reindexed. So “it got worse after the load” may mean that requests entered a different execution mode.

Create a control report before and after the update: number of points, number of unique IDs, number of vectors by version, dimensionality, share of zero vectors, norm distribution, and share of documents in each segment. It often finds the cause before the recall chart becomes a matter of debate.

Filters and quantization create separate kinds of loss

Filtered ANN and search over compressed vectors should be evaluated separately because they change the search task itself.

A filter is not a cosmetic condition applied after top-k. If you first obtain the twenty nearest candidates across the whole database and then discard forbidden ones, you may return fewer than k results and miss eligible neighbors that were slightly deeper in the list. OpenSearch documentation explicitly warns that post-filtering under a strict predicate can return significantly fewer than k results, while filtering during k-NN search works differently.

The baseline for a filtered query should be calculated like this:

exact_top_k = top_k(metric(query, vector)) among documents that pass the same filter

Do not compare ANN after filtering with exact top-k over the full collection. This experimental error will inevitably show an apparent “recall drop” under strict access rules, dates, languages, and tenant IDs.

Quantization requires another breakdown. You may have original float vectors, a compressed representation, and HNSW built over that compressed representation. To identify the source of the loss, measure three levels:

1. exact search over the original vectors;
2. exact search over the representation actually stored for search;
3. ANN over that same representation.

The difference between the first and second levels belongs to compression. The difference between the second and third belongs to approximate search. Increasing ef_search cannot compensate for quantization loss: HNSW will search more accurately for neighbors in geometry that has already been distorted.

Rebuild only after proving that it is necessary

Rebuild HNSW when measurements show that the same set of vectors gets better ANN recall on a fresh graph, or when the update path cannot guarantee the integrity of the current structure.

A proper rebuild does not begin by deleting the production collection. Create a new version beside the old one, load one verified export into it, check the invariants, build the index, run the frozen set, and compare exact results, ANN results, and latency. Then switch reads over. Keep the old version long enough to investigate discrepancies for specific query IDs.

Do not make periodic rebuilding your only quality policy. It may temporarily hide mixed embeddings, incorrect filters, or corpus drift, but these errors will return in the next update. A good system catches them at the input: the embedding version belongs in the record metadata, control queries run after each load, and the exact baseline lives separately from the online path.

For teams using a shared OpenAI-compatible layer, AI Router can simplify control over embedding versions and model routing, but it does not replace this discipline. The vector collection remains your responsibility: store which model and transformation produced every vector.

Keep two test sets instead of one overall recall

One aggregate recall metric hides both index degradation and data degradation. Keep two query sets for different purposes.

The first set is for ANN engineering. It is fixed, covers difficult geometric cases, and is always compared with exact search on one snapshot. Its purpose is to show the effect of ef_search, build parameters, segments, filters, and compression.

The second set is for the product. It contains real queries and verified relevant documents, is updated with the domain, and measures retrieval quality before and after the reranker. Its purpose is to show when the model, chunker, or corpus has stopped answering the user’s question.

When these sets diverge, that is good news: you know where to look. If only the first falls, investigate HNSW and storage. If only the second falls, investigate the data and semantics. If both fall, first freeze the new collection version and recalculate the exact baseline. Raising ef_search before that means buying latency instead of an answer.