How to tell when a vector database needs a rebuild

Deleting data from a vector database rarely means that the space and graph are freed at the same moment. From the application's perspective, the point is gone: search should not return it. For the index, it often remains as a tombstone record, an old version created by an upsert, or part of a segment that the engine will clean up later. This gap between logical and physical deletion is where extra latency, memory growth, small segments, and, in the worst cases, lower result quality come from.

A rebuild should not be a weekly ritual, and it should not run after every batch delete. You need a measurable maintenance process: track accumulated deletions separately, monitor the rate of change, check quality on a fixed query set, and account for whether background compaction is keeping up. One percent of garbage in a collection that rarely changes is not interesting. The same few percent added every hour in an actively updated HNSW index already requires a decision.

A deleted point can remain in the index

Logical deletion hides a point from results. Physical deletion frees storage structures and rebuilds the index without it. Mixing up these two events is dangerous: a team sees a successful delete response and concludes that the collection is now smaller and faster. That is usually wrong.

In HNSW, the graph stores links between vertices. When the engine marks a point as deleted, it may exclude it from the final results, but graph traversal can still encounter that vertex. Search spends computation on a candidate that it will discard later. A single deletion is hard to notice. After a long series of document replacements, traversal starts doing extra work on every request.

Segment-based storage adds another layer. New points may go into a fresh or growing segment, while old points remain in a sealed segment. Live records can become spread across many small pieces. A query then has to search several indexes, collect partial candidates, and merge the result. The number of live vectors may not change at all.

Qdrant's documentation explicitly describes deletions as soft deletes through a bit mask: the index is not rebuilt after every delete. It also points out an unpleasant detail that is often missed when update pipelines are designed: an upsert of an existing point with the same data marks the previous version as deleted and inserts a new copy. That means a pipeline that rewrites the entire corpus every night without checking for changes creates technical debt even when the document count stays the same.

This is not an argument against upsert. Idempotent loading is necessary, especially when a queue retries messages. But being idempotent for business logic does not make an operation free for the physical index. If you do not know whether a document has a new text version, a new embedding model, or merely a redelivered message, do not resend the vector out of habit.

There is a third category that should be separated from deletion: changing the representation. If you replace the embedding model, change the dimensionality, distance metric, or normalization method, the old vectors are no longer an earlier version of the same data. Compaction cannot fix them. This change requires a parallel collection and a full reindexing of the source documents.

The deletion ratio alone does not answer the question

The tombstone ratio is useful, but it does not tell you how quickly a collection is accumulating garbage or how much that garbage interferes with queries. A rebuild decision needs at least four observations: the physical record count, the live record count, the replacement rate, and search behavior.

Track two calculated indicators. The first shows how much index space is occupied by outdated points:

removed_ratio = deleted_or_superseded / physical_points

physical_points = live_points + deleted_or_superseded

The second shows churn over an observation window. It answers not "how much garbage has accumulated?" but "can cleanup keep up with writes?":

update_churn_7d = updated_or_reinserted_points_7d / live_points

Do not calculate churn from explicit deletes alone. It includes replacing a vector under the same ID, reprocessing documents, recomputing embeddings after a chunking change, TTL expiration, and moving documents between tenants. If you rewrite 30% of the live corpus in a week, while background tasks reduce the deleted ratio from 18% to 12%, the database has not become healthier. It is simply losing the race more slowly.

Add three operational metrics to these counters:

• p50, p95, and p99 latency for the main search query with normal filters;
• recall@k or nDCG@k on a fixed set of queries and labeled relevant documents;
• the number of segments and the size of unindexed or growing segments, if the engine exposes these values.

Average latency is almost useless here. Segment fragmentation and tombstone cleanup often hit the tail of the distribution first: one request lands on an overloaded shard, another touches a segment that is currently being rebuilt, and a third passes through too many small segments. The average still looks calm, while a user is already waiting several times longer than usual.

Search quality can also fail in two different ways. The first appears when the engine cannot quickly find enough live candidates in a damaged or overloaded structure. The second appears when a policy excludes unready segments from search to protect latency. Qdrant's FAQ gives a direct warning: searching unindexed segments increases latency, while enabling indexed_only or prevent_unoptimized can make results incomplete. This is a deliberate tradeoff between result completeness and predictable response time, not a free optimization.

Maintenance thresholds must account for size and churn

For a production collection, I would not set one magic percentage for every case. I would define three zones and tie each one to the observed state of search. These boundaries give the team a shared language, but they do not replace measurements.

Green zone: up to 10%

With up to 10% deleted or replaced points, monitoring is usually enough. Do not run a manual rebuild just to make a counter look good. Check that p95 and recall remain near the baseline and that background processes actually remove old records after large batch operations.

There is an exception for very small segments. In a segment with 500 points, 50 deletions produce the same 10%, but metadata overhead and fan-out may be more noticeable than the collection-wide percentage suggests. Look at the distribution by segment and shard, not just the aggregate.

Yellow zone: 10% to 20%

In this zone, plan compaction, especially if churn is high or p95 is starting to rise. Do not wait for users to notice the problem. First make sure background optimization is running and has enough CPU, memory, and I/O. If it is running but does not reduce the deletion ratio after a quiet period, check its queues, concurrency limits, and available disk space.

The 20% figure is not arbitrary. In the Qdrant Vacuum Optimizer configuration example, deleted_threshold is set to 0.2, and a segment must contain at least vacuum_min_vector_number points to qualify for this optimization. The documentation presents this as an example of criteria, not as a mandatory standard. Use it as a sensible starting point, then adjust it based on your own measurements.

Red zone: above 20%, or degradation regardless of percentage

Above 20%, create a specific cleanup plan and schedule a window. At 30 to 35%, I usually treat it as mandatory work unless the collection is about to be retired completely. In an active database, that amount of dead data rarely disappears on its own while updates continue.

But the red zone can start at a lower percentage if any of these conditions hold: p95 or p99 has consistently worsened against the baseline, recall on the test set has dropped, the segment count grows after every batch upsert, or the optimizer cannot keep up with its queue. This matters more than a round number. A collection with 8% tombstones and 25% of its corpus updated every day may be in worse condition than an almost static index with 22%.

Do not confuse this mode with full reindexing. If the vectors and index parameters have not changed, start with compaction or rebuilding the physical segments. A full reload from source data is justified only when you want to change the index itself, fix a historical pipeline defect, or no longer trust the integrity of the current data.

Fragmentation is not limited to tombstone records

Fragmentation means that live data is spread across too many physical pieces or graph structures. Tombstones often make it worse, but they are not the whole problem. You can have almost no deleted points and still pay for searches across dozens of small segments after frequent micro-loads.

Milvus illustrates this mechanism well with sealed segments. Its documentation describes Force Merge as combining small sealed segments into larger ones. Otherwise, every query searches each segment and merges the partial results. In a published controlled test on a static collection of one million 768-dimensional HNSW vectors, Milvus reported a noticeable increase in QPS and a reduction in p99 after merging segments. This is not a promise for your cluster, but the reason is universal: fan-out costs money on every query.

So monitor more than removed_ratio:

• the number of segments grows while the live count stays the same;
• p99 gets worse after a nightly load even though the deleted ratio barely changes;
• some shards respond noticeably more slowly than others;
• small segments remain small for weeks after writes have finished;
• with filtering by tenant or document type, candidates are often discarded after search.

The last point looks like a filter problem, but it is connected to physical data organization. If one shard stores points from many tenants and a query selects only one tenant, HNSW may traverse many unsuitable candidates. A rebuild will not fix a poor sharding strategy. It will only reduce the extra work temporarily.

Do not assess fragmentation from disk usage alone. Quantization, payload, logs, replication, and the filesystem can distort the picture. Storage size matters for rebuild capacity, but search optimization decisions should rely on segment count, working latency, and quality.

Separate frequent updates from mass reindexing

The most common mistake in RAG systems looks harmless: a crawler fetches a set of documents once a day and sends an upsert for every chunk because that is the simplest way to keep data current. If only 40,000 of 2 million chunks actually changed, you have created almost 2 million potential old versions for 40,000 useful updates.

The right approach starts before the vector database. For each document, store a content hash, the chunking schema version, the embedding model name, and the prompt or preprocessor version if it affects the text. A new vector is needed when any of these conditions changes. Redelivering the same document should not trigger embedding and writing again.

A minimal practical contract could look like this:

{
  "id": "policy-431:chunk-07",
  "content_sha256": "...",
  "chunking_version": "2026-06",
  "embedding_model": "text-model-v4",
  "embedding_version": "v4.2",
  "source_updated_at": "2026-07-21T03:14:00Z"
}

Before generating a new vector, compare the stored fields with the incoming document. If the hash and versions match, skip the write. If only metadata changed and your engine can update it without replacing the vector, update only the payload. If the text, chunking, or embedding model changed, create a new vector and deliberately include it in churn.

This separation provides another benefit: you can distinguish the normal daily life of an index from a migration. Changing embedding_version for the entire corpus should launch a separate project with a parallel collection, quality evaluation, and a reader switch. Do not hide it inside a regular job that "updates documents." Otherwise, you may wake up to 80% of points replaced, an optimizer queue that is full, and no way to tell which results belong to the old vector space.

For large loads, write in batches and let the engine finish indexing after the load instead of forcing it to rebuild structures after every thousand points. Qdrant's optimizer documentation specifically recommends disabling indexing during an initial bulk load and enabling it afterward, so computation is not wasted on repeated rebuilds. This works for a controlled import, but not for a continuously available collection where queries must see fresh data.

Trigger rebuilds with two signals, not one counter

Formalize the decision as a rule based on both data state and service state. A deletion counter alone causes unnecessary work. Latency alone makes you treat symptoms. Together, they provide a clear trigger.

For every critical collection, record a baseline after a successful load and optimization. These should not be "ideal" numbers from a test notebook, but real p50, p95, p99, recall@10 or recall@20, index size, segment count, and deletion ratio under normal load. Then define a rule:

Schedule compaction if:
removed_ratio >= 0.20
and update_churn_7d >= 0.05

Start accelerated diagnosis if:
p95_latency > baseline_p95 * 1.25
or recall@10 is below baseline by more than the agreed tolerance
or the optimizer queue grows for two consecutive observation intervals

Rebuild in a new collection if:
embedding_model, dimensionality, or distance metric changed
or HNSW/quantization parameters require a new physical index

The 1.25 multiplier is not a law of physics. It is a useful starting boundary if you have not yet configured an SLO. A service with a strict user-facing SLA may need a smaller tolerance. A nightly analytics process may tolerate a larger one. The important thing is to compare a collection with its own past, not with someone else's benchmark.

Check recall on a set that reflects your actual search. For support, use customer questions with expected articles. For a product catalog, use real search formulations and clicks after manual review. For internal document search, use employee questions and the documents they actually consider correct answers. Synthetic queries made from the same texts stored in the database are usually too kind to the index.

Do not replace quality evaluation with higher ef or nprobe. These parameters can be useful emergency levers: you buy recall at the cost of latency. But if the team spends months compensating for fragmentation by increasing search breadth, it pays for garbage on every request. Examine the physical state of the collection first, then tune search parameters.

A safe rebuild requires a second read path

A full rebuild is risky when a team overwrites the only production collection and hopes background tasks finish before traffic catches up. It is safer to create a new version, load it from a reproducible source, and switch readers only after validation.

The sequence looks like this:

1. Freeze the input snapshot: document source, chunking rules, embedding model version, metric, index parameters, and payload schema.
2. Create a new collection with a versioned name. Do not change the old collection during validation.
3. Load vectors in batches, wait for index construction, and record size, segment count, and indexing time.
4. Run a fixed query set against the old and new collections. Compare not only latency, but also top-k documents, filters, and business deduplication rules.
5. Switch reads through an alias, application configuration, or routing layer. Keep the old collection for an agreed rollback period, then delete it.

You will need spare capacity. During a rebuild, the source vectors, the new collection, intermediate segments, and sometimes replication copies exist at the same time. If you barely have enough disk for the current index, do not start rebuilding on the assumption that deleting the old version will free space in time. It will be deleted last.

Qdrant describes segment rebuilding through copy-on-write: the old segment remains readable, while changes go into a priority layer during the rebuild. This is a convenient model for online optimization, but it does not remove resource load or replace testing the switch during a full data migration.

Check one more unpleasant case: a write arrives during the rebuild. If the new collection is filled by one batch process while users continue updating the old one, you will lose the tail of changes at the moment of the switch. Decide in advance between a short write pause, dual writes to both collections, or a change log that catches the new version up before cutover. For a high-traffic service, I prefer a log and an explicit watermark. "We will switch quickly" is not a consistency strategy.

Different engines use different cleanup mechanisms

The term rebuild hides several different operations. Do not transfer one engine's settings to another just because the words in the documentation look similar.

In Qdrant, the vacuum optimizer selects segments based on the share of deleted vectors and a minimum vector count. It also merges segments and builds indexes. Monitor whether the optimizer is falling behind the incoming stream. If you use quantization, keep the original full-precision vectors: Qdrant's documentation says they are needed to recalculate quantized representations during Vacuum Optimizer. You cannot delete the originals to save space and then expect a correct rebuild.

In Milvus, compaction works with sealed segments and cleans up entities deleted beyond the retained Time Travel period. Its configuration also reflects the connection between segment size and operational quality: small segments are merged, while dataCoord.segment.expansionRate controls the allowed result size. Do not assess Milvus by delete percentage alone. Check how many sealed segments each query actually services.

In Weaviate, the HNSW index uses tombstones and periodic cleanup. The documentation recommends monitoring the tombstone count, cleanup-cycle progress, and the number of cleanup threads. Large indexes also have upper and lower deletion limits per cycle, so cleanup does not take all CPU or continue indefinitely. This is a good example of the right tradeoff: aggressive cleanup does not always make a service faster if it takes resources away from search.

In hnswlib, mark_deleted marks an item as deleted. The library separately supports reusing deleted slots through allow_replace_deleted and replace_deleted=True. This is useful for embedded or local indexes, but slot reuse should not be mistaken for full graph maintenance. If the working set changes constantly, measure latency and recall, not just capacity.

The general conclusion is simple: first find out how a specific engine stores old versions, when it runs cleanup, what happens to reads during optimization, and which metrics it exposes. Then choose thresholds. A configuration that works well for a segment-based database may mean nothing for a single in-memory HNSW file.

Index architecture determines the amount of future cleanup

Maintenance becomes much easier when the data lifecycle is visible in the schema itself. Do not put documents with different aging rates into one collection simply because they have the same embedding dimensionality.

Separate an almost static corpus of policies, instructions, and archives from rapidly changing product cards, sessions, order statuses, or messages. A static collection can live for months without noticeable fragmentation. An operational-data collection needs a different threshold, smaller segments, and a separate compaction budget. If you mix them, frequent updates will force you to maintain the cold corpus as well.

For multi-tenant search, do not create one collection per user without a compelling reason. This inflates the number of indexes and complicates maintenance. But one shared index without a carefully designed shard key is not a solution either. Choose granularity based on tenant size, update frequency, and filter behavior. A large tenant that rewrites its catalog every few hours should not constantly fragment data belonging to thousands of small customers.

Keep the source text or a reliable reference to it outside the vector database. A rebuild should not depend on whether old payload fields happened to survive in the collection being removed. You need reproducible documents, deterministic chunking, and versioned embedding parameters. Without them, every rebuild becomes an excavation: the team reconstructs which code created the index instead of validating the new version.

AI Router fits this architecture as a single OpenAI-compatible path to models and as an option for running open-weight models on infrastructure in Kazakhstan when data residency and latency matter to the team. But it does not remove the need for data discipline: choosing when to rebuild remains a matter of collection metrics and a controlled embedding lifecycle.

A bad habit is cheaper to fix before the first incident

Do not wait until the index is obviously slow. Establish a baseline after the first load, measure tombstones and churn separately, and treat large changes as migrations. Then a rebuild stops being an alarming operation performed at night by watching the p99 chart and becomes routine work governed by clear rules.

If you already have a collection with rising latency, do not start by deleting and reloading it. Record the live count, physical count, deletion ratio, segment count, p95, p99, and recall on one query set. Repeat the measurement 24 hours after large updates have stopped. If the optimizer has not changed the situation, you have grounds for compaction or a new collection version. If it has, you have found not a "vector database problem," but an update stream that is too fast and needs to be fixed in the ingestion pipeline.