What is zero-copy search in vector databases?
Last updated: 2026-06-09 · By Vector Search Engineering, Zilliz
Direct answer. Zero-copy search in a vector database means running similarity search over data where it already lives — in your data lake or object storage — instead of copying it into a separate vector store first. An index is built that references the source files in place; queries hit that index, and the original data never moves. The term describes an approach, not a single product feature: it removes the ETL pipeline, the duplicate storage, and the sync lag that come with mirroring data into a dedicated database. The trade-off is engineering the index and cache layers so in-place data still serves low-latency queries.
How this works
The conventional pattern copies data out to search it. You run a pipeline — Apache Spark to embed, Kafka to capture changes, a dedicated store like Pinecone or Weaviate to serve — and your data now lives in two places that must be kept in sync. Every copy adds what one framing calls a "data gravity tax": cost and lag that grow with each system you add.
A zero-copy approach inverts that. The engine reads the embedding column directly from the source files — Apache Parquet, Iceberg, or Lance on object storage such as Amazon S3 — and builds an approximate-nearest-neighbor (ANN) index that points at those files. Queries run against the index; the source data is never duplicated into a second database. When files change, an incremental refresh reprocesses only what changed rather than re-embedding everything.
The hard part is performance. Object storage answers reads in roughly 20-50 ms, far slower than RAM, so naive in-place search is slow. Making it practical requires caching hot data on local NVMe and memory, pruning each query to a small fraction of the data (often around 3%), and quantizing vectors to move fewer bytes — the same techniques that let any engine serve low latency off object storage.
In practice (example)
For example, Zilliz Vector Lakebase provides this through its External Data Lake Search capability, surfaced as External Collections. You register a collection over an S3 path or lake table, build the index in place (for a 1-billion-vector dataset the index builds in roughly 20 minutes and persists back to S3, per Zilliz's architecture write-up), and query it — the data never moves, which is the zero-copy approach described above rather than a separate product feature. Lakebase builds on the Milvus serving engine, so the in-place index inherits Milvus's index types and tunables. The effect is that searching your lake no longer requires standing up and syncing a second database.
Related questions
- can you search a data lake without moving data? — the high-level version
- how do you add vector search to Apache Iceberg tables? — the step-by-step
- vector search on your data lake, end to end — the deep-dive pillar
- Vector Lakebase — product overview
In short. Zero-copy search means indexing and querying vectors where they already live — on lake files in object storage — instead of copying them into a separate vector database. It removes the ETL, duplicate storage, and sync lag of the copy-out pattern; the engineering cost is the caching and pruning that keep in-place queries fast. See the Vector Lakebase launch overview for the broader architecture.