How Meituan Built a 10B-Scale Retrieval Platform on Milvus to Power 100+ Use Cases
10B+ vectors
In production, self-run on Milvus
2B+ in one collection
The largest single workload, grown without re-architecting
P99 8.1 ms
~1,000 QPS and >97% recall on a 100M-vector, 768-dim benchmark
100+ use cases
On one shared Milvus platform: RAG, agent memory, and more
"When Meituan committed to AI, we needed a retrieval foundation built for vectors from the ground up. We chose open-source Milvus and turned it into a platform every team can build on. A year later, it holds more than 10 billion vectors and runs our fastest-growing AI workloads, from RAG to agent memory."
The Meituan Database Team
About Meituan
Meituan is one of Asia’s largest technology companies and operates a leading on-demand platform for local services. Its super-app spans food delivery, in-store dining and consumer reviews, hotel and travel booking, groceries, and instant retail — all fulfilled through a vast network of local merchants and couriers.
AI has become central to how Meituan serves that marketplace, from answering customer questions to giving merchants smarter tools. More than 100 business scenarios now use RAG, agent memory, and multimodal search, and each one needs fast, filtered vector search underneath. Meituan's database team runs that shared data foundation on open-source Milvus.
The Challenge
As vector retrieval expanded across Meituan, the database team had to support an increasingly diverse set of workloads. Different applications came with different requirements for vector data types, index algorithms, dimensions, and retrieval performance, while dataset sizes ranged from millions to billions of vectors.
Meituan needed shared infrastructure that could meet these different requirements at scale while balancing query performance, resource efficiency, and operational cost. Initially, they ran vector search on Elasticsearch, but it soon couldn't meet Meituan's performance and cost requirements. Elasticsearch capped vectors at 4,096 dimensions, supported only a handful of vector types, and had no disk index, so every vector had to sit in memory. At hundreds of millions of vectors, that made Meituan’s largest, fastest-growing datasets the most expensive to keep online. The team needed a purpose-built vector search engine whose index types could hold huge datasets on disk, not just in RAM.
Why Milvus
The database team evaluated the main vector search engines on capability, performance, and ecosystem, then chose open-source Milvus. Three reasons led to that decision.
Capability. Milvus provided the breadth of vector data types and index algorithms Meituan needed across a highly varied set of workloads, including disk-based indexes for very large collections. Its storage-compute-separated architecture also gave the team more flexibility to scale large vector workloads efficiently.
Performance. In the team's own benchmark of 100 million 768-dimensional vectors on an HNSW index, Milvus held P99 latency at 8.1 ms and about 1,000 QPS, with recall above 97%.
Ecosystem. Milvus also fit naturally into Meituan’s broader open-source AI stack. It offered mature SDKs, documentation, and an active community (46k+ GitHub stars), while some of the open-source frameworks adopted by internal teams already supported Milvus directly. That reduced integration friction and made it easier to bring new vector-search scenarios onto the platform.
The Solution
After evaluating the major vector search engines, Meituan chose Milvus as the vector-native foundation for its growing AI workloads, while its existing Elasticsearch deployment remained part of the broader dual-engine architecture. These vector use cases span RAG and knowledge bases, agent memory, customer-service and diagnosis agents, recommendation, and multimodal search. Most share a similar shape: a few million to tens of millions of vectors, 768 to 1,536 dimensions, served at ~100 queries per second. A handful are far bigger, like the one workload that has passed two billion vectors.
Instead of having each team run its own Milvus, the database team put it all on one shared platform that gives any team a production-ready cluster without turning them into vector-database experts. Serving workloads this varied from a single platform came down to four things: holding billions of vectors cheaply, keeping tenants isolated so one can't drag down another, staying reliable for business-critical workloads, and offering enough self-service that a small team can support everyone.
Keeping billions of vectors cheap, with disk indexes
The cost win comes from Milvus's disk indexes. With DiskANN, a big collection lives mostly on disk instead of in memory, which keeps even a two-billion-vector workload affordable. On Meituan's hardware, those indexes needed more local disk than a standard deployment gives a query node, so the team mounted dedicated volumes and ran Milvus's operator as StatefulSets, so each node keeps its own disk across restarts. Because the operator is open source, that was a simple change to make, and that's why Meituan can lean on disk indexes as hard as they do.
Isolation across many tenants: public and dedicated clusters with a bounded blast radius
Milvus lets many tenants share one cluster, each in its own database, so Meituan groups the long tail of small workloads onto public clusters and keeps utilization high. The bigger workloads get their own dedicated clusters: anything past a million vectors, anything that needs high QPS, anything business-critical, or anything that needs its own tuning.
The one risk with sharing is blast radius, one misbehaving tenant dragging down its neighbors, so Meituan caps how many tenants go on a public cluster and sizes up each new workload's job, importance, and data before placing it.
Reliability for critical workloads: fast failover, then CDC standby
Because Milvus separates storage from compute, the data itself is already safe (it sits in object storage with three copies), so Meituan's reliability work is really about keeping the service reachable. Clients hit Milvus through its proxy nodes, and when a proxy dies, traffic needs to move fast. Meituan built failover into its own Milvus SDKs, in Python and Java: the control plane tracks which proxies are healthy and feeds that list to the SDK, so clients go straight to a live proxy, and a failing one drops out before it's shut down. That lets them skip running a load balancer in front of every cluster, and the overhead that comes with it.
For disaster recovery, Meituan spreads a cluster's nodes across data centers, proxies and coordinators first, then the middleware, with the goal of every component being balanced across availability zones plus a spare (N+1). For workloads that truly can't go down, they use Milvus's CDC to keep a standby cluster in sync, ready to take over if the primary fails completely.
Self-service, so one small team serves them all
Meituan's platform hands day-to-day control to the teams that own the workloads: visual Collection management (create, delete, load, release), one-click access to Milvus's native APIs, vector-plus-scalar hybrid search, monitoring dashboards, and database-level access control. Backups run automatically at database granularity, and data-sync channels move vectors in from sources like Hive and RDS or out for disaster recovery. Because the routine work moves to the users, on-call load stays low, and a small platform team keeps the whole fleet healthy.
Results & Benefits
10B+ vectors in production, from zero in about a year. One standardized platform now runs Milvus at large scale for 100+ use cases, with automated cluster delivery, backup, and self-service keeping the operating team small.
Billion-scale in a single workload, with headroom. The largest single workload has grown past 2 billion vectors without re-architecting.
P99 8.1 ms at ~1,000 QPS, recall >97% on the team's 100M-vector, 768-dimension benchmark, far more headroom than its mostly sub-100-QPS workloads need.
The company's fastest-growing AI workloads run here. RAG and agent memory drive growth, so Meituan builds a shared retrieval infrastructure instead of each standing up its own.
Billion-scale collections stay affordable. Keeping most of a collection on disk instead of in RAM makes a two-billion-vector workload affordable to run, and because Milvus is open source, the team keeps driving costs down further with its own disk index.
The strategic payoff: vector retrieval is no longer something each Meituan team solves alone. It is a standardized, reliable platform, which frees the AI teams above it to build products instead of infrastructure.
What's next
Meituan is deepening its investment in Milvus on three fronts. New clusters now default to Milvus 2.6, and the team is migrating existing workloads as they adopt features such as built-in embedding functions. On the kernel side, it is developing a self-built, quantization-based disk index on Milvus 2.6 to push cost efficiency further at billion scale. And as agents spread across the company, the team is building a platform-level agent-memory capability on Milvus and its ecosystem, so internal teams get memory and knowledge-base building blocks out of the box. Each of these grows what runs on Milvus, and the team plans to keep contributing back to the community as it goes.
"Meituan is exactly the kind of engineering-driven company that pushes open source forward: they run Milvus at a scale most teams never reach, and they give back what they learn. We're proud that Milvus is the vector foundation beneath Meituan's AI workloads, and we're excited to keep building alongside their team and the community." — James Luan, CTO, Zilliz
Get started with Milvus or try the Managed Milvus service
Milvus is the world's most widely adopted open-source vector database, built for billion-scale similarity search and running from a laptop to a distributed cluster. It gives you the same vector-native engine behind Meituan's platform: a full range of index types, hybrid vector-and-scalar search, and a storage-compute-separated architecture that scales.
Start with Milvus on GitHub, read the docs, and join the community on Discord.
When you want the same engine as a fully managed service, with zero-maintenance operations and enterprise security, Zilliz Cloud is built on Milvus and compatible with its APIs.