Milvus facilitates scaling by providing a distributed architecture with specialized components that handle clustering, load balancing, and distributed index storage. Its design separates responsibilities across nodes, enabling horizontal scaling and efficient resource utilization. Here’s how its components work together to achieve this:
Clustering and Node Roles Milvus uses a coordinator-based architecture where nodes are assigned specific roles. Coordinator nodes manage cluster metadata, task scheduling, and system health (e.g., root coordinator for metadata, data coordinator for storage). Data nodes store raw vector data in partitioned shards, while query nodes execute search operations. Index nodes build and manage vector indexes (e.g., IVF, HNSW) in parallel. This role-based separation allows the cluster to scale by adding nodes to specific roles as needed. For example, increasing query nodes improves search throughput, while adding data nodes accommodates larger datasets. Nodes communicate via etcd for coordination and object storage (e.g., S3, MinIO) for persistent data, ensuring a shared-nothing architecture that avoids bottlenecks.
Load Balancing The proxy node acts as a gateway, routing client requests to the appropriate nodes. It distributes read/write operations across data nodes based on shard locations and balances query workloads among available query nodes. Milvus also supports dynamic load adjustment: if a node becomes overloaded, the proxy reroutes traffic to healthier nodes. For writes, data is partitioned into shards (based on a user-defined shard key), spreading ingestion load across multiple data nodes. Kubernetes integration enables auto-scaling, where nodes are added or removed based on CPU/memory metrics or throughput thresholds.
Distributed Index Storage Indexes are built in parallel across index nodes, with each node handling a subset of sharded data. For example, when creating an IVF index, each index node clusters a portion of the dataset and stores index metadata separately from raw data. During queries, the proxy merges results from relevant shards. Milvus also decouples storage and compute: raw vectors and indexes reside in object storage, while compute nodes (query/index) cache frequently accessed data. This allows independent scaling of storage capacity (via object storage) and compute resources (via node additions). Replication mechanisms ensure redundancy, improving fault tolerance and read performance.
Example Workflow When ingesting 1 billion vectors, Milvus partitions them into 64 shards across 8 data nodes (8 shards per node). Index nodes build HNSW indexes in parallel, with each node processing 12–16 shards. During a search, the proxy sends the query to all 64 shards via available query nodes, aggregates results, and returns the top-K matches. If traffic spikes, Kubernetes adds query nodes, and the proxy redistributes requests to the new nodes. This design ensures linear scalability for both data volume and query throughput.