How Kling AI Runs Vector Search on Milvus at 10B+ Scale and 10,000+ QPS
10B+
Vectors in the largest single production cluster
10,000+ QPS
Peak throughput, writes and queries at once
10,000+ dims
Dimensions per vector, past the old engine's 4,096 cap
Zero interruption
Single-AZ failure leaves search and writes running
About Kling AI
Kling AI is an AI image and video generation model with more than 100 million users across 224 countries and regions. It is part of Kuaishou, which also runs one of the world’s largest short-video and live-streaming platforms, with more than 400 million daily active users.
Image created via Kling AI
Kling AI and its parent company’s other products run vector search on one shared platform built on Milvus. It serves a wide range of systems that match and retrieve content by meaning rather than just keywords: screening uploads for safety and risk, catching videos that duplicate published ones, powering multimodal and semantic search, and grounding RAG and agent features. Its largest single cluster holds more than 10 billion vectors.
The Challenge
AI pushed the products past what keyword matching could do, and vector retrieval became a production requirement. The team initially met it by extending Elasticsearch with approximate nearest-neighbor indexing, but that couldn't keep up for long, especially as the workload climbed toward tens of billions of vectors. At that scale, three limits became hard blockers.
Reads and writes fought for the same machines. Risk control and content safety ingest new vectors continuously while serving online queries at the same time. In Elasticsearch, writes and queries share the same nodes, so under heavy concurrent load, query latency spiked and jittered exactly when the safety systems needed it steady.
Storage and compute could not scale apart. Elasticsearch binds the two together, so growing into the tens of billions of vectors meant adding whole nodes, memory, and CPU in lockstep, even when only one was the bottleneck. Both the cost curve and the scheduling complexity grew faster than the data itself.
A hard ceiling on vector dimensions. Elasticsearch capped vectors at 4,096 dimensions, short of what newer embedding models already produced. In practice, the search engine capped which embedding models teams could adopt.
Why Milvus
The Kling AI database team needed a purpose-built vector engine that directly addressed those three limits. After an internal evaluation of vector search vendors on the market, the database team chose Milvus to support its various AI use cases, because Milvus provides:
A vector-native, disaggregated architecture. Query, write, index-building, and coordination each run as separate components, with storage decoupled from compute.
Millisecond retrieval, even at billions of vectors. Where retrieval sits in front of a user or a safety decision, that is the number that matters.
Multiple index types, so each team tunes its own tradeoff. HNSW, IVF, and DiskANN sit alongside quantization schemes such as SQ8 and RaBitQ, with support for vector dimensions into the tens of thousands. A team serving 10,000 QPS and a team searching a far larger corpus on a tight memory budget can each choose their own point on the recall, latency, memory, and cost curve.
A complete feature set behind one engine. The team needed more than top-K lookups: scalar filtering, full-text and semantic search, hybrid search, and aggregation. Milvus covers all of it, so scenarios that would otherwise span several systems sit behind one API.
Open source, with a community a large company can build on. Milvus is licensed under Apache 2.0 and backed by a large, active community: 46,000+ GitHub stars, 10,000+ enterprise users, and 100M+ downloads.
The Solution
The platform behind Kling AI and the other products in its group runs Milvus 2.5 and 2.6 and serves a wide range of use cases, including risk control and content safety, video deduplication, RAG and knowledge bases, agent memory, recommendation and user profiling, and multimodal and cross-modal search. Embeddings range from 64 to over 10,000 dimensions, the largest single cluster holds more than 10 billion vectors, and peak traffic exceeds 10,000 QPS.
Those workloads have very different needs. Some need the most searches per second, some combine vector, keyword, and filter search in one query, and some must answer within a strict time limit. Many take in new data nonstop while serving live searches, and all of them have to stay up. The highlights below show how Milvus and the team’s own engineering meet those demands.
Risk control and content safety: read/write separation at 10,000+ QPS on billions of vectors
New content is written into a single cluster around the clock and must be searchable almost as soon as it lands, while searches keep pouring in. Milvus runs writes and searches on separate nodes, so a surge of new content doesn’t slow searches down. HNSW, a graph-based index, keeps each search fast; SQ8 compression cuts each vector to about a quarter of its size, so billions fit in memory affordably; and several replicas share the query load. Reaching 10,000+ QPS also took tuning Milvus for this scale, mainly matching the size of Milvus’s data chunks to the team’s machines and balancing load across replicas in different zones. Milvus 2.6 has since built part of that tuning into the product.
Video deduplication: real-time HNSW search across 10B+ vectors
Uploaded video is sampled into frames, each frame is embedded, and a collection of more than 10 billion vectors is searched in real time to catch reuploads and manipulated copies, such as a swapped watermark or a mirrored picture, that titles and tags miss. Deduplication is one of the platform’s two largest vector workloads: new videos stream in constantly, the collection answers 5,000 to 10,000+ searches per second, and it has to stay up around the clock. Milvus serves it with HNSW, and because videos are almost always added rather than updated, the index rarely needs rewriting and stays fast.
Company-wide service: 100+ clusters automated with the Milvus Operator
Serving teams across the group means running Milvus as a fleet, not a single cluster. The team built that fleet on the Milvus Operator, which automates provisioning, scaling, rolling upgrades, and configuration changes, so a new production cluster is ready in minutes. Monitoring builds on Milvus’s own open-source Grafana dashboards, and Apache SeaTunnel moves data between clusters and off the old engine. The team also added an LLM-based capacity-planning agent that turns a team’s vector count, QPS, and budget into a recommended cluster size and index, which DBAs review before large launches.
Datacenter outage: three-AZ recovery on Milvus resource groups
The platform runs across three datacenters (availability zones) in the same city, so losing an entire datacenter stops neither search nor writes. Milvus’s architecture does much of the work. Most Milvus services keep no data of their own, so any copy can take over from another; the team runs copies in all three datacenters, and if one goes down, the others carry on. The query nodes are the exception, since each caches part of the data, and Milvus’s resource groups handle them by keeping a full replica of the searchable data in every datacenter. The team added enough spare capacity for the other two datacenters to carry core traffic, and runs the metadata store and message queue behind Milvus in all three as well.
Results & Benefits
Tens of billions of vectors in production. Milvus runs Kling AI and its parent company's largest workloads, including risk control, content safety, and video deduplication. The largest single cluster holds 10B+ vectors across 400+ query nodes.
10,000+ QPS at peak on billions of vectors, with writes and searches running simultaneously. Because Milvus separates writing from searching, a flood of new content no longer slows searches down- the problem that pushed Kling AI off its previous engine.
100+ production clusters, run as a company-wide service, built out in under a year. A new cluster is ready in minutes, and the platform holds a 99% alerting SLA.
Embeddings up to 10,000+ dimensions, well past the old 4,096 cap. Teams use whatever dimension their models produce, from 64 up, without changing their vector infrastructure.
Zero interruption when a data center fails. Milvus runs across three data centers in the same city with a full replica of the searchable data in each, so losing one stops neither search nor writes.
Capacity grows only where it's needed. Storage and compute scale separately, so the team adds only the resource it actually lacks, and compressed or disk-based indexes (SQ8, DiskANN) cut memory costs where the workload allows.
Overall, Milvus has become the core vector search engine for Kling AI and the other products in its group, serving workloads from risk control and deduplication to RAG and agents. When a new team needs vector search, it simply requests a Milvus cluster.
What's next
The Kling AI team aims to run Milvus even more stably and cost-efficiently, and to extend it to more scenarios across the group. It is tracking Milvus 3.0 architecture, measuring the upgrade gains, and mapping a gradual path to production. It is also exploring the lakebase capabilities arriving with 3.0, using S3 external tables to tier cold data, reuse offline data, and manage the full lifecycle of its vectors. And it plans to deepen its work with the Milvus community, turning what it learns in production into feedback and contributions.