How a World-Leading Music Platform Searches 100M+ Songs by Sound and Lyrics with Milvus
100M+ tracks
The entire catalog embedded and searchable, including the long tail
~10 ms P99
A match across billions of audio embeddings, at hundreds of searches a second
By sound and meaning
Find a song from a hum, a clip, or a paraphrased lyric, not just its title
About the Music Platform
The platform is one of the world's largest music streaming services, with a catalog of more than 100 million tracks and hundreds of millions of listeners across more than 100 countries.
Its listeners do far more than press play. They hold up a phone to name a song playing across a room, type a half-remembered line to find a track, ask for "more like this" to fill an evening, and upload their own audio that has to be cleared against the catalog for rights. Each one searches the whole catalog, and it has to land on the right track instantly. That search layer runs on Milvus.
The Challenge
The platform's catalog runs past 100 million tracks, and the way listeners search it created three demands at once.
Music search has to work on sound and meaning, not just keywords. A listener hums a tune, plays five seconds from a speaker, or types a lyric the way they remember it rather than the way it was written. There is no title or tag to match on. The platform needed search that works on what a song sounds like and what a lyric means — something keyword matching alone cannot do.
The catalog is billions of vectors, and it all has to stay searchable. Matching on sound means slicing every track into short segments and embedding each one, so 100 million-plus tracks become billions of audio vectors, plus a text vector for every set of lyrics. Every one has to stay online at once, including the long tail, or the obscure track nobody has played in a year is exactly the one search fails to find.
Every search has to return in real time, at scale. Identifying a song or answering a lyric query is something a listener waits on, so it has to come back in milliseconds, at hundreds of searches a second, across those billions of vectors. Speed cannot degrade as the catalog keeps growing.
Why Milvus
Milvus answers all three demands directly, which is why the team built its search on it.
Purpose-built for vector, full-text, and metadata-filtered search in one query. Milvus stores audio and lyric embeddings and finds nearest neighbors based on how a clip sounds or what a lyric means, so a hum or a paraphrased line resolves to the right track. For lyrics, it combines full-text keyword search with vector search in a single hybrid query — matching the words and the meaning together — and applies metadata filters (artist, genre, territory, rights) in the same call. That is hybrid search a keyword-only system cannot deliver.
Proven at billions of vectors. Milvus is a distributed database designed for billion-scale collections, so the entire embedded catalog lives in one system and scales out with the cluster as the library grows, with no re-architecture at each new milestone.
Millisecond latency at high throughput. Milvus returns a match in about 10 milliseconds at hundreds of queries a second across billions of vectors, and isolates heavy indexing from live queries so search stays fast while new music is added.
Open source, run on their own infrastructure. Beyond meeting these demands, Milvus is open source, so teams operating at this scale run it on their own infrastructure, tune and extend it per workload, and integrate it into their stack with no vendor lock-in.
The Solution
Milvus is the similarity-search layer beneath every feature that finds music by how it sounds or what it means. Audio and text both become vectors and live in Milvus next to the metadata used to filter them.
Anonymous Music Insert.png
Indexing the catalog. Every track is cut into short, overlapping clips, and an audio model turns each clip into a vector that captures how it sounds: its timbre, melody, and rhythm. One track therefore becomes a run of vectors, one every few seconds, instead of a single point. Lyrics are handled separately by a text model: each set is embedded as a vector and indexed for full-text search, so a query can match on meaning or on the exact words. Artist, album, genre, release date, popularity, and territory/rights are written as scalar fields on the same records. The audio clips form a billion-scale collection; the lyric vectors, full-text index, and metadata sit beside them. A graph-based ANN index (HNSW) keeps recall fast, and scalar indexes make the metadata filterable.
Answering a query. Four product features all resolve to a similarity search over Milvus:
- Song recognition. A few seconds of captured audio are embedded and searched against the audio collection in Milvus, where each track is stored as many short clips in order. The recording matches a run of consecutive clips from the same song, something only the right track can produce, so Milvus pins it to that one song and returns it in real time.
- Lyric search. Each lyric is stored in Milvus with both a semantic dense vector and a full-text (BM25) index on the same record, so a single hybrid search matches meaning and wording together and returns one ranked list. Whether a listener remembers the line word-for-word or only its gist, the query lands on the right song.
- Recommendation. The track a listener is playing becomes the query, and Milvus returns sonically similar tracks for radio and discovery, filtered to the listener's territory and taste.
- Copyright matching. An uploaded clip is chunked and searched against the catalog; a run of high-similarity hits flags reuse of protected material, and the rights fields resolve ownership.
All four features search the same data in Milvus: the same audio and lyric vectors, the same metadata, and the same filters carried in each query. What differs is the tuning. The team self-hosts the retrieval layer as a fleet of Milvus clusters and sizes each to its job: the recognition path for the lowest latency, the full-catalog collections for throughput and recall as they scale into the billions. New music flows through the same indexing path and is searchable the moment it lands.
Results & Benefits
- Any song is identified in about 10 milliseconds, at hundreds of searches a second. A few seconds of audio, or a single line of text, returns the exact track fast enough to feel instant to the listener, even under high concurrent load.
- All 100 million-plus tracks stay searchable, across billions of vectors. Every track is embedded and online, including the long tail, so no song is unfindable because the catalog grew too large.
- Songs are found by sound and by meaning, not just by title. A hummed melody, a five-second clip, or a misremembered lyric resolves to the right track, something keyword search could never do.
- Recognition, lyric search, recommendation, and copyright matching run on one engine. A new audio feature starts from a layer that already searches the full catalog in real time, not from a new system to stand up.
- It all runs on the team's own infrastructure, scaling by growing the cluster. As traffic and the catalog grow, the team adds capacity to the Milvus clusters it already operates, with no redesign of the search layer.
The strategic payoff is that "search by sound" stops being a project and becomes a capability. Anything the platform wants to build on how music sounds starts from a foundation that already searches all 100 million-plus tracks in real time.
Get started with Milvus
Milvus is the world's most widely adopted open-source vector database with 46K+ GitHub stars, built for billion-scale similarity search. It runs anywhere from a laptop to a distributed cluster and gives teams full control over how they index, filter, and scale vector workloads.
Get started with Milvus on GitHub, read the docs, or join the community on Discord.
Teams that would rather not run Milvus themselves can use the same engine as a fully managed service on Zilliz Cloud, with a free tier to start.
- About the Music Platform
- The Challenge
- Why Milvus
- The Solution
- Results & Benefits
- Get started with Milvus
Content
Industry
Entertainment