Beatoven.ai Switches from Pinecone to Zilliz Cloud to Fuel AI-Driven Music Generation
6x
Cost Savings without Performance Loss
Smooth
Migration with Minimal Downtime
2-3 seconds
Overall Improvement in the Generation Time After Switching to Zilliz Cloud
Flexible
Data Management, Easily Unloading Datasets
Zilliz is a very integral part of our workflow. If we just swapped Zilliz Cloud with something else, the kind of loops that we get might not make sense, which means the end composition might not sound very nice or might not be very accurate to the text prompt you had given.
Sangarshanan Veera
About Beatoven.ai
Beatoven.ai is an innovative AI-driven music generation platform that empowers creators to produce original, royalty-free soundtracks tailored to their content. By leveraging advanced AI and machine learning technologies, Beatoven.ai enables users to generate and customize music based on mood, genre, and text prompts. The platform serves content creators, filmmakers, podcasters, and game developers who need high-quality, customizable audio without the complexity of music production or licensing concerns.
With an impressive reach of 1.5 million creators and over 6 million tracks generated, Beatoven.ai has established itself as a leading solution in the AI music generation space. The platform eliminates the need for either expensive music producers or limited stock music libraries by offering personalized music creation through simple prompts.
The Challenge: Finding a Reliable and Scalable Vector Database
As Beatoven.ai grew and expanded its functionality, the team faced significant challenges with their data infrastructure. Initially, they used custom mathematical formulas and algorithms to handle similarity searches for audio matching, storing data directly on disk. This approach quickly revealed limitations as the platform scaled.
"When we realized we could do better and couldn't store everything on disk as we scaled, we started looking at alternatives," explained Sangarshanan, a Senior Software Engineer at Beatoven.ai. "We needed something that would allow us to clean up our code base by removing all the functions doing similarity operations and replacing them with simple queries."
Beatoven's journey to find the right vector database solution involved multiple platforms:
Chroma DB: Their first experiment was with Chroma DB. However, Chroma is a lightweight vector database optimized for rapid prototyping and experiments. It wasn't suitable for production workloads at all.
MongoDB: Since their data was already in MongoDB, they also tried using MongoDB's vector search extension for their workloads. Unfortunately, MongoDB’s indexing algorithms didn't deliver the quality of results they expected.
Pinecone: Beatoven.ai then tried Pinecone, which was considered a great option at the time. So, they moved their data to Pinecone. While Pinecone initially performed very well in testing, it presented several challenges in production:
Network errors requiring a switch from HTTP to gRPC
Significant cost increases as they scaled up operations
"As we scaled up things on Beatoven, adding editing capabilities and more fine-grained functionalities, we realized that as we were querying Pinecone more frequently, the costs ballooned like crazy," said Sangarshanan. "We realized this wasn't something we could scale with."
The Journey to Zilliz Cloud
Facing these challenges, the Beatoven.ai team began evaluating other vector database options, including pgvector and Zilliz Cloud. While pgvector offered lower costs, its performance was "not up to the mark" compared to Pinecone, particularly in returning accurate results for audio similarity searches.
When they discovered Zilliz Cloud, they conducted A/B testing by moving one specific workload to the platform. The results were impressive: "We saw that we were getting very similar performance in Zilliz Cloud. We did not see a lot of errors, and the performance was quite comparable in terms of the results."
After this successful evaluation, Beatoven.ai decided to migrate their entire workload to Zilliz Cloud. Despite some initial challenges during the migration, the team was able to quickly transfer their embeddings to Zilliz Cloud with minimal downtime.
"The migration was quite easy," Sangarshanan noted. "I just used the Python client that was already part of Zilliz Cloud, and I just had to chunk our embeddings into 5,000 embeddings at a time and then bulk insert to Zilliz Cloud. I was able to insert all of our embeddings in probably a couple of hours once I had the script ready."
Why Beatoven.ai Choose Zilliz Cloud?
Beatoven's decision to choose Zilliz Cloud was based on several key factors:
Performance and Accuracy: For Beatoven.ai, "performance" primarily meant the accuracy of the search results. When users type prompts like "ambient soft piano" or "hard jazzy piano," the system needs to return loops that precisely match those descriptions. Zilliz Cloud delivered results that were "quite close to what we wanted," matching or exceeding the quality they experienced with Pinecone. Since the number of queries were quite high for each generation of musical piece, we saw an overall improvement of 2-3 seconds in the generation time after switching to Zilliz Cloud.
Cost-Effective Scaling Without Compromise: Unlike Pinecone, where costs increased dramatically as query volume grew, Zilliz Cloud provided seamless scalability at just one-seventh of the cost while maintaining high performance. This efficiency allowed Beatoven.ai to increase their workload significantly: "Now we're firing a lot more queries to Zilliz Cloud than we ever did with Pinecone. We fetch more loops to make our compositions more varied, yet we’re not seeing performance drop—or costs spiral out of control." With Zilliz Cloud, they’re able to get a more predictable (and flexible) pricing for varying loads.
Intuitive API Design: Sangarshanan particularly appreciated the intuitive design of Zilliz Cloud's APIs: "I found Pinecone's APIs to be a bit more different compared to how a general database API works. With Zilliz Cloud, it was quite smooth. I was able to fully understand through the documentation how things are loaded and how they work."
Flexible Data Management: The platform's ability to easily unload datasets, perform maintenance, and migrate embeddings with minimal downtime proved valuable for Beatoven's operations.
How Beatoven.ai Uses Zilliz Cloud
Zilliz Cloud is now an integral part of Beatoven's workflow, playing a crucial role in their audio generation process:
When a user provides a text prompt for music generation, it is first converted into structured queries that define musical elements such as chords, melody, bass, and percussion. These queries are then transformed into high-dimensional vector embeddings using the Contrastive Language-Audio Pretraining (CLAP) model along with a fine-tuned version of COLA, a self-supervised pre-training approach for learning a general-purpose representation of audio.
After conversion, these embeddings are sent to Zilliz Cloud for similarity search. By comparing the query embeddings with those stored in the database, Zilliz Cloud efficiently retrieves loops that best match the desired instrumental roles.
As the composition evolves—particularly for longer tracks—Zilliz Cloud continues to fetch cohesive and contextually similar loops that align with the existing structure, ensuring smooth transitions across sections like verse, chorus, and bridge. This enables Beatoven.ai to generate diverse yet harmonized compositions at scale.
"Zilliz is a very integral part of our workflow," Sangarshanan emphasized. "If we just swapped Zilliz Cloud with something else, the kind of loops that we get might not make sense, which means the end composition might not sound very nice or might not be very accurate to the text prompt you had given."
Future Plans
Looking ahead, Beatoven.ai has ambitious plans to develop a foundation model for music generation—something that doesn't currently exist in the open-source world the way it does for text.
"We're in the process of building a foundation model with Musical AI, which gives us a lot of data to work with," Sangarshanan shared. "What we want to do is create a large dataset of properly licensed songs, loops, samples, and sounds that we own and train so that the model can be used to compose and be used commercially."
While the exact role Zilliz Cloud will play in this new endeavor remains to be seen, Sangarshanan envisions possibilities for it to contribute to both training and inference processes as they build their foundation model.
Conclusion
Beatoven.ai's journey to Zilliz Cloud illustrates the importance of finding the right vector database solution for AI-driven applications. By switching from Pinecone to Zilliz Cloud, Beatoven.ai has been able to improve the accuracy of their music generation system, scale their operations cost-effectively, and lay the groundwork for future innovations in AI-generated music.
For AI companies seeking to leverage vector similarity search at scale, Beatoven's experience offers valuable insights into the benefits of a purpose-built vector database like Zilliz Cloud that combines performance, reliability, and cost-effectiveness.