Ivy.ai Scales GenAI-Powered Communication with Zilliz Cloud
Improved Reliability
Since adopting Zilliz Cloud, Ivy.ai has experienced no outages or significant issues, ensuring consistent service for its clients.
Enhanced Scalability
The ability to easily scale resources has supported Ivy.ai's growth and its free trial approach for customer acquisition.
Optimized Performance
Zilliz Cloud's high-performance vector database has enabled Ivy.ai to maintain quick response times, even as their data has grown by over 200% since they started using Zilliz Cloud.
I was benchmarking different vector databases, and Milvus was at the top among all the popular options on the market.
Saumil Patel
About Ivy.ai
Ivy.ai is the brainchild of a team dedicated to enhancing communication in higher education, healthcare, and other public sectors. Their flagship product, IvyQuantum™, released in 2023, is an advanced AI chatbot beyond simple Q&A. It offers:
Omnichannel capabilities (web, SMS, voice, WhatsApp, social media)
Live agent support
AI-generated email responses
Seamless integrations for accessing account-specific information and lead collection
Beyond its rich capabilities, what sets Ivy.ai apart is also its commitment to data security and privacy, a crucial factor for institutions handling sensitive information. By leveraging institution-specific data sources, Ivy.ai ensures that each implementation is tailored to the unique needs of its clients.
The Challenge: Scaling GenAI for Education
The team at Ivy.ai encountered several hurdles as they transitioned from their retrieval-based Ivy 1.0 to the more advanced, generative AI-powered IvyQuantum™. The team needed a solution that could:
Handle billions of vectors for their new product
Deliver high-speed queries for real-time AI-generated responses
Provide reliable performance as data volume and requirements grew
Allow the team to focus on innovation rather than infrastructure management
In their quest for the right technology, Ivy.ai's dedication was unwavering. They initially chose Milvus as their vector database. Saumil Patel, the Senior Data Scientist from Ivy.ai, explains, "I was benchmarking different vector databases, and Milvus was at the top among all the popular options on the market." The team was impressed by Milvus' performance in queries per second, inserts, and recalls—crucial metrics for their GenAI-powered communication platform. They needed a solution to find the top hits from the vector database within sub-seconds when users asked questions.
With Milvus selected, Ivy.ai attempted to self-host it using Kubernetes on the Google Cloud Platform. However, this approach proved challenging. Michal Oglodek, the CTO of Ivy.ai, recounts, "We encountered significant issues with the system failing in production due to the growing data load and increasing requirements. Recognizing the need for a more sustainable solution, we made the strategic business decision to avoid dedicating extensive time and resources to managing the system ourselves. Instead, we opted for a more reliable and scalable approach."These stability issues, coupled with the increasing demands of their growing business, led Ivy.ai to seek a more scalable, managed solution that could provide the performance of Milvus without the operational headaches.
Enter Zilliz Cloud: A Fully-managed Vector Database Solution at Scale
After a thorough evaluation, Ivy.ai chose Zilliz Cloud, a fully managed version of the Milvus vector database they selected initially. This decision was based on several key factors that aligned perfectly with Ivy.ai's needs. Zilliz Cloud provided a high-performance, scalable solution capable of handling billions of vectors, which was essential for IvyQuantum™'s Retrieval Augmented Generation (RAG) applications. By opting for Zilliz Cloud’s fully managed service, Ivy.ai eliminated the need to handle complex database administration, allowing them to focus on their core business.
How lvy.ai uses Zilliz Cloud to power its chatbot
The scalability of Zilliz Cloud's solution was particularly appealing. Oglodek explains, "What we wanted to do was scale easily, and with your software and service, we could keep increasing our capabilities to match our needs." This flexibility was crucial for Ivy.ai's growing business demands. Moreover, the managed service provided by Zilliz Cloud resolved the stability issues Ivy.ai had experienced with self-hosted solutions, ensuring reliable performance.
Patel particularly appreciated the monitoring and analytics capabilities, stating, "I can see exactly the queries per second and the total capacity of the database we have." This level of insight into their database performance proved invaluable for optimizing their operations and planning for future growth.
The Results: A Foundation for Innovation
Implementing Zilliz Cloud as their vector database solution has yielded significant benefits for Ivy.ai:
Improved Reliability: Since adopting Zilliz Cloud, Ivy.ai has experienced no outages or significant issues, ensuring consistent service for its clients.
Enhanced Scalability: The ability to easily scale resources has supported Ivy.ai's growth and its free trial approach for customer acquisition.
Optimized Performance: Zilliz Cloud's high-performance vector database has liberated Ivy.ai from the worry of maintaining quick response times, even as their data has grown by over 200% since they started using Zilliz Cloud.
Oglodek succinctly summarized their experience: "Reliability, scalability, and performance are great. And the fact that we can outsource all this administration management and don't have to worry about servers breaking helps us put resources on product development and innovation, which truly differentiates our business".
Looking to the Future
With a stable, scalable, and high-performance foundation provided by Zilliz Cloud, Ivy.ai is well-positioned for future growth enabled by GenAI. Sharon Harrison, the SVP of Marketing from Ivy.ai, emphasizes that "innovation is at the core of our company," hinting at several projects on the horizon that will require continued support from Zilliz Cloud.
As GenAI continues transforming the education landscape, Ivy.ai, powered by Zilliz Cloud's vector database solution, is at the forefront, ready to meet the evolving needs of institutions and students alike.
Reliability, scalability, and performance are great. The fact that we can outsource all this kind of administration management and don't have to worry about servers breaking helps us put resources into product development and innovation, which truly differentiates our business.
Michal Oglodek