Blog
How Milvus Powers Credal’s Mission for “Useful AI, Made Safe”

How Milvus Powers Credal’s Mission for “Useful AI, Made Safe”

Nov 22, 20234 min read

In today's data-driven world, integrating artificial intelligence (AI) into enterprise processes has become essential for staying competitive. But with the power of AI also comes the challenge of managing data securely, controlling access, and ensuring privacy. This is where Credal steps in, offering a solution that empowers organizations to harness the potential of Generative AI while mitigating risks. At the heart of their solution lies Milvus, an open-source vector database that is pivotal in enabling Credal's vision for "Useful AI, made safe."

Credal: Bridging the gap between GenAI and enterprise data security

Credal aims to make GenAI integration safer and more accessible for enterprises. Their core focus is seamlessly integrating data from various sources, such as Microsoft Office, Google Drive, Google Workspace, Slack, Confluence, and more, making this data available to users with full permissions and reflections. The result is a user-friendly, IT-friendly, secure environment for AI-driven insights and decision-making.

Credal: an end-to-end solution for secure, enterprise-grade GenAI deployment

From the user's perspective, this means they can tap into a wealth of data, harnessing the power of GenAI without becoming proficient in coding or complex algorithms. Credal also empowers users to create agents and experts using their data, ensuring they can make the most of GenAI technology.

For administrators and IT teams, Credal offers observability and governance tools. These tools include features for enforcing acceptable use policies, audit trails, logging, data cataloging, and data governance — all crucial elements for deploying GenAI in large enterprises effectively. Credal's Chat UI and APIs implement vital functions like PII redaction, audit logging, and data access controls, providing a pragmatic solution for integrating LLMs with corporate data.

The need for a production-ready vector database

The company faced significant technical challenges before Credal incorporated Milvus into its platform. The need for a vector database initially arose when Credal sought to implement a third-party-hosted semantic search solution. This led to the realization that vector databases could have broader applications, including allowing users to search their data and create custom workflows within the Credal platform.

While experimenting with vector search, they discovered that many solutions available in the market were suitable for prototyping and proofs of concept but needed more robustness for enterprise-scale implementations. The need for a scalable, production-ready vector database became apparent.

Choosing Milvus: an informed decision

To address their vector database needs, Credal embarked on an evaluation process. Their criteria included self-hosting, active development, scalability, and community support. In this assessment, Milvus emerged as the clear winner. Credal was impressed by Milvus's active development and level of community engagement. Milvus's GitHub stars indicated a thriving open-source community, which boded well for long-term reliability and support. Milvus also offered an official Helm chart for Kubernetes deployment, demonstrating a commitment to user-friendly installation and deployment paths.

The availability of hybrid search, combining vector search with metadata filtering, was another key factor influencing Credal's decision. This feature proved to be conspicuously missing in other options they considered.

Scalability and separation of storage and compute

Credal recognized the importance of scalability and separating storage and compute resources. Milvus's design, which intentionally separates storage and compute, offered their needed flexibility and scalability. Unlike single-node solutions, Milvus's architecture allowed a more agile response to changing demands.

Credal's Co-founder & CTO, Jack Fischer, emphasizes the significance of this design decision: “Especially when we were just getting started, we didn't know exactly what our access patterns would be like. As our product evolves, our access patterns definitely evolve...So we were happy to see the separation between storage and compute. We were confident that we could get that to work for us, regardless of where we took the product.”

Milvus: the clear choice for Credal's vector database needs

Milvus outshone all open-source alternatives considered. Credal's evaluation revealed that only one platform met all their vector database requirements. Milvus's ability to support various use cases and its technical prowess made it the ideal choice for Credal's mission.

"Lucky for us, we didn't have to choose the least bad one, but the one that actually really excelled at everything we wanted," says Fischer, "If you take the middle of the Venn diagram, at eight circles and they're all overlapping, I see there is only one in the middle and it's Milvus, so Milvus hit everything, and nothing else hit everything. So if you only care about some of these, you could pick other options, but if you want everything to work, it has to be Milvus."

Integrating Milvus into Credal's application ecosystem has proven to be a game-changer. With Milvus's text embeddings, Credal can efficiently process and distill large pieces of content, making them accessible to advanced LLM applications. The versatility of text embeddings significantly enhances Credal's search, filtering, and data curation capabilities.

Credal's architecture not only focuses on advanced technology but also on high-quality data integrations. They prioritize interpreting data from source systems in their proprietary format and ensuring that this data is effectively communicated to the GenAI models. This meticulous approach sets Credal apart from typical demoware development and enables them to deliver precise and intuitive results.

Benefits and future plans

Milvus enables Credal to offer a scalable, easy-to-use solution to their customers. This eliminates the need for extensive engineering efforts from scratch and reduces the operational overhead. Milvus's scalability and robustness ensure the platform can grow with Credal as its product evolves.

As for future plans, Credal is considering the use of Zilliz Cloud for its cloud-based customers. Security, governance, and effective GenAI functionality are central to their roadmap. Credal primarily serves enterprises with 1,000 to 5,000 employees and is committed to making GenAI work seamlessly in real-life enterprise scenarios. With Milvus at its core, Credal is poised to lead the way in delivering "Useful AI, made safe" to businesses worldwide.

Anya Sage
Anya Sage is a freelance tech marketing copywriter/copyeditor. She has extensive experience writing and editing for the tech industry. Through her company The Write Cure, she helps tech companies build and shape their marketing content for clarity and impact.