Blog
New for Zilliz Cloud: Cardinal Search Engine, GCP Marketplace, Databricks Connector and More

New for Zilliz Cloud: Cardinal Search Engine, GCP Marketplace, Databricks Connector and More

Jan 30, 20247 min read

Our latest Zilliz Cloud product launches are live, with new features to drive real business impact through AI innovations with our vector database and data services. What's new? Zilliz Cloud significantly enhances vector search performance while ensuring enterprise-grade security and seamless data integration.

Key introductions include delivering a remarkable 10x performance boost, simplifying budgeting, payment, and procurement processes, eliminating the need for custom code, and enabling smooth data processing and indexing for unstructured data.

Here’s an overview of our new features–read on for more details:

New Zilliz Cloud features:
Join us in our Launch webinar to see these innovations in action

Cardinal Search Engine: 10X faster search, 50% more capacity

Designed from the ground up with modern C++ and practical Approximate Nearest Neighbor Search (ANNS) methods, Cardinal is a multi-threaded, high-efficiency vector search engine. It handles tasks ranging from brute-force searches to modifying ANNS indices and working with various data formats, including FP32, FP16, and BF16. Cardinal's primary focus is speed and efficiency, enabling you to process more user requests within your resource constraints.

The innovation behind Cardinal is a testament to our humble beginnings, where we wholeheartedly embraced heterogeneous computing. Our team has fine-tuned algorithms, employed low-level specialized optimized kernels for compute-intensive operations, and ensured support for multiple hardware platforms, including x86 and ARM. Cardinal leverages cutting-edge technologies like AVX-512 extensions for x86 and NEON and SVE instruction sets for ARM, delivering optimized code for efficient computation. These meticulous optimizations guarantee that Cardinal operates at peak performance, making it the fastest vector search engine in the industry.

With Cardinal, our customers can achieve a remarkable 10x performance boost versus open-source Milvus and process queries at lightning speed while maintaining a high recall rate. Whether you're dealing with large datasets or require fast responses, Cardinal ensures your vector searches execute swiftly and efficiently. Speedy query results enhance user experiences and provide a competitive advantage for your AI applications. You can read more about what real-world use cases inspired us to build Cardinal.

"We've been impressed with the performance of Zilliz Cloud, particularly under heavy data loads," said Alex Alexander, CEO & Founder of Picdmo. "The introduction of the Cardinal search engine, with its promise to increase Milvus' performance and double that of the previous Zilliz Cloud, will be a game-changer for us. This leap in efficiency, coupled with the latest advancements in security, will significantly elevate our capability to offer advanced picture similarity searches for our customers."

Check out our Cardinal Search Engine blog to learn more.

Milvus 2.3 generally available on Zilliz Cloud: advanced vector search features ready for production workloads

After a four-month beta phase, the much-anticipated Milvus 2.3 is generally available on Zilliz Cloud. Zilliz Cloud users can now confidently utilize these advanced vector search and data management features in a production environment.

A few feature highlights in this version include:

Cosine Similarity: Advanced capabilities in similarity calculations, eliminating the need for prior vector normalization and streamlining query processes.
Upsert Function: Simplified data management with enhanced efficiency in updating and deleting datasets, particularly valuable in dynamic environments where data consistency and atomicity are crucial.
Range Search: This feature provides more precise data retrieval methods for some use cases by broadening the scope of vector querying beyond top-K search. It's particularly beneficial for recommendation engines, ensuring more relevant suggestions for users.
Apache Parquet File Support: Improved data handling capabilities with Parquet's efficient columnar storage format, offering better compression and query performance, especially with complex datasets.
Array Data Type Support: Enabling precise metadata filtering for advanced searches based on multiple attributes. For example, in e-commerce, this feature allows advanced searches based on different product tags, ensuring highly relevant search results for users.

GCP Marketplace integration: simplifying budget planning, payment, and procurement

Zilliz Cloud's integration with Google Cloud Marketplace is live, providing developers a seamless experience to leverage its capabilities. You can tap into Google Cloud's resources and infrastructure without the complexity of separate payment and procurement procedures. This integration also simplifies your experience by eliminating the need for upfront budget planning and commitments for your new AI initiatives. As your project evolves, you can scale your clusters and pay as you need, aligning resources with your project's growth.

You have flexibility in choosing how to access Zilliz Cloud through Google Cloud Marketplace. You can set up GCP Marketplace as your payment method in Zilliz Cloud's UI or subscribe directly through the Google Marketplace listing. For developers who sign up through GCP Marketplace, there is an additional bonus: you'll receive the standard $100 credit available to new users and an extra $100 credit.

Add GCP Marketplace in Zilliz Cloud UI

So far, Zilliz Cloud has been available on three major cloud providers, AWS, Azure, and GCP, across 8 regions worldwide. This extensive availability ensures you can access Zilliz Cloud's powerful capabilities wherever your projects are hosted. It provides flexibility and accessibility, allowing you to deploy your AI applications easily.

Role-based Access Control: enable granular access controls to critical resources at scale

Role-based access control (RBAC) in Zilliz Cloud provides a structured and scalable approach to managing permissions for data access security. Over the past few months, we have introduced multiple features, making Zilliz Cloud's RBAC system more granular and comprehensive than any vector database vendor on the market. RBAC is separated into two layers: the control and data layers. In the control layer, roles govern the operational permissions for resources such as clusters, projects, users, and billing. Zilliz Cloud supports four roles in the control layer, with organization administrators, project owners, and project members being the three commonly used roles:

Organization Administrator: This role has full management permissions for the organization, including organization settings, payment methods, billing, API keys, all projects within the organization, and related resources.
Project Owner: Project owners have full management permissions for projects, including project settings, all clusters within the project, API keys, and other related resources.
Project Member: Project members have read-write permissions for all clusters within the project, the ability to view cluster details, and manage collections and indexes.

In contrast, in the data layer, roles focus on controlling the ability to add, delete, modify, and access data within clusters. Zilliz Cloud offers three built-in roles in the data layer: Admin, Read-Only, and Read-Write, which control read and write permissions and management rights for cluster data. Additionally, for specific business requirements, such as allowing authorized personnel access to sensitive data or using collections as multi-tenant isolation units to ensure secure data access, Zilliz Cloud enables users to create custom roles. These custom roles can define permissions to be upheld for specific collections, partitions, or operations, ensuring the principle of minimizing data permissions when using Zilliz Cloud.

With granular RBAC across both control and data layers, companies can achieve fine-grained control over access to their data, enhancing security and compliance while facilitating collaboration among teams and ensuring that users have the appropriate level of access for their roles and responsibilities.

Databricks Connector: leverage Databricks' powerful data processing alongside Zilliz Cloud's vector index and search with no custom code required

We've recently expanded our data connectivity and transformation capabilities by introducing the Confluent and Airbyte Connectors. Our commitment to providing out-of-the-box solutions for our users' data integration needs drove these releases. Let's move forward to our next milestone of creating an unstructured data platform–the Databricks Connector.

The Databricks Connector offers an easy data migration and transformation solution, simplifying the development process for many AI use cases. Whether you're a team with ML research capabilities looking to update embedding models or an individual user wanting to upload data frame records directly from Spark to Milvus, this connector provides the flexibility you need.

With the Databricks Connector, you can import data to Zilliz Cloud in two ways: streaming for real-time updates and batch for large datasets. Check out our example notebooks for a step-by-step guide on how to use it effectively.

Other security updates

Zilliz Cloud has achieved SOC2 Type 2 and ISO27001 compliance certificates in the past few months. These certifications demonstrate our commitment to upholding the highest security and data protection standards so that you can trust Zilliz Cloud to manage your valuable data securely, ensuring confidentiality, integrity, and availability.

Building more robust AI applications with Zilliz Cloud features

Want to learn more? Please sign up for the Zilliz Cloud Q1 ‘24 Launch webinar on Feb 1st. You’ll learn how Zilliz Cloud empowers transforming unstructured data and AI applications through unparalleled performance and scalability from our VP of engineering.

Save Your Spot

Ready to get going? if you still haven’t done so, start for free with Zilliz Cloud with no installation hassles and without requiring a credit card. You can also start your 30-day free trial of the Standard plan with $100 worth of credits upon registration. You can contact us through our support portal if you encounter problems or questions when using Zilliz Cloud.

Steffi Li
Steffi is the Director of Product Marketing at Zilliz, with prior experience leading the GTM strategies for open-source data technologies such as Apache Kafka and Apache Airflow. Passionate about travel, yoga, and non-fiction reading, she brings a balanced and enriched worldview to her professional pursuits.