How to Pick the Right Vector Database for Your Use Case

How to Choose A Vector Database: Weaviate Cloud vs. Zilliz Cloud

Sep 21, 20236 min read

Compare Weaviate Cloud vs. Zilliz Cloud and Milvus in this in-depth benchmark, cost, and features comparison.

Read the entire series

This Weaviate comparison was last updated on September 22, 2023. To provide you with the latest findings, this blog will be regularly updated with the latest benchmark figures.

Not all vector databases are made equal: unveiling the distinction

As the demand for vector databases surges, the industry is evolving beyond traditional databases and search systems with hastily integrated vector search plugins. For instance, Weaviate, an open-source vector database, has carved out a niche by emphasizing ease of use and developer-friendliness, complete with a streamlined setup process and well-documented APIs. Zilliz/Milvus, meanwhile, stands out for its ability to handle extremely large-scale, high-performance, low-latency applications.

Both databases are purpose-built to manage vector data but serve different needs. Weaviate is a strong choice for developers seeking quick and straightforward implementation. For applications that demand extreme scale, high performance, and low latency, the developers at Zilliz/Milvus engineered it to excel. Its architecture is optimized for these critical performance metrics, making it a robust and innovative solution for the most demanding vector database applications.

We will venture beyond these distinctions within this Weaviate vs. Zilliz Cloud comparison, delving into several other comparison areas. We'll delve into benchmarks to offer a performance perspective and perform an in-depth Weaviate Cloud feature analysis.

Benchmarking Weaviate Cloud and Zilliz Cloud

At Zilliz, developers and architects alike ask us, "How does Zilliz measure up to Weaviate Cloud for vector embedding workloads?" This curiosity might spring from various motivations, all stemming from building semantic similarity search for capabilities like product recommenders or reverse image search, and lately, when building LLM applications with Retrieval Augmented Generation (RAG).

Over the past few weeks, we undertook a comprehensive analysis, delving into the performance and attributes of Zilliz Cloud vs. Weaviate Cloud regarding vector embedding workloads. With our open-source benchmark tool, VectorDBBench, our exploration focused intently on critical aspects such as queries per second (QPS), queries per dollar (QP$), and latency.

Datasets tested

We’ve tested two types of datasets to benchmark Zilliz Cloud and Weaviate Cloud.

Dataset 1: 1,000,000 vectors with 768 dimensions
Dataset 2: 500,000 vectors with 1,536 dimensions

Products tested

We’ve tested the following options of Zilliz Cloud and Weaviate Cloud.

Zilliz Cloud (1cu-perf): Zilliz Cloud with one performance-optimized compute unit
Zilliz Cloud (1cu-cap): Zilliz Cloud with one capacity-optimized compute unit
Zilliz Cloud (2cu-cap): Zilliz Cloud with two capacity-optimized compute units
Weaviate Cloud (Standard)
Weaviate Cloud (Business Critical)

Note: For more information about Zilliz Cloud’s compute units, see Zilliz blog introducing Zilliz Cloud CU type.

Queries per second (QPS)

Zilliz provides different hardware options that outperform the standard and business-critical plans of Weaviate Cloud in terms of QPS when processing 1 million vectors with 768 dimensions. Specifically, Zilliz achieves 9x, 8x, and 5x higher QPS than Weaviate Cloud's business plan, which is their best option for QPS.

When processing 500,000 vectors with 1,563 dimensions, Zilliz options outperform the standard and business-critical plans of Weaviate Cloud in terms of QPS. Specifically, Zilliz achieves 8x, 6x, and 3x higher QPS than Weaviate Cloud's best option for QPS.

This is a 1-100 score based on each system's performance in different cases according to a specific rule. A higher score denotes better performance.

Our tests reveal that Weaviate Cloud's two plans score around 12 points for comprehensive QPS performance, significantly lower than Zilliz's more favorable plan by 84 points and 35 points less than Zilliz's least favorable option.

Queries per $ (QP$)

When processing 1 million vectors with 768 dimensions, all Zilliz options with different hardware resource setups are more efficient than Weaviate Cloud in terms of QP$. Specifically, Zilliz is 520x, 332x, and 292x more cost-effective than Weaviate's standard plan, their best option for QP$.

When processing 500,000 vectors with 1,536 dimensions, all Zilliz options with different hardware resource setups are more efficient than Weaviate Cloud in terms of QP$. Specifically, Zilliz is 403x, 258x, and 194x more cost-effective than Weaviate's standard plan, their best option for QP$.

This is a 1-100 score based on each system's performance in different cases according to a specific rule. A higher score denotes better performance.

According to our test results, Weaviate's two plans score approximately 1 point for comprehensive QP$ performance, which is significantly lower than Zilliz's most favorable plan by 71 points and 49 points less than Zilliz's least favorable option.

Latency

When processing 1 million vectors with 768 dimensions, all Zilliz options with different hardware resource setups are much faster than Weaviate Cloud in terms of P99 latency. Specifically, Zilliz is 26x, 20x, and 19x faster than Weaviate's business plan, their best option for P99 latency.

When processing 500,000 vectors with 1,536 dimensions, all Zilliz options with different hardware resource setups are much faster than Weaviate Cloud in terms of P99 latency. Specifically, Zilliz is 32x, 18x, and 7x faster than Weaviate's business plan, their best option for P99 latency.

This is a >1 score based on each system's performance in different cases according to a specific rule. A lower score denotes better performance.

According to our test results, Weaviate Cloud's two plans score approximately 6 points for comprehensive P99 latency performance, which is much higher (slower) than Zilliz Cloud's various options by 4 points.

We conducted these benchmarks using the open-source VectorDB Bench tool on GitHub, where you’ll also find the updated leaderboard. VectorDBBench provides unbiased vector database benchmark results for mainstream vector databases and cloud services. Designed for ease of use, VectorDB Bench makes finding the optimal choice among a plethora of vector database cloud services and open-source vector databases a breeze.

Weaviate Cloud features comparison

Performance is the biggest challenge with vector databases as the number of unstructured data elements stored in a vector database grows into hundreds of millions or billions, and horizontal scaling across multiple nodes becomes paramount. Furthermore, differences in insert rate, query rate, and underlying hardware may result in different application needs, making overall system tunability a mandatory feature for vector databases.

What’s your vector database for?

A vector database is a fully managed solution for storing, indexing, and searching across a massive dataset of unstructured data that leverages the power of embeddings from machine learning models. A vector database should have the following features:

Scalability and tunability
Multi-tenancy and data isolation
A complete suite of APIs
An intuitive user interface/administrative console

Scalability

	Zilliz Cloud	Weaviate Cloud
Separation of storage and compute	√	x
Separation of query and insertions	Yes. At the component level (which provides more fine-grained scalability).	No. Only scale at the server level.
Multi-replication	√	√
Billion-scale vector support	√	√
Dynamic segment placement vs. static data sharding	Dynamic segment placement	Static sharding

Functionality

	Zilliz Cloud	Weaviate Cloud
Role-based Access Control (RBAC)	√	x
Disk Index support	√	x
Hybrid Search (ie Scalar filtering)	Yes with Scalar filtering	Yes (combine Sparse and Dense Vectors)
Partitions/namespaces/logical groups	√	x

Purpose-built

Features	Zilliz Cloud	Weaviate Cloud
Purpose-built for vectors	√	√
Tunable consistency	√	√
Support for both stream and batch of vector data	√	√
Binary vector support	x	√
Multi-language SDK	Python, Java, Go, C++, Node.js	Python, Java, Go

For more details, visit our Milvus vs Weaviate comparison page.

What’s next?

Steffi Li
Steffi is the Director of Product Marketing at Zilliz, with prior experience leading the GTM strategies for open-source data technologies such as Apache Kafka and Apache Airflow. Passionate about travel, yoga, and non-fiction reading, she brings a balanced and enriched worldview to her professional pursuits.

Next: Benchmarking Vector Database Performance: Techniques and Insights

Content

Start Free, Scale Easily

Try the fully-managed vector database built for your GenAI applications.

Try Zilliz Cloud for Free