How to Choose A Vector Database: Weaviate Cloud vs. Zilliz Cloud
This Weaviate comparison was last updated on September 22, 2023. To provide you with the latest findings, this blog will be regularly updated with the latest benchmark figures.
Not all vector databases are made equal: unveiling the distinction
As the demand for vector databases surges, the industry is evolving beyond traditional databases and search systems with hastily integrated vector search plugins. For instance, Weaviate, an open-source vector database, has carved out a niche by emphasizing ease of use and developer-friendliness, complete with a streamlined setup process and well-documented APIs. Zilliz/Milvus, meanwhile, stands out for its ability to handle extremely large-scale, high-performance, low-latency applications.
Both databases are purpose-built to manage vector data but serve different needs. Weaviate is a strong choice for developers seeking quick and straightforward implementation. For applications that demand extreme scale, high performance, and low latency, the developers at Zilliz/Milvus engineered it to excel. Its architecture is optimized for these critical performance metrics, making it a robust and innovative solution for the most demanding vector database applications.
We will venture beyond these distinctions within this Weaviate vs. Zilliz Cloud comparison, delving into several other comparison areas. We'll delve into benchmarks to offer a performance perspective and perform an in-depth Weaviate Cloud feature analysis.
Benchmarking Weaviate Cloud and Zilliz Cloud
At Zilliz, developers and architects alike ask us, "How does Zilliz measure up to Weaviate Cloud for vector embedding workloads?" This curiosity might spring from various motivations, all stemming from building semantic similarity search for capabilities like product recommenders or reverse image search, and lately, when building LLM applications with Retrieval Augmented Generation (RAG).
Over the past few weeks, we undertook a comprehensive analysis, delving into the performance and attributes of Zilliz Cloud vs. Weaviate Cloud regarding vector embedding workloads. With our open-source benchmark tool, VectorDBBench, our exploration focused intently on critical aspects such as queries per second (QPS), queries per dollar (QP$), and latency.
We’ve tested two types of datasets to benchmark Zilliz Cloud and Weaviate Cloud.
Dataset 1: 1,000,000 vectors with 768 dimensions
Dataset 2: 500,000 vectors with 1,536 dimensions
We’ve tested the following options of Zilliz Cloud and Weaviate Cloud.
Zilliz Cloud (1cu-perf): Zilliz Cloud with one performance-optimized compute unit
Zilliz Cloud (1cu-cap): Zilliz Cloud with one capacity-optimized compute unit
Zilliz Cloud (2cu-cap): Zilliz Cloud with two capacity-optimized compute units
Weaviate Cloud (Standard)
Weaviate Cloud (Business Critical)
Note: For more information about Zilliz Cloud’s compute units, see Zilliz blog introducing Zilliz Cloud CU type.
Queries per second (QPS)
Zilliz provides different hardware options that outperform the standard and business-critical plans of Weaviate Cloud in terms of QPS when processing 1 million vectors with 768 dimensions. Specifically, Zilliz achieves 9x, 8x, and 5x higher QPS than Weaviate Cloud's business plan, which is their best option for QPS.
When processing 500,000 vectors with 1,563 dimensions, Zilliz options outperform the standard and business-critical plans of Weaviate Cloud in terms of QPS. Specifically, Zilliz achieves 8x, 6x, and 3x higher QPS than Weaviate Cloud's best option for QPS.
Our tests reveal that Weaviate Cloud's two plans score around 12 points for comprehensive QPS performance, significantly lower than Zilliz's more favorable plan by 84 points and 35 points less than Zilliz's least favorable option.
Queries per $ (QP$)
When processing 1 million vectors with 768 dimensions, all Zilliz options with different hardware resource setups are more efficient than Weaviate Cloud in terms of QP$. Specifically, Zilliz is 520x, 332x, and 292x more cost-effective than Weaviate's standard plan, their best option for QP$.
When processing 500,000 vectors with 1,536 dimensions, all Zilliz options with different hardware resource setups are more efficient than Weaviate Cloud in terms of QP$. Specifically, Zilliz is 403x, 258x, and 194x more cost-effective than Weaviate's standard plan, their best option for QP$.
According to our test results, Weaviate's two plans score approximately 1 point for comprehensive QP$ performance, which is significantly lower than Zilliz's most favorable plan by 71 points and 49 points less than Zilliz's least favorable option.
When processing 1 million vectors with 768 dimensions, all Zilliz options with different hardware resource setups are much faster than Weaviate Cloud in terms of P99 latency. Specifically, Zilliz is 26x, 20x, and 19x faster than Weaviate's business plan, their best option for P99 latency.
When processing 500,000 vectors with 1,536 dimensions, all Zilliz options with different hardware resource setups are much faster than Weaviate Cloud in terms of P99 latency. Specifically, Zilliz is 32x, 18x, and 7x faster than Weaviate's business plan, their best option for P99 latency.
According to our test results, Weaviate Cloud's two plans score approximately 6 points for comprehensive P99 latency performance, which is much higher (slower) than Zilliz Cloud's various options by 4 points.
We conducted these benchmarks using the open-source VectorDB Bench tool on GitHub, where you’ll also find the updated leaderboard. VectorDBBench provides unbiased vector database benchmark results for mainstream vector databases and cloud services. Designed for ease of use, VectorDB Bench makes finding the optimal choice among a plethora of vector database cloud services and open-source vector databases a breeze.
Weaviate Cloud features comparison
Performance is the biggest challenge with vector databases as the number of unstructured data elements stored in a vector database grows into hundreds of millions or billions, and horizontal scaling across multiple nodes becomes paramount. Furthermore, differences in insert rate, query rate, and underlying hardware may result in different application needs, making overall system tunability a mandatory feature for vector databases.
What’s your vector database for?
A vector database is a fully managed solution for storing, indexing, and searching across a massive dataset of unstructured data that leverages the power of embeddings from machine learning models. A vector database should have the following features:
Scalability and tunability
Multi-tenancy and data isolation
A complete suite of APIs
An intuitive user interface/administrative console
|Zilliz Cloud||Weaviate Cloud|
|Separation of storage and compute||√||x|
|Separation of query and insertions||Yes. At the component level (which provides more fine-grained scalability).||No. Only scale at the server level.|
|Billion-scale vector support||√||√|
|Dynamic segment placement vs. static data sharding||Dynamic segment placement||Static sharding|
|Zilliz Cloud||Weaviate Cloud|
|Role-based Access Control (RBAC)||√||x|
|Disk Index support||√||x|
|Hybrid Search (ie Scalar filtering)||Yes with Scalar filtering||Yes (combine Sparse and Dense Vectors)|
|Features||Zilliz Cloud||Weaviate Cloud|
|Purpose-built for vectors||√||√|
|Support for both stream and batch of vector data||√||√|
|Binary vector support||x||√|
|Multi-language SDK||Python, Java, Go, C++, Node.js||Python, Java, Go|
For more details, visit our Milvus vs Weaviate comparison page.
- Not all vector databases are made equal: unveiling the distinction
- Benchmarking Weaviate Cloud and Zilliz Cloud
- Weaviate Cloud features comparison
- What’s next?
Take Zilliz for a Spin for FreeGet Started Free
Share this article