How to Choose A Vector Database: Elastic Cloud vs. Zilliz Cloud
This Elastic Cloud comparison was last updated on September 6, 2023. To provide you with the latest findings, this blog will be regularly updated with the latest benchmark figures.
Vector databases vs. vector search plugins: unveiling the distinction
As vector databases continue to garner attention, traditional databases and search systems like Elasticsearch are racing to integrate dedicated vector search plugins. A prime example is Elasticsearch 8.0, which introduces upgraded features, including vector insertion and Approximate Nearest Neighbor (ANN) search capabilities, conveniently accessible through RESTful API endpoints. In this landscape, as the fully managed version of Elasticsearch, Elastic Cloud offers its own take on vector search capabilities.
However, it's essential to recognize that vector search plugins don't provide a comprehensive solution to embedding management and vector search. These plugins function more as supplementary components to existing systems, which can limit their overall performance concerning factors such as latency, capacity, and throughput. The endeavor to construct applications for unstructured data atop conventional databases resembles retrofitting lithium batteries and electric motors into a gasoline-powered car — an approach that falls short in innovation.
Within this Elastic Cloud vs. Zilliz Cloud comparison, we will venture beyond this distinction, delving into several other comparison areas. We'll delve into benchmarks to offer a performance perspective and perform an in-depth Elastic Cloud feature analysis.
Benchmarking Elastic Cloud and Zilliz Cloud
At Zilliz, developers and architects alike ask us, "How does Zilliz measure up to Elastic Cloud for vector embedding workloads?" This curiosity might spring from various motivations, all stemming from building semantic similarity search for capabilities like product recommenders or reverse image search, and lately, when building LLM applications with Retrieval Augmented Generation (RAG).
Over the past few weeks, we undertook a comprehensive analysis, delving into the performance and attributes of Zilliz Cloud vs. Elastic Cloud regarding vector embedding workloads. With our open-source benchmark tool, VectorDBBench, our exploration focused intently on critical aspects such as queries per second (QPS), queries per dollar (QP$), and latency.
Datasets tested
We’ve tested two types of datasets to benchmark Zilliz Cloud and Elastic Cloud.
- Dataset 1: 1,000,000 vectors with 768 dimensions
- Dataset 2: 500,000 vectors with 1,536 dimensions
Products tested
We’ve tested Zilliz Cloud and Elastic Cloud with similar hardware resources.
- Zilliz Cloud (1cu-perf): Zilliz Cloud with one performance-optimized compute unit
- Zilliz Cloud (1cu-cap): Zilliz Cloud with one capacity-optimized compute unit
- Elastic Cloud (up to 2.5c8g): Elastic Cloud with up to 2.5vCPU and 8 GB of RAM
Note: For more information about Zilliz Cloud’s compute units, see Zilliz blog introducing Zilliz Cloud CU type.
Queries per second (QPS)
Zilliz outperforms Elastic Cloud by 34x and 22x, respectively, in terms of QPS when processing 1 million vectors with 768 dimensions.
Zilliz outperforms Elastic Cloud by 26x and 13x in terms of QPS when processing 500,000 vectors with 1,563 dimensions.
This is a 1-100 score based on each system's performance in different cases according to a specific rule. A higher score denotes better performance.
According to our testing results, Zilliz outperforms Elastic Cloud cost for comprehensive performance.
Queries per $ (QP$)
Zilliz is 102x and 65x more cost-effective than Elastic Cloud when processing 1 million vectors with 768 dimensions.
Zilliz is 79x and 38x more cost-effective than Elastic Cloud when processing 500,000 vectors with 1,536 dimensions.
This is a 1-100 score based on each system's performance in different cases according to a specific rule. A higher score denotes better performance.
According to our testing results, Zilliz outperforms Elastic Cloud for comprehensive QP$ performance.
Latency
Zilliz outperforms Elastic Cloud by 123x and 96x in terms of P99 latency when processing 1 million vectors with 768 dimensions.
Zilliz outperforms Elastic Cloud by 502x and 108x in terms of P99 latency when processing 500,000 vectors with 1,536 dimensions.
This is a >1 score based on each system's performance in different cases according to a specific rule. A lower score denotes better performance.
According to our testing results, Zilliz Cloud outperforms Elastic Cloud for comprehensive latency performance.
These benchmarks were conducted using the open-source VectorDB Bench tool on GitHub, where you’ll also find the leaderboard. VectorDBBench provides unbiased vector database benchmark results for mainstream vector databases and cloud services. Designed for ease of use, VectorDB Bench makes finding the optimal choice among a plethora of vector database cloud services and open-source vector databases a breeze.
Elastic Cloud Features Comparison
Performance is the biggest challenge with vector databases as the number of unstructured data elements stored in a vector database grows into hundreds of millions or billions, and horizontal scaling across multiple nodes becomes paramount. Furthermore, differences in insert rate, query rate, and underlying hardware may result in different application needs, making overall system tunability a mandatory feature for vector databases.
What’s your vector database for?
A vector database is a fully managed solution for storing, indexing and searching across a massive dataset of unstructured data that leverages the power of embeddings from machine learning models. A vector database should have the following features:
- Scalability and tunability
- Multi-tenancy and data isolation
- A complete suite of APIs
- An intuitive user interface/administrative console
Scalability
Functionality
Purpose-built
For more details, visit our comparison page.