VectorDBBench - A Vector Database Benchmark Tool
VectorDBBench provides unbiased vector database benchmark results for mainstream vector databases and cloud services, and it's your go-to tool for the ultimate performance and cost-effectiveness of vector database comparison. Designed with ease-of-use in mind, VectorDBBench is devised to help users, even non-professionals, reproduce results or test new systems, making the hunt for the optimal choice amongst a plethora of vector database cloud services and open-source vector databases a breeze.
Prepare to delve into the world of VectorDBBench, and let it guide you in uncovering your perfect vector database match.
- Zilliz Cloud
- Milvus
- PgVector
- Elastic Cloud
- Pinecone
- Qdrant Cloud
- Weaviate Cloud
- Medium(2 datasets: Cohere, 1M of 768-dim vectors; OpenAI, 500K of 1,536-dim vectors)
- Large(2 datasets: Cohere, 10M of 768-dim vectors; OpenAI, 5M of 1,536-dim vectors)
Scalar filter
- None
- Low (>=1%)
- High (>=99%)
Comparison Results
The Performance Ranking table summarizes the performance of mainstream vector databases, showcasing rankings, comprehensive scores, QPS, and recall rates. The Cost Ranking table compares the cost of mainstream vector databases, detailing their rankings, dollars per one million queries, and cost-performance ratio.
- Performance Ranking
- Cost Ranking
Rankings Ranking positions based on each vector database’s comprehensive performance score. | Databases with different hardware resources | Score Comprehensive scoring results demonstrating a vector database’s vector search performance. See our scoring rule for detailed information. | QPS/RecallMedium OpenAI | QPS/RecallMedium OpenAI | QPS/RecallMedium OpenAI | QPS/RecallMedium Cohere | QPS/RecallMedium Cohere | QPS/RecallMedium Cohere |
1 | ZillizCloud-8cu-perf | 100 | 1871 / 0.96 | 1583 / 0.984 | 2345 / 1 | 2884.689 / 0.88 | 1689.58 / 0.949 | 1517.679 / 1 |
2 | Milvus-16c64g-hnsw | 56.0628 | 633.603 / 0.919 | 599.421 / 0.996 | 2098.211 / 1 | 1258.704 / 0.98 | 1075.878 / 0.98 | 1494.849 / 1 |
3 | QdrantCloud-4c16g-5node | 35.0992 | 626.524 / 0.995 | 434.406 / 0.918 | 1509.329 / 1 | 579.942 / 0.921 | 467.179 / 0.97 | 1156.29 / 0.999 |
4 | Pinecone-p2.x1-8node | 21.5045 | 379.972 / 0.982 | 303.8 / 0.948 | 730.7 / 0.959 | 537.498 / 0.89 | 425.253 / 0.969 | 596.794 / 0.969 |
5 | ZillizCloud-2cu-cap | 20.9947 | 322.7 / 0.948 | 303.255 / 0.988 | 584 / 1 | 536.073 / 0.973 | 372.047 / 0.89 | 431.751 / 1 |
6 | Milvus-4c16g-disk | 20.8002 | 321.605 / 0.989 | 287 / 0.987 | 526.885 / 1 | 516.27 / 0.946 | 354.842 / 0.98 | 427.523 / 1 |
7 | ZillizCloud-1cu-perf | 19.8116 | 297.5 / 0.974 | 228.3 / 0.994 | 445.329 / 1 | 365.084 / 0.945 | 325.527 / 0.945 | 397.054 / 1 |
8 | Milvus-2c8g-hnsw | 13.7139 | 228.4 / 0.935 | 181.5 / 0.935 | 412 / 1 | 330.014 / 0.951 | 271.659 / 0.968 | 313.512 / 1 |
9 | QdrantCloud-4c16g-1node | 12.4027 | 188.644 / 0.918 | 179.003 / 0.994 | 394.542 / 1 | 274.541 / 0.981 | 236.567 / 0.981 | 309.483 / 1 |
10 | ZillizCloud-1cu-cap | 10.1329 | 180.276 / 0.994 | 155.699 / 0.917 | 205.7 / 0.959 | 261.798 / 0.926 | 189.44 / 0.889 | 216.523 / 1 |
11 | Pinecone-p2.x1 | 9.7872 | 143 / 0.982 | 106 / 0.989 | 189 / 1 | 240.721 / 0.889 | 166.185 / 0.926 | 138.948 / 0.998 |
12 | Pinecone-p1.x1 | 3.9933 | 67.63 / 0.806 | 63.35 / 0.807 | 176.7 / 1 | 100.667 / 0.991 | 101.14 / 0.991 | 121.717 / 0.969 |
13 | Milvus-2c8g-disk | 3.2446 | 46.862 / 0.996 | 37.07 / 0.998 | 81.192 / 1 | 67.912 / 0.991 | 42.486 / 0.874 | 75.706 / 1 |
14 | Pinecone-s1.x1-2node | 1.129 | 43.502 / 0.996 | 17.327 / 0.996 | 45.067 / 1 | 63.137 / 0.991 | 20.299 / 0.929 | 52.261 / 1 |
15 | Pinecone-s1.x1 | 0.9388 | 37.432 / 0.998 | 16.18 / 0.879 | 41.544 / 1 | 46.619 / 0.874 | 18.362 / 0.874 | 32 / 1 |
16 | ElasticCloud-upTo2.5c8g | 0.9279 | 16.34 / 0.879 | 15.13 / 0.807 | 36.11 / 1 | 20.744 / 0.929 | 15.175 / 0.989 | 30.136 / 1 |
17 | WeaviateCloud-standard | 0.6901 | 15.33 / 0.806 | 1.839 / 0.996 | 26.26 / 1 | 18.763 / 0.874 | 10.851 / 0.89 | 27.618 / 1 |
18 | WeaviateCloud-bus_crit | 0.6712 | 11.295 / 0.996 | 1.567 / 0.996 | 17.41 / 1 | 15.227 / 0.989 | 0.764 / 0.991 | 26.472 / 1 |
19 | PgVector-2c8g | 0.2337 | 0.884 / 0.853 | 0.894 / 0.853 | 1.215 / 0.749 | 10.627 / 0.89 | 0.751 / 0.991 | 25.274 / 0.998 |
Rankings: Ranking positions based on each vector database’s comprehensive performance score.
Scores: Comprehensive scoring results demonstrating a vector database’s vector search performance. See our scoring rule for detailed information.
QPS: A vector database’s capability to handle concurrent queries per second. Higher QPS values indicate better vector database performance.
Recall: Vector search accuracy of a vector database. Higher recall rates correspond to more accurate vector search results.
Unavailable Data
Please make sure you select at least one .