VectorDBBench - A Vector Database Benchmark Tool
VectorDBBench provides unbiased vector database benchmark results for mainstream vector databases and cloud services, and it's your go-to tool for the ultimate performance and cost-effectiveness of vector database comparison. Designed with ease-of-use in mind, VectorDBBench is devised to help users, even non-professionals, reproduce results or test new systems, making the hunt for the optimal choice amongst a plethora of vector database cloud services and open-source vector databases a breeze.
Prepare to delve into the world of VectorDBBench, and let it guide you in uncovering your perfect vector database match.
- Zilliz Cloud
- Milvus
- Elastic Cloud
- PgVector
- Pinecone
- Qdrant Cloud
- Weaviate Cloud
- Medium(2 datasets: Cohere, 1M of 768-dim vectors; OpenAI, 500K of 1,536-dim vectors)
- Large(2 datasets: Cohere, 10M of 768-dim vectors; OpenAI, 5M of 1,536-dim vectors)
Scalar filter
- None
- Low (>=1%)
- High (>=99%)
Comparison Results
The Performance Ranking (QPS) table shows the throughput of mainstream vector databases, showcasing their comprehensive QPS scores and rankings based on the scores, QPS, and recall rates.
The Performance Ranking (P99 Latency) table demonstrates the response speed of mainstream vector databases, showcasing their comprehensive P99 latency scores and rankings based on the scores, P99 latency, and recall rates.
The Cost Ranking table compares the cost of mainstream vector databases, detailing their cost-performance ratios, rankings based on the ratio, and dollars per one million queries.
- Performance Ranking(QPS)
- Performance Ranking(P99 Latency)
- Cost Ranking
Rankings Ranking positions based on each vector database’s comprehensive QPS scores. | Databases with different hardware resources | QPS Scores Comprehensive scoring results demonstrating a vector database’s vector search performance. See our scoring rule for detailed information. | Test time | QPS/RecallMedium OpenAI | QPS/RecallMedium OpenAI | QPS/RecallMedium OpenAI | QPS/RecallMedium Cohere | QPS/RecallMedium Cohere | QPS/RecallMedium Cohere |
1 | ZillizCloud-8cu-perf-v2024.1 | 100 | 2024-01 | 5115.53 / 0.947 | 3685.077 / 0.974 | 4742.162 / 0.994 | 6054.443 / 0.916 | 4104.26 / 0.951 | 4252.127 / 0.996 |
2 | Milvus-16c64g-hnsw-v2.2.12 standalone mode | 23.5105 | 2023-08 | 722.032 / 0.976 | 599.421 / 0.996 | 2098.211 / 1 | 1258.704 / 0.98 | 1075.878 / 0.98 | 1494.849 / 1 |
3 | ZillizCloud-1cu-perf-v2024.1 | 15.7792 | 2024-01 | 633.603 / 0.919 | 467.58 / 0.99 | 1509.329 / 1 | 873.371 / 0.948 | 571.426 / 0.967 | 1156.29 / 0.999 |
4 | QdrantCloud-4c16g-5node | 14.7192 | 2023-08 | 626.524 / 0.995 | 434.406 / 0.918 | 975.25 / 0.994 | 789.123 / 0.94 | 544.62 / 0.977 | 930.916 / 0.997 |
5 | ZillizCloud-2cu-cap-v2024.1 | 10.967 | 2024-01 | 503.228 / 0.968 | 413.323 / 0.981 | 730.7 / 0.959 | 537.498 / 0.89 | 467.179 / 0.97 | 596.794 / 0.969 |
6 | Pinecone-p2.x1-8node | 9.0181 | 2023-08 | 322.7 / 0.948 | 303.8 / 0.948 | 526.885 / 1 | 536.073 / 0.973 | 372.047 / 0.89 | 431.751 / 1 |
7 | Milvus-4c16g-disk-v2.2.12 standalone mode | 8.7228 | 2023-08 | 321.605 / 0.989 | 303.255 / 0.988 | 445.329 / 1 | 392.883 / 0.958 | 343.82 / 0.968 | 411.765 / 0.997 |
8 | ZillizCloud-1cu-cap-v2024.1 | 5.9353 | 2024-01 | 269.546 / 0.978 | 240.036 / 0.982 | 425.549 / 0.994 | 365.084 / 0.945 | 325.527 / 0.945 | 313.512 / 1 |
9 | Milvus-2c8g-hnsw-v2.2.12 standalone mode | 5.7511 | 2023-08 | 228.4 / 0.935 | 181.5 / 0.935 | 394.542 / 1 | 274.541 / 0.981 | 236.567 / 0.981 | 309.483 / 1 |
10 | QdrantCloud-4c16g-1node | 5.2012 | 2023-08 | 188.644 / 0.918 | 179.003 / 0.994 | 218.063 / 0.994 | 261.798 / 0.926 | 189.44 / 0.889 | 216.677 / 0.997 |
11 | Pinecone-p2.x1 | 4.1044 | 2023-08 | 180.276 / 0.994 | 155.699 / 0.917 | 205.7 / 0.959 | 240.721 / 0.889 | 166.185 / 0.926 | 138.948 / 0.998 |
12 | Pinecone-p1.x1 | 1.6747 | 2023-08 | 67.63 / 0.806 | 63.35 / 0.807 | 176.7 / 1 | 100.667 / 0.991 | 101.14 / 0.991 | 121.717 / 0.969 |
13 | Milvus-2c8g-disk-v2.2.12 standalone mode | 1.3607 | 2023-08 | 46.862 / 0.996 | 37.07 / 0.998 | 81.192 / 1 | 67.912 / 0.991 | 42.486 / 0.874 | 75.706 / 1 |
14 | Pinecone-s1.x1-2node | 0.4735 | 2023-08 | 43.502 / 0.996 | 17.327 / 0.996 | 45.067 / 1 | 63.137 / 0.991 | 20.299 / 0.929 | 52.261 / 1 |
15 | Pinecone-s1.x1 | 0.3937 | 2023-08 | 37.432 / 0.998 | 16.18 / 0.879 | 41.544 / 1 | 46.619 / 0.874 | 18.362 / 0.874 | 32 / 1 |
16 | ElasticCloud-upTo2.5c8g | 0.3891 | 2023-08 | 16.34 / 0.879 | 15.13 / 0.807 | 36.11 / 1 | 20.744 / 0.929 | 15.175 / 0.989 | 30.136 / 1 |
17 | WeaviateCloud-standard | 0.2894 | 2023-08 | 15.33 / 0.806 | 1.839 / 0.996 | 26.26 / 1 | 18.763 / 0.874 | 10.851 / 0.89 | 27.618 / 1 |
18 | WeaviateCloud-bus_crit | 0.2815 | 2023-08 | 11.295 / 0.996 | 1.567 / 0.996 | 17.41 / 1 | 15.227 / 0.989 | 0.764 / 0.991 | 26.472 / 1 |
19 | PgVector-2c8g | 0.098 | 2023-08 | 0.884 / 0.853 | 0.894 / 0.853 | 1.215 / 0.749 | 10.627 / 0.89 | 0.751 / 0.991 | 25.274 / 0.998 |
Rankings: Ranking positions based on each vector database’s comprehensive QPS scores
QPS Scores: Comprehensive scoring results demonstrating a vector database’s vector search throughput. See our scoring rule for detailed information.
QPS: A vector database’s capability to handle concurrent queries per second. Higher QPS values indicate better vector database performance.
Recall: Vector search accuracy of a vector database. Higher recall rates correspond to more accurate vector search results.
Unavailable Data
Please make sure you select at least one .