VDBBench Leaderboard
zilliz.com

MethodologyReproduce and Validate the ResultsAdd a SystemReport Mistake

Vector Search Latency and QPS at $1,000 Monthly Cost

Dataset

P99 Latency (ms)

QPS

ZillizCloud-8cu-perf

2.5

9,704.42

Milvus-16c64g-sq8

2.2

3,465.17

OpenSearch-16c128g-force_merge

7.2

3,055.01

QdrantCloud-16c64g

6.4

1,242.43

Pinecone-p2.x8-1node

13.7

1,146.53

OpenSearch-16c128g

13.2

950.63

Streaming Performance

This chart benchmarks search performance on the Cohere-10M dataset under constant ingestion pressure. We measure p99 serial search latency and maximum concurrent QPS at 90% data capacity while the insertion workload remains active. These 'streaming' results are then contrasted with the 'static' performance on a fully indexed database.

Plot values

static
constant ingestion (500 rows/s)
constant ingestion (1000 rows/s)

ZillizCloud

8cu-perf

3,957

2,119

1,860

Pinecone

p2.x8-1node

1,131

367.4

369.7

OpenSearch

16c128g

505.7

161.7

149.7

QdrantCloud

16c64g

446.9

393.8

347.6

Milvus

16c64g-sq8

437.2

306

156

Performance vs. Recall

Dataset

Plot values

QPSP99 Latency

Filtering Performance

Dataset

Plot values

QPSP99 LatencyRecall

Dataset

Plot values

Dataset

Dataset

AI Assistant