Dataset Note:

For "1M Dataset" below, we refer to the "Cohere Medium" () Dataset (1M * 768 float dense vectors).
For "10M" Dataset below, we refer to the "Cohere Large" () Dataset (10M * 768 float dense vectors).

Vector Search Latency and QPS at $1,000 Monthly Cost

Dataset

P99 Latency (ms)

QPS

Recall

ZillizCloud-8cu-perf

13,316.23

0.9383

Milvus-16c64g-sq4u-fp16-force_merge

2.3

9,575.69

0.9189

Milvus-16c64g-sq8-force_merge

2.4

5,973

0.9192

OpenSearch-16c128g-force_merge

7.2

3,055.01

0.9066

ElasticCloud-8c60g-force_merge

17.1

2,353.89

0.9143

QdrantCloud-16c64g

6.4

1,242.43

0.9474

Pinecone-p2.x8-1node

13.7

1,146.53

0.9262

OpenSearch-16c128g

13.2

950.63

0.9140

Streaming Performance

This chart benchmarks search performance on the Cohere-10M dataset under constant ingestion pressure. We measure p99 serial search latency and maximum concurrent QPS at 90% data capacity while the insertion workload remains active. These 'streaming' results are then contrasted with the 'static' performance on a fully indexed database.

Plot values

static
constant ingestion (500 rows/s)
constant ingestion (1000 rows/s)

ZillizCloud

8cu-perf

7,385

Recall: 0.9384

2,119

1,860

Milvus

16c64g-sq8-force_merge

2,747

Recall: 0.9204

306

156

ElasticCloud

8c60g-force_merge

1,452

Recall: 0.9042

61.67

61.82

Pinecone

p2.x8-1node

1,131

Recall: 0.9024

367.4

369.7

TurboPuffer

2026-03-31

649.9

Recall: 0.8352

536

442.6

OpenSearch

16c128g

505.7

Recall: 0.9068

161.7

149.7

QdrantCloud

16c64g

446.9

Recall: 0.9357

393.8

347.6

Performance vs. Recall

Dataset

Plot values

QPSP99 Latency

Filtering Performance

Dataset

Plot values

QPSP99 LatencyRecall