# What metrics would you track to ensure the vector store is performing well under load (for instance, QPS it’s handling, average search time, recall at given latency)?

Source: https://zilliz.com/ai-faq/what-metrics-would-you-track-to-ensure-the-vector-store-is-performing-well-under-load-for-instance-qps-its-handling-average-search-time-recall-at-given-latency
Author: zilliz

To ensure a vector store performs well under load, track metrics across three categories: throughput, latency, and accuracy. These metrics help identify bottlenecks, validate responsiveness, and confirm result quality.

First, measure **throughput and latency**. **Queries per second (QPS)** indicates how many requests the system handles. If QPS plateaus or drops under load, it suggests resource saturation or scalability limits. **Average search latency** (time per query) is critical for user experience—consistent spikes under load may signal inefficient indexing or hardware constraints. Track **p95/p99 latency** to identify outliers affecting a subset of requests. For example, if average latency is 50ms but p99 is 500ms, a small fraction of slow queries might degrade performance. Additionally, monitor **timeouts or errors** (e.g., failed queries due to resource exhaustion) to ensure reliability.

Second, evaluate **accuracy and recall**. Vector stores often trade accuracy for speed (e.g., using approximate nearest neighbors). **Recall@k** (proportion of true top-k results returned) under varying loads ensures quality isn’t compromised. For instance, if recall drops from 95% to 70% when QPS increases, the indexing parameters (like search depth in HNSW graphs) may need adjustment. Pair this with **latency distributions** to find acceptable thresholds (e.g., maintaining 90% recall within 100ms). If recall degrades at higher QPS, consider load balancing or sharding to distribute the workload.

Third, monitor **resource utilization and scalability**. Track CPU, memory, disk I/O, and network usage to identify hardware bottlenecks. For example, high CPU usage during queries suggests computational limits, while memory spikes may indicate inefficient caching. **Indexing time** and **memory footprint** during updates are also critical if the vector store supports real-time data ingestion. For cloud-based systems, **autoscaling metrics** (e.g., instance spin-up time) ensure the system scales horizontally under load. Tools like Prometheus or Grafana can visualize these metrics to correlate QPS spikes with resource trends and guide optimizations.