When using Annoy (Approximate Nearest Neighbors Oh Yeah), the number of trees and the search_k parameter directly influence the trade-off between query accuracy and speed. Here’s how they work and how to choose their values:
Number of Trees Annoy builds a forest of binary trees, where each tree partitions the data using random hyperplanes. More trees increase the likelihood of finding the true nearest neighbors because the search explores more diverse paths through the data. For example, 100 trees will generally provide higher recall (accuracy) than 10 trees, as the algorithm samples more regions of the space. However, more trees also increase index build time and memory usage, and slightly slow down queries due to the overhead of traversing multiple trees. For instance, with a dataset of 1 million embeddings, increasing trees from 50 to 200 might improve recall from 85% to 95% but could double query latency.
Search_k Parameter
The search_k parameter controls how many leaf nodes are examined during a query. Specifically, search_k = n_trees * k, where k is the number of neighbors requested. A higher search_k means more nodes are checked, which improves accuracy but slows down queries. For example, if n_trees=50 and k=10, the default search_k=500 (50*10). If you set search_k=1000, Annoy will search twice as many nodes, potentially finding better neighbors at the cost of longer query times. This is useful when precision is critical, such as in recommendation systems where missing a top result impacts user experience.
Choosing Values
Start with a baseline of n_trees=50-100 for moderate-sized datasets (e.g., 100k-1M items) and adjust based on accuracy needs. If recall is insufficient, incrementally increase n_trees (e.g., to 200-300) and validate using a holdout dataset. For search_k, begin with the default n_trees * k and increase it (e.g., search_k=2*n_trees*k) if results are inconsistent. Always benchmark with real-world queries: if latency is unacceptable, reduce n_trees or cap search_k at a lower value. For example, a search system requiring <50ms latency might use n_trees=50 and search_k=500, while a batch process prioritizing accuracy could use n_trees=200 and search_k=2000. The optimal balance depends on your specific tolerance for speed versus precision.