Scikit-learn provides flexibility in distance metrics for algorithms like KNeighborsClassifier and KMeans, but with limitations. For example, KNeighbors allows specifying metric (e.g., Euclidean, Manhattan, Cosine) via parameters, and it supports custom metrics. However, KMeans in scikit-learn is optimized for Euclidean distance and does not natively support Cosine similarity. While you can mimic spherical K-Means (Cosine-based) by normalizing data before clustering, the algorithm itself isn’t designed for non-Euclidean metrics. Additionally, some metrics (e.g., Mahalanobis) require precomputing parameters, adding complexity. Performance can also vary: tree-based algorithms like BallTree or KDTree in KNeighbors work efficiently with Euclidean but may degrade with high-dimensional Cosine comparisons.
FAISS (Facebook AI Similarity Search) focuses on L2 (Euclidean) and inner product metrics. While it doesn’t directly support Cosine similarity, you can achieve it by normalizing vectors to unit length and using the inner product metric. This workaround is efficient but requires explicit preprocessing. FAISS is heavily optimized for GPU/CPU acceleration with L2 and inner product, making these choices faster than alternatives. Custom metrics are not supported, limiting flexibility to these two options unless you implement a wrapper. For example, metrics like Manhattan or Jaccard would require significant custom code, reducing FAISS’s utility in those cases.
Annoy (Approximate Nearest Neighbors Oh Yeah) supports Euclidean, Manhattan, Cosine, Hamming, and Dot Product distances. It uses tree-based structures optimized for Euclidean and Cosine, but performance for other metrics (e.g., Manhattan) may be slower due to less tailored splitting heuristics. Annoy’s strength lies in its simplicity and support for multiple metrics without requiring GPU acceleration. However, it lacks native support for custom metrics, unlike scikit-learn. Elasticsearch’s vector search supports Euclidean, Cosine, and Dot Product, but choice depends on the index type (e.g., dense_vector). Pre-7.10 versions had limited metric support, but newer versions and the knn query API expand options. Limitations include higher memory usage for non-Euclidean metrics and lack of support for domain-specific metrics like Wasserstein distance. Tools like TensorFlow/PyTorch enable metric flexibility via custom implementations but lack built-in abstractions for distance choice in higher-level APIs, requiring manual coding.