Blog
MongoDB vs Aerospike: Selecting the Right Database for GenAI Applications

MongoDB vs Aerospike: Selecting the Right Database for GenAI Applications

Oct 20, 202410 min read

As AI-driven applications evolve, the importance of vector search capabilities in supporting these advancements cannot be overstated. This blog post will discuss two prominent databases with vector search capabilities: MongoDB and Aerospike. Each provides robust capabilities for handling vector search, an essential feature for applications such as recommendation engines, image retrieval, and semantic search. Our goal is to provide developers and engineers with a clear comparison, aiding in the decision of which database best aligns with their specific requirements.

What is a Vector Database?

Before we compare MongoDB vs Aerospike, let's first explore the concept of vector databases.

A vector database is specifically designed to store and query high-dimensional vectors, which are numerical representations of unstructured data. These vectors encode complex information, such as the semantic meaning of text, the visual features of images, or product attributes. By enabling efficient similarity searches, vector databases play a pivotal role in AI applications, allowing for more advanced data analysis and retrieval.

Common use cases for vector databases include e-commerce product recommendations, content discovery platforms, anomaly detection in cybersecurity, medical image analysis, and natural language processing (NLP) tasks. They also play a crucial role in Retrieval Augmented Generation (RAG), a technique that enhances the performance of large language models (LLMs) by providing external knowledge to reduce issues like AI hallucinations.

There are many types of vector databases available in the market, including:

Purpose-built vector databases such as Milvus, Zilliz Cloud (fully managed Milvus)
Vector search libraries such as Faiss and Annoy.
Lightweight vector databases such as Chroma and Milvus Lite.
Traditional databases with vector search add-ons capable of performing small-scale vector searches.

MongoDB is a NoSQL database that stores data in JSON-like documents ****and Aerospike is a distributed, scalable NoSQL database. Both have vector search capabilities as an add-on. This post compares their vector search capabilities.

MongoDB: The Basics

MongoDB Atlas Vector Search is a feature that allows you to do vector similarity searches on data stored in MongoDB Atlas. You can index and query high-dimensional vector embeddings along with your document data and do AI and machine learning right in the database.

At its core, Atlas Vector Search uses the Hierarchical Navigable Small World (HNSW) algorithm for indexing and searching vector data. This creates a multi-level graph of the vector space so you can do Approximate Nearest Neighbor (ANN) searches. It’s a balance of speed and accuracy for large scale vector search. Atlas Vector Search also supports Exact Nearest Neighbors (ENN) searches which prioritizes accuracy over performance for queries of up to 10,000 documents.

One of the big advantages of Atlas Vector Search is its integration with MongoDB’s flexible document model. You can store vector embeddings along with other document data so you can search more contextually and precisely. You can query any kind of data that can be embedded up to 4096 dimensions. Atlas Vector Search allows you to combine vector similarity searches with traditional document filtering. For example, a semantic search for products could be filtered by category, price range or availability.

Atlas Vector Search also supports hybrid search, combining vector search with full text search for more granular results. This is different from Atlas Search which is focused on keyword based search. The platform integrates with popular AI services and tools so you can use it with embedding models from providers like OpenAI, VoyageAI and many others listed on Hugging Face. It also supports open-source frameworks like LangChain and LlamaIndex for building applications that use Large Language Models (LLMs).

To ensure scalability and performance, MongoDB Atlas provides Search Nodes, which provides dedicated infrastructure for Atlas Search and Vector Search workloads. This allows you to have optimized compute resources and independent scaling of search needs so you get better performance at scale.

By having these capabilities in the MongoDB ecosystem, Atlas Vector Search is a full solution for developers building AI powered applications, recommendation systems or advanced search features. No need for a separate vector database, you can use MongoDB’s scalability and rich features along with vector search.

Aerospike: The Basics

Aerospike is a NoSQL database for high-performance real-time applications. It has added support for vector indexing and searching so it’s suitable for vector database use cases. The vector capability is called Aerospike Vector Search (AVS) and is in Preview. You can request early access from Aerospike.

AVS only supports Hierarchical Navigable Small World (HNSW) indexes for vector search. When updates or inserts are made in AVS, record data including the vector is written to the Aerospike Database (ASDB) and is immediately visible. For indexing, each record must have at least one vector in the specified vector field of an index. You can have multiple vectors and indexes for a single record, so you can search for the same data in different ways. Aerospike recommends assigning upserted records to a specific set so you can monitor and operate on them.

AVS has a unique way of building the index, it’s concurrent across all AVS nodes. While vector record updates are written directly to ASDB, index records are processed asynchronously from an indexing queue. This is done in batches and distributed across all AVS nodes, so it uses all the CPU cores in the AVS cluster and is scalable. Ingestion performance is highly dependent on host memory and storage layer configuration.

For each item in the indexing queue, AVS processes the vector for indexing, builds the clusters for each vector and commits those to ASDB. An index record contains a copy of the vector itself and the clusters for that vector at a given layer of the HNSW graph. Indexing uses vector extensions (AVX) for single instruction, multiple data parallel processing.

AVS queries during ingestion to “pre-hydrate” the index cache because records in the clusters are interconnected. These queries are not counted as query requests but show up as reads against the storage layer. This way, the cache is populated with relevant data and can improve query performance. This shows how AVS handles vector data and builds indexes for similarity search so it can scale for high-dimensional vector searches.

Key Differences

When it comes to vector search, both MongoDB and Aerospike have good options. As a developer looking to add vector search to your application, understanding the differences between the two will help you make a decision. Let’s compare MongoDB Atlas Vector Search and Aerospike Vector Search (AVS) on a few key points.

Search Methodology

Both MongoDB Atlas Vector Search and Aerospike Vector Search use the Hierarchical Navigable Small World (HNSW) algorithm for indexing and searching vector data. This algorithm creates a multi-level graph of the vector space so you can do Approximate Nearest Neighbor (ANN) searches. MongoDB Atlas also supports Exact Nearest Neighbors (ENN) searches for queries up to 10,000 documents. This gives MongoDB users more flexibility to balance speed and precision based on their use case.

Data

MongoDB’s flexible document model lets you store vector embeddings along with other document data. This allows for more contextual and precise searches as you can combine vector similarity searches with traditional document filtering. For example, you can do a semantic search for products and filter results by category, price range or availability.

Aerospike is primarily a key-value store and has added vector indexing and searching capabilities. It allows multiple vectors and indexes for a single record, so you have flexibility in how you can search your data. But its data model may not be as flexible as MongoDB’s document-based approach for handling semi-structured or unstructured data.

Scalability and Performance

MongoDB Atlas has Search Nodes which provide dedicated infrastructure for Atlas Search and Vector Search workloads. This allows for optimized compute resources and independent scaling of search needs so you get better performance at scale.

Aerospike’s approach to building the index is different. It processes index records from an indexing queue asynchronously across all AVS nodes. This uses all CPU cores in the AVS cluster, so it could be more scalable for index construction. Aerospike also uses a “pre-hydration” technique during ingestion to populate the index cache with relevant data which can improve query performance.

Flexibility and Customization

MongoDB Atlas Vector Search supports multiple distance metrics for similarity calculations and can work with embeddings from any provider up to 4096 dimensions. It also supports hybrid search, combining vector search with full-text search for more precise results.

Aerospike Vector Search, while limited to the current preview, allows multiple vectors and indexes per record. This can be useful for searching the same data in different ways.

Integration and Ecosystem

MongoDB Atlas integrates with popular AI services and tools, works with embedding models from OpenAI, VoyageAI and many others listed on Hugging Face. It also supports integration with LangChain and LlamaIndex for building AI-powered applications.

Aerospike also integrates with popular frameworks like LangChain.

Ease of Use

MongoDB is known for being developer friendly with lots of documentation and a large community. Atlas as a managed service can simplify setup and maintenance.

Aerospike has a steeper learning curve if you’re not familiar with its architecture. Its vector search is in preview so may have less documentation and community support compared to more established options.

Cost

MongoDB Atlas has a tiered pricing model, costs vary based on your usage and features. You need to see if the vector search fits in your budget.

Aerospike’s pricing is not publicly available, you need to contact them for more information. Consider both software costs and infrastructure costs to run Aerospike.

When to Use Each

MongoDB Atlas Vector Search is great for applications that need a flexible data model and need to integrate vector search with regular document querying. It’s perfect for projects that involve complex, semi-structured data and need to do hybrid searches that combine vector similarity with full text search. MongoDB is great when you need to store and query different data types alongside vector embeddings, like content recommendation systems, semantic search engines or AI powered analytics platforms that use large language models. It’s also great for developers building AI applications that need to work on both structured and unstructured data and integrate with popular AI services and tools.

Aerospike Vector Search is great for high performance, real-time applications that are mostly key-value but need vector search. It’s perfect for use cases that require extremely low latency and high throughput, like real-time bidding systems, fraud detection engines or personalized content delivery networks. Aerospike’s approach to index building and cache pre-hydration can give significant performance benefits in scenarios where data ingestion speed and query speed are critical. While its data model may not be as flexible as MongoDB’s, Aerospike is the better choice for applications that prioritize raw performance and scale over complex data modeling requirements.

Summary

MongoDB Atlas Vector Search is a full solution that combines the flexibility of a document database with vector search capabilities, great for complex AI driven applications that need to handle diverse data. Its strength is in vector search integration with regular queries, hybrid searches and a rich AI tool ecosystem. Aerospike Vector Search is more specialized and excels in high performance, real-time scenarios where low latency and high throughput are key. Its indexing approach and focus on performance makes it a good fit for specific use cases that need speed and scale. Ultimately the choice between MongoDB and Aerospike for vector search should be driven by your application requirements, data complexity, performance needs and scalability demands. Consider your existing infrastructure, data nature, query patterns, level of integration with AI tools and services when making your decision.

Read this to get an overview of MongoDB and Aerospike but to evaluate these you need to evaluate based on your use case. One tool that can help with that is VectorDBBench, an open-source benchmarking tool for vector database comparison. In the end, thorough benchmarking with your own datasets and query patterns will be key to making a decision between these two powerful but different approaches to vector search in distributed database systems.

Using Open-source VectorDBBench to Evaluate and Compare Vector Databases on Your Own

VectorDBBench is an open-source benchmarking tool for users who need high-performance data storage and retrieval systems, especially vector databases. This tool allows users to test and compare different vector database systems like Milvus and Zilliz Cloud (the managed Milvus) using their own datasets and find the one that fits their use cases. With VectorDBBench, users can make decisions based on actual vector database performance rather than marketing claims or hearsay.

VectorDBBench is written in Python and licensed under the MIT open-source license, meaning anyone can freely use, modify, and distribute it. The tool is actively maintained by a community of developers committed to improving its features and performance.

Download VectorDBBench from its GitHub repository to reproduce our benchmark results or obtain performance results on your own datasets.
Take a quick look at the performance of mainstream vector databases on the VectorDBBench Leaderboard.
Read the following blogs to learn more about vector database evaluation.

Further Resources about VectorDB, GenAI, and ML

Chloe Williams
Chloe Williams is a technical writer at Zilliz.

Content

Start Free, Scale Easily

Try the fully-managed vector database built for your GenAI applications.

Try Zilliz Cloud for Free