Why Milvus?
It is simple to start building a prototype for similarity search, but it is hard to put into production. Embeddings, being highly complex, multi-coordinate numbers, are not well suited to legacy, tabular data structures. Some developers use vector libraries to store vector embeddings in in-memory indexes to perform similarity searches.
However, updating the index is non-trivial and requires vector workflow tools inherent in vector databases. And to top it off, building a solution to host vectors becomes costly quickly when solving the scale requirement.
How does Milvus Work?
Milvus consists of a storage layer and a compute layer, and to enhance elasticity and flexibility, all components in Milvus are stateless. The system comprises of four levels:
Access layer
The access layer comprises a group of stateless proxies and serves as the front layer of the system and endpoint to users.
Coordinator service
The coordinator service assigns tasks to the worker nodes and functions as the system's brain.
Worker nodes
The worker nodes function as arms and legs and are dumb executors that follow instructions from the coordinator service and execute user-triggered DML/DDL commands.
Storage
Storage is the bone of the system and is responsible for data persistence. It comprises meta storage, log broker, and object storage.
What is Milvus Used For?
Today, hundreds of Milvus use cases are commonly used to build similarity search-based applications. Any company that relies on or works with unstructured data can find numerous benefits.
Semantic text search
Processing and querying text across multiple vectors like intent, location, and previous search history can provide the context necessary for more accurate and nuanced results.
Targeted advertising
Vector databases can be used in targeted advertising to improve the relevance and effectiveness of ad targeting. In this context, the database can store and index large amounts of data related to user behavior, demographics, and interests as high-dimensional vectors. Ads are then mapped to the same space as the users, making targeted advertising as simple as performing a query in Milvus.
E-commerce
Vector databases such as Milvus can power product recommendation engines by combining multiple sources of unstructured data such as search history and past purchases.
UGC recommendation
User-generated content comes in various formats, ranging from simple text (blog posts, news articles) to short- and long-form videos. Each piece of content is a single vector representation in a vector database. This vector representation makes recommending new content as easy as querying over content users have liked or engaged with previously.
Risk-control and anti-fraud
Anti-fraud systems can also use vector representations to encode similarities between actions and other data points. For example, an anti-fraud system can compare the vectors representing different transactions or behaviors to identify similarities that may indicate a higher risk of fraud or other illicit activities.
New drug discovery
In drug discovery, vector representations of compounds include the overall structure and biological properties. A vector database can store and index this data as high-dimensional vectors, enabling new drug discovery simply by querying.
