We are proud to announce Milvus v1.0, a stable, long-term support version of the world’s most popular open-source vector database. Discover what Milvus v1.0 has to offer, and plans for the platform’s future.

Zilliz is proud to announce the release of Milvus v1.0. After months of extensive testing Milvus v1.0, which is based on a stable version of Milvus v0.10.6, is available for use.

Milvus v1.0 offers the following key features:

- Support for mainstream similarity metrics, including Euclidean distance, inner product, Hamming distance, Jaccard coefficient, and more.
- Integration with, and improvements to, SOTA ANNs algorithms, including Faiss, Hnswlib, Annoy, NSG, and more. 
- Scale-out capability through the Mishards sharding proxy.
- Support for processors commonly used in AI scenarios, including X86, Nvidia GPU, Xilinx FPGA, and more.

See the [Release Notes](https://www.milvus.io/docs/v1.0.0/release_notes.md) for additional Milvus v1.0 features. 

Milvus is an ongoing open-source software (OSS) project. Its first major release has the following implications for users:

- Milvus v1.0 will receive long-term support (3+ years).
- The most stable Milvus release to date is well structured and ready for integration with existing AI ecosystems.

### The first version of Milvus with long-term support

Thanks in part to sponsorship from Zilliz, the Milvus community will provide bug fix support for Milvus v1.0 until December 31st, 2024. New features will be available only in releases following v1.0.

See [The Milvus release guideline](https://milvus.io/docs/v1.0.0/milvus_release_guideline.md) for information about release cadences and more.

### Toolchain enhancements and seamless AI ecosystem integration

Beginning with v1.0, Milvus' toolchain will be a primary development focus. We plan to create the necessary tooling and utilities to meet the needs of the Milvus user community. 

Stability makes integrating Milvus with AI ecosystems a breeze. We are seeking further collaboration between the Milvus community and other AI-focused OSS communities. We encourage contributions to the new AI ASICs (application-specific integrated circuits) in Milvus. 

### The future of Milvus

We believe Milvus has a bright future thanks to the following factors:

- Regular contributions from developers in the Milvus community.
- Support for integration with any cloud-native environment.

We have drafted [community charters](https://milvus.io/docs/v1.0.0/milvus_community_charters.md) to help guide, nurture, and advance the Milvus community as our technology and user base grows. The charters include several technical decisions made to attract more participants to the community.

- Golang will now be used to develop the Milvus engine however, the ANNS algorithm component will still be developed in C++.
- The forthcoming distributed version of Milvus will use existing cloud components as much as possible.

We are thrilled to partner with the open-source software community to build the next-generation cloud data fabric made for AI. Let's get to work!

### Don’t be a stranger

- Find or contribute to Milvus on [GitHub](https://github.com/milvus-io/milvus/)
- Interact with the community via [Slack](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ).
- Connect with us on [Twitter](https://twitter.com/milvusio).

large_Milvus.jpeg

medium_Milvus.jpeg

small_Milvus.jpeg

thumbnail_Milvus.jpeg

Milvus.jpeg

Milvus v1.0 is available now. This stable, long-term support version powers image/video search, chatbots, and many more AI applications.

Milvus 1.0: The World's Most Popular Open-Source Vector Database Just Got Better

Artificial intelligence makes it possible to extract deep insights from massive unstructured datasets. By converting abstract data like video and audio into vectors, a conceptual task becomes a rigorous mathematical one well suited for machines. Milvus is a vector data management platform that is the ideal backbone for your next AI project.

Searching data using easily defined criteria, for example querying a movie database by actor, director, genre, or release date, is simple. A relational database is well equipped for these types of basic searches using a query language such as SQL. But when searches involve complex objects and more abstract queries, such as searching a video streaming library using natural language or a video clip, simple similarity metrics like matching words in a title or description are no longer sufficient.

[Artificial intelligence (AI)](https://medium.com/unstructured-data-service/the-easiest-way-to-search-among-1-billion-image-vectors-d6faf72e361f#d62b) has made computers significantly more capable of understanding the semantics of language, as well as helping people make sense of massive, difficult to analyze unstructured datasets (e.g., audio, video, documents, and social media data). AI makes it possible for Netflix to create sophisticated content recommendation engines, Google users to search the web by image, and pharmaceutical companies to discover new drugs.

### The challenge of searching large unstructured datasets

These feats of technology are accomplished by using AI algorithms to convert dense unstructured data into vectors, a numerical data format that is easily read by machines. Next, additional algorithms are used to calculate the similarity between vectors for a given search. The large size of unstructured datasets makes searching them in their entirety far too time consuming for most machine learning applications. To overcome this, approximate nearest neighbor (ANN) algorithms are used to cluster similar vectors together, then only search the portion of the dataset most likely to contain similar vectors to the target search vector.

This results in [drastically faster](https://medium.com/unstructured-data-service/how-to-choose-an-index-in-milvus-4f3d15259212#7a9a) (though slightly less accurate) similarity search, and is key to building useful AI tools. Thanks to vast public resources, it has never been easier or cheaper to build machine learning applications. However, AI-powered vector similarity search often requires interlacing different tools that vary in number and complexity depending on specific project requirements. Milvus is an open-source AI search engine that aims to simplify the process of building machine learning applications by providing robust functionality under a unified platform.

### What is Milvus?

[Milvus](https://milvus.io/) is an open-source data management platform built specifically for massive-scale vector data and streamlining machine learning operations (MLOps). Powered by Facebook AI Similarity Search (Faiss), Non-Metric Space Library (NMSLIB), and Annoy, Milvus brings a variety of powerful tools together in one place while extending their standalone functionality. The system was purpose built for storing, processing, and analyzing large vector datasets, and can be used to build AI applications spanning computer vision, recommendation engines, and much more.

![Blog_Milvus Is an Open-Source Scalable AI Search Engine_1.jpg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Blog_Milvus_Is_an_Open_Source_Scalable_AI_Search_Engine_1_997255eb27.jpg)
###### *The structure of an AI-powered similarity search engine.*

### Milvus was made to power vector similarity search

Milvus was designed for flexibility, allowing developers to optimize the platform for their specific use case. Support for CPU/GPU-only and heterogeneous computing makes it possible to accelerate data processing and optimize resource requirements for any scenario. Data is stored in Milvus on a distributed architecture, making it trivial to scale data volumes. With support for various AI models, programming languages (e.g., C++, Java, and Python), and processor types (e.g., x86, ARM, GPU, TPU, and FPGA), Milvus offers high compatibility with a wide variety of hardware and software.

For more information about Milvus, check out the following resources:

- Explore Milvus' [technical documentation](https://milvus.io/) and learn more about the platform’s inner workings. 
- Learn how to launch Milvus, build applications, and more using [Milvus tutorials](https://tutorials.milvus.io/).
- Make contributions to the project and engage with Milvus' open-source community on [GitHub](https://github.com/milvus-io).





large_milvus.jpg

medium_milvus.jpg

small_milvus.jpg

thumbnail_milvus.jpg

milvus.jpg

Build powerful machine learning applications and manage massive-scale vector data with Milvus.

Milvus Is an Open-Source Scalable Vector Database

Technological progress means machine-scale analytics and artificial intelligence are more accessible than ever before. Discover how Milvus, an open-source vector data management platform, unlocks the potential of unstructured data. With just one server and 10 lines of code, it is possible to build a real-time search engine capable of processing 1+ billion images. 

Rising compute power and declining compute costs have made machine-scale analytics and artificial intelligence (AI) more accessible than ever before. In practice, this means with just one server and 10 lines of code, it is possible to build a reverse image search engine capable of querying 1+ billion images in real time. This article explains how [Milvus](https://milvus.io/), an open-source vector data management platform, can be used to create powerful systems for unstructured data processing and analysis, as well as the underlying technology that makes this all possible.

**Jump to:**

- [How does AI enable unstructured data analytics?](#how-does-ai-enable-unstructured-data-analytics)
- [What are vector data management platforms?](#what-are-vector-data-management-platforms)
- [What are limitations of existing approaches to vector data management?](#what-are-limitations-of-existing-approaches-to-vector-data-management)
- [What are applications for vector data management platforms and vector similarity search?](#what-are-applications-for-vector-data-management-platforms-and-vector-similarity-search)


### How does AI enable unstructured data analytics?

An oft-cited statistic is that 80% of the world’s data is unstructured, but just 1% ever gets analyzed. Unstructured data, including images, video, audio, and natural language, doesn’t follow a predefined model or manner of organization. This makes processing and analyzing large unstructured datasets difficult. As the proliferation of smartphones and other connected devices pushes unstructured data production to new heights, businesses are increasingly aware of how important insights derived from this nebulous information can be.

For decades, computer scientists have developed indexing algorithms tailored for organizing, searching, and analyzing specific data types. For structured data, there is bitmap, hash tables, and B-tree, which are commonly used in relational databases developed by tech giants such as Oracle and IBM. For semi-structured data, inverted indexing algorithms are standard, and can be found in popular search engines like [Solr](http://www.solrtutorial.com/basic-solr-concepts.html) and [ElasticSearch](https://www.elastic.co/blog/found-elasticsearch-from-the-bottom-up#inverted-indexes-and-index-terms). However, unstructured data indexing algorithms rely on compute-intensive artificial intelligence that has only become widely accessible in the past decade.

### Neural networks convert unstructured data into computer-friendly feature vectors

Using neural networks (e.g. [CNN](https://en.wikipedia.org/wiki/Convolutional_neural_network), [RNN](https://en.wikipedia.org/wiki/Recurrent_neural_network), and [BERT](https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270)) unstructured data can be converted into feature vectors (a.k.a., embeddings), which are a string of integers or floats. This numerical data format is far more readily processed and analyzed by machines. Applications spanning reverse image search, video search, natural language processing (NLP) and more can be built by embedding unstructured data into feature vectors, then calculating similarity between vectors using measures like Euclidean distance or cosine similarity.

![Blog_Thanks to AI, Anyone Can Build a Search Engine for 1+ Billion Images_2.jpeg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Blog_Thanks_to_AI_Anyone_Can_Build_a_Search_Engine_for_1_Billion_Images_2_db8c16aea4.jpeg)

###### *AI algorithms convert unstructured data to vectors*

Computing vector similarity is a relatively simple process that relies on established algorithms. However, unstructured datasets, even after being converted into feature vectors, are typically several orders of magnitude larger than traditional structured and semi-structured datasets. Vector similarity search is complicated by the sheer storage space and compute power required to efficiently and accurately query massive-scale unstructured data.
However, if some degree of accuracy can be sacrificed, there are various approximate nearest neighbor (ANN) search algorithms that can drastically improve query efficiency for massive datasets with high dimensionality. These ANN algorithms decrease storage requirements and computation load by clustering similar vectors together, resulting in faster vector search. Commonly used algorithms include tree-based, graph-based, and combined ANNs.

### What are vector data management platforms?

Vector data management platforms are purpose-built applications for storing, processing, and analyzing massive vector datasets. These tools are designed to easily interface with large amounts of data, and include functionality that streamlines vector data management. Unfortunately, few systems exist that are both flexible and powerful enough to solve modern big data challenges. Milvus, a vector data management platform initiated by [Zilliz](https://zilliz.com/) and released under an open-source license in 2019, attempts to fill this void.

### What are limitations of existing approaches to vector data management?

A common way to build an unstructured data analytics system is to pair algorithms like ANN with open-source implementation libraries such as Facebook AI Similarity Search (Faiss). Due to several limitations, these algorithm-library combinations are not equivalent to a full-fledged vector data management system like Milvus. Existing technology used for managing vector data faces the following problems:

1. **Flexibility:** By default, existing systems typically store all data in main memory, meaning they cannot be run across multiple machines and are poorly suited for handling massive datasets.
2. **Dynamic data handling:** Data is often assumed to be static once fed into existing systems, complicating processing for dynamic data and making near real-time search impossible.
3. **Advanced query processing:** Most tools do not support advanced query processing (e.g., attribute filtering and multi-vector queries), which is essential for building useful similarity search engines.
4. **Heterogeneous computing optimizations:** Few platforms offer optimizations for heterogenous system architectures on both CPUs and GPUs (excluding Faiss), leading to efficiency losses.

Milvus attempts to overcome all of these limitations. The system enhances flexibility by offering support for a variety of application interfaces (including SDKs in Python, Java, Go, C++ and RESTful APIs), multiple vector index types (e.g., quantization-based indexes and graph-based indexes), and advanced query processing. Milvus handles dynamic vector data using a log-structured merge-tree (LSM tree), keeping data insertions and deletions efficient and searches humming along in real time. Milvus also provides optimizations for heterogeneous computing architectures on modern CPUs and GPUs, allowing developers to adjust systems for specific scenarios, datasets, and application environments.

![Blog_Thanks to AI, Anyone Can Build a Search Engine for 1+ Billion Images_3.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Blog_Thanks_to_AI_Anyone_Can_Build_a_Search_Engine_for_1_Billion_Images_3_380e31d32c.png)

###### *An overview of Milvus’ architecture.*

Using various ANN indexing techniques, Milvus is able to achieve a 99% top-5 recall rate. The system is also capable of loading 1+ million data entries per minute. This results in query time of less than one second when running a reverse image search on 1 billion images. As a cloud native application that can operate as a distributed system deployed across multiple nodes, Milvus can easily and reliably achieve similar performance on datasets that contain 10, or even 100, billion images. Additionally, the system is not limited to image data, with applications spanning computer vision, conversational AI, recommendation systems, new drug discovery, and more.

### What are applications for vector data management platforms and vector similarity search?

As outlined above, a capable vector data management platform like Milvus paired with approximate nearest neighbor algorithms enables similarity search on gigantic volumes of unstructured data. This technology can be used to develop applications that span a diverse array of fields. Below we briefly explain several common use cases for vector data management tools and vector similarity search.

### Reverse image search

Major search engines like Google already give users the option to search by image. Additionally, e-commerce platforms have realized the benefits this functionality offers online shoppers, with Amazon incorporating image search into its smartphone applications.

![Blog_Thanks to AI, Anyone Can Build a Search Engine for 1+ Billion Images_4.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Blog_Thanks_to_AI_Anyone_Can_Build_a_Search_Engine_for_1_Billion_Images_4_7884aabcd8.png)

###### *Google’s “search by image” feature.*

Open-source software like Milvus makes it possible for any business to create their own reverse image search system, lowering the barriers to entry for this increasingly in-demand feature. Developers can use pre-trained AI models to convert their own image datasets into vectors, and then leverage Milvus to enable searching for similar products by image.


#### Video recommendation systems

Major online video platforms like YouTube, which receives [500 hours of user generated content each minute](https://www.tubefilter.com/2019/05/07/number-hours-video-uploaded-to-youtube-per-minute/), present unique demands when it comes to content recommendation. In order to make relevant, real-time recommendations that take into consideration new uploads, video recommendation systems must offer lightning-fast query time and efficient dynamic data handling. By converting key frames into vectors and then feeding the results into Milvus, billions of videos can be searched and recommended in near real time.

#### Natural language processing (NLP)

Natural language processing is a branch of artificial intelligence that aims to build systems that can interpret human languages. After converting text data into vectors, Milvus can be used to quickly identify and remove duplicate text, power semantic search, or even [build an intelligent writing assistant](https://medium.com/unstructured-data-service/how-artificial-intelligence-empowered-professional-writing-f433c7e5b561%22%20/). An effective vector data management platform helps maximize the utility of any NLP system.

### Learn more about Milvus

If you would like to learn more about Milvus visit our [website](https://milvus.io/). Additionally, our [bootcamp](https://github.com/milvus-io/bootcamp) offers several tutorials, with instructions for setting up Milvus, benchmark testing, and building a variety of different applications. If you’re interested in vector data management, artificial intelligence, and big data challenges, please join our open-source community on [GitHub](https://github.com/milvus-io) and chat with us on [Slack](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ).

Want more information about building an image search system? Check out this case study:
- [The Journey to Optimizing Billion-scale Image Search (1/2)](https://dzone.com/articles/the-journey-to-optimize-billion-scale-image-search)
- [The Journey to Optimizing Billion-scale Image Search (2/2)](https://medium.com/unstructured-data-service/the-journey-to-optimizing-billion-scale-image-search-2-2-572a36d5d0d)


large_build search.jpg

medium_build search.jpg

small_build search.jpg

thumbnail_build search.jpg

build search.jpg

AI and open-source software make it possible to build a reverse image search engine with just one server and 10 lines of code. Search 1+ billion images in real time and more with Milvus, an open-source vector data management platform. 

Thanks to Milvus, Anyone Can Build a Vector Database for 1+ Billion Images 

Making search results more relevant and boosting online product sales on Tokopedia's e-commerce platform.

Tokopedia is a technology company specializing in e-commerce. Its marketplace is the most popular online shopping platform in Indonesia, with 100+ million monthly active users and 9.7+ million merchants. To make search results more relevant and improve user engagement, Tokopedia built a smart search engine capable of pairing low fill rate keywords with high fill rate keywords.

Milvus, an open-source vector database built by Zilliz, is used to index the underlying product database and optimize Tokopedia's AI-enabled shopping experience.

small_tokopedia.jpeg

thumbnail_tokopedia.jpeg

tokopedia.jpeg

large_beike-blog-cover.jpg

medium_beike-blog-cover.jpg

small_beike-blog-cover.jpg

thumbnail_beike-blog-cover.jpg

beike-blog-cover.jpg

Tokopedia’s AI-enabled E-commerce Site Search

Matching what people see in everyday life with artworks from the Cleveland Museum of Art's extensive collection.

Determined to adapt amid the limitations placed on in-person experiences by the coronavirus pandemic, the Cleveland Museum of art sought new, creative, and easy ways for people to engage with art. In this vein, CMA developed a reverse image search tool called "Artlens AI: Share Your View" that is capable of recognizing shapes, patterns, objects, and even architecture in user uploaded photos. After a user submits a photo, ArtLens AI returns the closest match from the CMA's vast art collection. The museum's hope is that by placing artwork next to contemporary views people will be given new perspectives and opportunities for critical thinking.

Milvus, an open-source vector database built by Zilliz, was used to power Artlens AI's core vector management module.

thumbnail_CMA logo.png

CMA logo.png

large_Blog_ArtLens AI_ 1.jpeg

medium_Blog_ArtLens AI_ 1.jpeg

small_Blog_ArtLens AI_ 1.jpeg

thumbnail_Blog_ArtLens AI_ 1.jpeg

Blog_ArtLens AI_ 1.jpeg

The Cleveland Museum of Art Uses AI to Connect Daily Life with Art History

large_artlens AI interface.png

medium_artlens AI interface.png

small_artlens AI interface.png

thumbnail_artlens AI interface.png

artlens AI interface.png

Recommending engaging content at lightening speed inside Xiaomi’s mobile news aggregator. 

Xiaomi is a global electronics company that manufactures and invests in smartphones, mobile applications, laptops, home appliances, laptops, and many other products. The company ships its smartphones with a home brewed web browser that has a news aggregator embedded within it. To consistently make content recommendations that appeal to individual users Xiaomi leveraged artificial intelligence (AI).

Milvus, an open-source vector database built by Zilliz, is used to power the core recommendation engine inside Xiaomi's smart news aggregator.


small_xiaomi.png

thumbnail_xiaomi.png

xiaomi.png

large_05.jpg

medium_05.jpg

small_05.jpg

thumbnail_05.jpg

05.jpg

Xiaomi’s Smart News Aggregator

large_beike.png

medium_beike.png

small_beike.png

thumbnail_beike.png

beike.png

Enriching video content recommendations for Opera.

Opera is a Norwegian software company that creates some of the world's most popular web browsers, Opera News, and other services. The company is a pioneer in leveraging artificial intelligence to make digital content recommendations. To ensure users are served a wide variety of videos, and to avoid recommending the same content multiple times, Opera built a smart video reduplication system inside its video content recommendation platform.

Milvus, an open-source vector database built by Zilliz, is used to power Opera’s smart video deduplication system.

thumbnail_Frame 247.jpg

Frame 247.jpg

large_opera-blog-cover.png

medium_opera-blog-cover.png

small_opera-blog-cover.png

thumbnail_opera-blog-cover.png

opera-blog-cover.png

Opera’s Intelligent Video Deduplication System

large_opera-img.png

medium_opera-img.png

small_opera-img.png

thumbnail_opera-img.png

opera-img.png

Mitigating threats to critical data and strengthening cybersecurity with real-time virus detection.

Trend Micro is a global leader in hybrid cloud security, network defense, small business security, and endpoint security. To protect Android devices from viruses, the company built Trend Micro Mobile Security—a mobile app that compares APKs from the Google Play Store to a database of known malware. If an APK with nefarious code is found, the app provides users with detailed information about the virus and its threat level.


Milvus, an open-source vector database built by Zilliz, is used to power Trend Micro Mobile Security’s virus vector comparison module. 



thumbnail_trend icon.png

trend icon.png

large_trend-blog-cover.png

medium_trend-blog-cover.png

small_trend-blog-cover.png

thumbnail_trend-blog-cover.png

trend-blog-cover.png

Trend Micro’s Mobile Security App

large_Trend Micro interface.png

medium_Trend Micro interface.png

small_Trend Micro interface.png

thumbnail_Trend Micro interface.png

Trend Micro interface.png

Extracting highlights from massive video data for individual marathon runners or participants in other events.

iYUNDONG builds artificial intelligence (AI) tools that analyze media captured during sporting events to automatically extract key highlights. For example, by uploading a single photo, images of individual marathon runners can be automatically retrieved from a massive dataset that includes photos and videos from the event. 

iYUNDONG leverages Milvus, an open-source vector database built by Zilliz, to power its core image retrieval system.

thumbnail_iyundong.png

iyundong.png

large_iyundong.jpg

medium_iyundong.jpg

small_iyundong.jpg

thumbnail_iyundong.jpg

iyundong.jpg

iYUNDONG‘s AI Automatically Extracts Sporting Event Highlights

large_iyundong.png

medium_iyundong.png

small_iyundong.png

Helping people create relevant, high-quality documents at the click of a button inside WPS Office's Writer.

WPS Office is a productivity tool developed by Kingsoft with over 300M monthly active users worldwide. The company’s artificial intelligence (AI) department built a smart writing assistant from scratch using semantic matching algorithms such as intent recognition and text clustering. The tool helps users quickly create outlines, individual paragraphs, and entire documents by simply inputting a title and selecting up to five keywords.

Kingsoft’s writing assistant uses Milvus, an open-source vector database built by Zilliz, to power its core vector processing module.



small_wps.png

thumbnail_wps.png

wps.png

large_wps-blog-cover.png

medium_wps-blog-cover.png

small_wps-blog-cover.png

thumbnail_wps-blog-cover.png

wps-blog-cover.png

Kingsoft’s AI-Powered Writing Assistant

Powering semantic search inside LucidWorks Fusion data discovery platform.

Lucidworks is an enterprise search technology company that offers an application development platform, commercial support, and value-add software for open-source Apache Lucene and Apache Solr. The company built Lucidworks Fusion in a modern, containerized, cloud-native architecture that simplifies the process of deploying AI-powered data discovery and search applications. To ensure users find relevant data fast, Lucidworks implemented semantic search using the vector similarity approach.

Milvus, an open-source vector database built by Zilliz, is used to power the core semantic search module inside Lucidworks Fusion.

Products

Resources

Open Source

About

Contribute

Meet the company behind Milvus.

Milvus:

The world's most popular open-source vector database

Learn More

Vector Similarity Search

AI for Everything

Ready for the Cloud

Optimal MLOps

Milvus is trusted by over 1000 organizations worldwide and has applications in almost every industry.

Mitigating threats to critical data and strengthening cybersecurity with real-time virus detection.

Read More

Industries

Wherever there's data we thrive.

Retail

Pharmaceutical

Real Estate

Entertainment

Intellectual Property

Sports

Financial Services

Healthcare

Advertising

Blog

Articles, updates, news and more.

Milvus 1.0: The World's Most Popular Open-Source Vector Database Just Got Better

Milvus Is an Open-Source Scalable Vector Database

Thanks to Milvus, Anyone Can Build a Vector Database for 1+ Billion Images