Everything as a vector. That’s the future that we see for neural search. It’s already working remarkably well in image, natural language and sound, and research is constantly driving us closer to building a single embedding space for all modalities.


Those of you who follow our GitHub repository will know that there has been a lot of activity on [Haystack](https://github.com/deepset-ai/haystack) recently! Not only have we been working hard on a set of new features, but we’ve received a lot of great code contributions and had a lot of interesting and productive conversations with our community. The sum of all this interaction is [our latest release v0.8.0](https://github.com/deepset-ai/haystack/releases/tag/v0.8.0), which features more changes than we could list here. Nonetheless, we wanted to walk you through the most exciting new features that will help you build your own semantic search pipeline.

### Milvus

Everything as a vector. That’s the future that we see for neural search. It’s already working remarkably well in image, natural language and sound, and research is constantly driving us closer to building a single embedding space for all modalities.
With this vision comes the need for vector optimized databases that can perform indexing and similarity calculations fast and so we are very excited to be introducing the [MilvusDocumentStore](https://github.com/deepset-ai/haystack/blob/master/haystack/document_store/milvus.py) in Haystack. Milvus encapsulates multiple Approximate Nearest Neighbours (ANN) libraries such as FAISS and ANNOY but is also built with the robustness and reliability required in production systems. It allows for dynamic data management, it can also run as a separate service via Docker and we found their library very smooth to work with. And of course, they are open source!

A MilvusDocumentStore is the perfect complement to a DPRRetriever or an EmbeddingRetriever and in our experimental runs, we have already seen great performance from it. You can expect to see a blog from us soon with benchmarks of all our retrievers and document stores.

Thanks go to our Github contributor lalitpagaria who was the driving force behind [this new feature](https://github.com/deepset-ai/haystack/pull/771)!

 

### Knowledge Graph QA

Our ambition with Haystack is to build the most comprehensive and practical framework for anything in the field of search or information retrieval and as such, we could not ignore Knowledge Graphs. Many companies build large, well maintained knowledge graphs to accurately capture the interrelations between different concepts and entities in their domain. Following strong interest from our community, we challenged ourselves to build a system that could interpret natural language questions and fetch structured data from such a graph.

![Deepsetai-1.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Deepsetai_1_12f42ea866.png)

###### *From [Yashu Seth](https://yashuseth.blog/2019/10/08/introduction-question-answering-knowledge-graphs-kgqa/)*

 

In the latest release, you will see the first small steps we’ve made in this direction. Haystack now has a [GraphDBKnowledgeGraph](https://github.com/deepset-ai/haystack/blob/master/haystack/knowledge_graph/graphdb.py) class which stores your triples and executes SPARQL queries. There is also a [Text2SparqlRetriever](https://github.com/deepset-ai/haystack/blob/master/haystack/graph_retriever/text_to_sparql.py) class which connects to a GraphDBKnowledgeGraph and functions as a semantic parser. It turns natural language questions into SPARQL queries using a BART sequence-to-sequence model. Thanks must go to the [Transformers](https://github.com/huggingface/transformers) library maintained by Huggingface which has made it incredibly easy to integrate new transformers models into Haystack!

Knowledge graph QA systems have a key advantage over extractive QA systems in that they can handle questions that require counting or operations like taking the maximum or minimum. However, the extraction of relational triples and the training of the semantic parser is by no means straightforward. But when done right, we see a lot of potential in the ensembling of the two methods.

If you want to see how to work with Knowledge Graphs in Haystack, please check out [our end-to-end tutorial](https://github.com/deepset-ai/haystack/blob/master/tutorials/Tutorial10_Knowledge_Graph.ipynb)! This will give you a sense of the first small successes we’ve had with the technique and we would love to hear from you if you want to drive Knowledge Graph QA forward with us!

### Pipeline Configs
The flexibility of Pipelines in Haystack has proven to be very popular with our community and we ourselves have enjoyed experimenting with different configurations. For example, we created Pipelines where a BM25 retriever is ensembled with a DPR retriever to get the best of both worlds and it is now possible in Haystack to build retrieval systems that return not only the full document text, but also their summaries and translations.

Now, with the introduction of Pipeline config files, it is easier than ever to define, save and load your custom Pipeline. Rather than writing lines of code to tweak Pipelines, everything that configures a Pipeline can be written in a YAML file allowing for easier and quicker experimentation cycles and also simpler maintenance of production systems.

This is an example of what a YAML config might look like.

![deepsetai-2.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/deepsetai_2_4a04c0ad31.png)

These few lines are all you need to load that config and run a Pipeline.

![deepsetai-3.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/deepsetai_3_8d18335643.png)

These configs are here to lower the barriers to entry, and allow you to get creative with your Pipeline designs.

 

### Confidence Scores
Apart from seeing your model’s top predictions, it is often very helpful to know how confident the model is in its predictions. For example, you might be building a system where you expect 90% accuracy and would rather route a request elsewhere if the model is unsure. The Softmaxed scores coming from a Question Answering head do not necessarily capture the model’s confidence of an answer being correct and can create many problems in practice.

The latest release takes us a step in the right direction as each prediction is now accompanied by a `probability` score in the range of 0 to 1, with values near 0 meaning very low confidence in the prediction and values near 1 meaning very high confidence. Using our transfer learning framework FARM, you can even go one step further and tune this `probability` score to be aligned with your metrics so that you can take confidence=0.9 to mean that the model will be right 90% of the time. If this is something that you’re interested in, have a look at [this tutorial](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering_confidence.py).

There are many different approaches and complexities to calculating model confidence but you can expect further support and new tools to help you in tuning your system.

 

### Web Crawler

Finally, it is central to Haystack’s design that it connects easily to other components and data sources. For this reason, we want to highlight a very nice community contribution from DIVYA-19 that has added a [web crawler](https://github.com/deepset-ai/haystack/pull/775) to Haystack! Simply provide URLs of pages you’d like to search over and have the Crawler convert them into Haystack documents.

We anticipate that more use cases will pop up where there is a need to connect data from different sources, formats and streams. If you are working with a certain application, data source or platform that is missing a connection to Haystack, please feel free to bring this to our attention in our GitHub repository or [through Slack](https://haystack.deepset.ai/community/join)! Haystack is designed to be an end-to-end search system but it is also our goal to make sure it integrates seamlessly into your tech stack.

 

### Conclusion

This latest release is packed full of new features and improvements, so much so that we didn’t even find space in this blog to talk about Evaluation Nodes or SQuAD to DPR data scripts. Haystack v0.8.0 is available now and you can see a full list of changes in [our release notes](https://github.com/deepset-ai/haystack/releases/tag/v0.8.0).

 

This blog article is reposted from: https://medium.com/deepset-ai/semantic-search-with-milvus-knowledge-graph-qa-web-crawlers-and-more-837451eae9fa

small_Deepsetai-1.png

thumbnail_Deepsetai-1.png

Deepsetai-1.png

Build a fully featured pipeline using the latest features from Haystack v0.8.0

Semantic Search with Milvus, Knowledge Graph QA, Web Crawlers and more!

Learn more about using semantic machine learning methodologies to power more relevant search results across your organization.

[Semantic search](https://lucidworks.com/post/what-is-semantic-search/) is a great tool to help your customers—or your employees—find the right products or information. It can even surface difficult-to-index information for better results. That said, if your semantic methodologies aren’t being deployed to work fast, they won’t do you any good. The customer or employee isn’t just going to sit around while the system takes its time responding to their query—and a thousand others are likely being ingested at the same time.

How can you make semantic search fast? Slow semantic search isn’t going to cut it.

Fortunately, this is the kind of problem Lucidworks loves to solve. We recently tested a modest-sized cluster—read on for more details—that resulted in 1500 RPS (requests per second) against a collection of over one million documents, with an average response time of roughly 40 milliseconds. Now that’s some serious speed.

 

### Implementing Semantic Search
To make lightning-fast, machine learning magic happen, Lucidworks has implemented semantic search using the semantic vector search approach. There are two critical parts.

 

#### Part One: The Machine Learning Model

First, you need a way to encode text into a numerical vector. The text could be a product description, a user search query, a question, or even an answer to a question. A semantic search model is trained to encode text such that text that is semantically similar to other text is encoded into vectors that are numerically “close” to one another. This encoding step needs to be fast in order to support the thousand or more possible customer searches or user queries coming in every second.

 

#### Part Two: The Vector Search Engine
Second, you need a way to quickly find the best matches to the customer search or user query. The model will have encoded that text into a numerical vector. From there, you need to compare that to all the numerical vectors in your catalog or lists of questions and answers to find those best matches—the vectors that are “closest” to the query vector. For that, you will need a vector engine that can handle all of that information effectively and at lightning speed. The engine could contain millions of vectors and you really just want the best twenty or so matches to your query. And of course, it needs to handle a thousand or so such queries every second.

To tackle these challenges, we added the vector search engine [Milvus](https://doc.lucidworks.com/fusion/5.3/8821/milvus) in our [Fusion 5.3 release](https://lucidworks.com/post/enhance-personalization-efforts-with-new-features-in-fusion/). Milvus is open-source software and it is fast. Milvus uses FAISS ([Facebook AI Similarity Search](https://ai.facebook.com/tools/faiss/)), the same technology Facebook uses in production for its own machine learning initiatives. When needed, it can run even faster on [GPU](https://en.wikipedia.org/wiki/Graphics_processing_unit). When Fusion 5.3 (or higher) is installed with the machine learning component, Milvus is automatically installed as part of that component so you can turn on all of these capabilities with ease.

The size of the vectors in a given collection, specified when the collection is created, depends on the model that produces those vectors. For example, a given collection could store the vectors created from encoding (via a model) all of the product descriptions in a product catalog. Without a vector search engine like Milvus, similarity searches would not be feasible across the entire vector space. So, similarity searches would have to be limited to pre-selected candidates from the vector space (for example, 500) and would have both slower performance and lower quality results. Milvus can store hundreds of billions of vectors across multiple collections of vectors to ensure that search is fast and results are relevant.

 

### Using Semantic Search

Let’s get back to the semantic search workflow, now that we’ve learned a little about why Milvus might be so important. Semantic search has three stages. During the first stage, the machine learning model is loaded and/or trained. Afterwards, data is indexed into Milvus and Solr. The final stage is the query stage, when the actual search occurs. We’ll focus on those last two stages below.

 

### Indexing into Milvus
![Lucidworks-1.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_1_47a9221723.png)

As shown in the above diagram, the query stage begins similarly to the indexing stage, just with queries coming in instead of documents. For each query:

1. The query is sent to the [Smart Answers](https://lucidworks.com/products/smart-answers/) index pipeline.
2. The query is then sent to the ML model.
3. The ML model returns a numeric vector (encrypted from the query). Again, the type of model determines the size of the vector.
4. The vector is sent to Milvus, which then determines which vectors, in the specified Milvus collection, best match the provided vector.
5. Milvus returns a list of unique IDs and distances corresponding to the vectors determined in step four.
6. A query containing those IDs and distances is sent to Solr.
7. Solr then returns an ordered list of the documents associated with those IDs.

 

### Scale Testing
In order to prove that our semantic search flows are running at the efficiency we require for our customers, we run scale tests using Gatling scripts on the Google Cloud Platform using a Fusion cluster with eight replicas of the ML model, eight replicas of the query service, and a single instance of Milvus. Tests were run using the Milvus FLAT and HNSW indexes. The FLAT index has 100% recall, but is less efficient – except when the datasets are small. The HNSW (Hierarchical Small World Graph) index still has high quality results and it has improved performance on larger datasets.

Let’s jump into some numbers from a recent example we ran:

![Lucidworks-2.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_2_3162113560.png)

![Lucidworks-3.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_3_3dc17f0ed8.png)

![Lucidworks-4.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_4_8a6edd2f59.png)

 

### Getting Started

The [Smart Answers](https://lucidworks.com/products/smart-answers/) pipelines are designed to be easy-to-use. Lucidworks has [pre-trained models that are easy-to-deploy](https://doc.lucidworks.com/how-to/734/set-up-a-pre-trained-cold-start-model-for-smart-answers) and generally have good results—though training your own models, in tandem with pre-trained models, will offer the best results. Contact us today to learn how you can implement these initiatives into your search tools to power more effective and delightful results.


> This blog is reposted from: https://lucidworks.com/post/how-to-build-fast-semantic-search/?utm_campaign=Oktopost-Blog+Posts&utm_medium=organic_social&utm_source=linkedin

medium_lucidworks.png

small_lucidworks.png

thumbnail_lucidworks.png

lucidworks.png

Build Semantic Search at Speed

Milvus accelerates machine learning application development and machine learning operations (MLOps). With Milvus, you can rapidly develop a minimum viable product (MVP) while keeping costs at lower limits.

![blog cover.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/blog_cover_4a9807b9e0.png)

[Milvus](https://milvus.io/) is an open-source vector database. It supports adding, deleting, updating, and near real-time search of massive vector datasets created by extracting feature vectors from unstructured data using AI models. With a comprehensive set of intuitive APIs, and support for multiple widely adopted index libraries (e.g., Faiss, NMSLIB, and Annoy), Milvus accelerates machine learning application development and machine learning operations (MLOps). With Milvus, you can rapidly develop a minimum viable product (MVP) while keeping costs at lower limits.

"What resources are available for developing an AI application with Milvus?” is commonly asked in the Milvus community. Zilliz, the [company](https://zilliz.com/) behind Milvus, developed a number of demos that leverage Milvus to conduct lightening-fast similarity search that powers intelligent applications. Source code of Milvus solutions can be found at [zilliz-bootcamp](https://github.com/zilliz-bootcamp). The following interactive scenarios demonstrate natural language processing (NLP), reverse image search, audio search, and computer vision.


Feel free to try out the solutions to gain some hands-on experience with specific scenarios! Share your own application scenarios via: 
- [Slack](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ)
- [GitHub](https://github.com/milvus-io/milvus/discussions)

 

**Jump to:**

- [Natural language processing (chatbots)](#natural-language-processing-chatbots)
- [Reverse image search](#reverse-image-search-systems)
- [Audio search](#audio-search-systems)
- [Video object detection (computer vision)](#video-object-detection-computer-vision)

 

### Natural language processing (chatbots)
Milvus can be used to build chatbots that use natural language processing to simulate a live operator, answer questions, route users to relevant information, and reduce labor costs. To demonstrate this application scenario, Zilliz built an AI-powered chatbot that understands semantic language by combining Milvus with [BERT](https://en.wikipedia.org/wiki/BERT_(language_model)), a machine learning (ML) model developed for NLP pre-training.

👉Source code：[zilliz-bootcamp/intelligent_question_answering_v2](https://github.com/zilliz-bootcamp/intelligent_question_answering_v2)

![1.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/1_c301a9e4bd.png)

#### How to use

1. Upload a dataset that includes question-answer pairs. Format questions and answers in two separate columns. Alternatively, a [sample dataset](https://www.zilliz.com/solutions/qa) is available for download.

2. After typing in your question, a list of similar questions will be retrieved from the uploaded dataset.

3. Reveal the answer by selecting the question most similar to your own.

👉Video：[[Demo] QA System Powered by Milvus](https://www.youtube.com/watch?v=ANgoyvgAxgU)

#### How it works

Questions are converted into feature vectors using Google’s BERT model, then Milvus is used to manage and query the dataset.

**Data processing:**

1. BERT is used to convert the uploaded question-answer pairs into 768-dimensional feature vectors. The vectors are then imported to Milvus and assigned individual IDs.
2. Question, and corresponding answer, vector IDs are stored in PostgreSQL.

**Searching for similar questions:**

1. BERT is used to extract feature vectors from a user's input question.
2. Milvus retrieves vector IDs for questions that are most similar to the input question.
3. The system looks up the corresponding answers in PostgreSQL.

 

### Reverse image search systems
Reverse image search is transforming e-commerce through personalized product recommendations and similar product lookup tools that can boost sales. In this application scenario, Zilliz built a reverse image search system by combining Milvus with [VGG](https://towardsdatascience.com/how-to-use-a-pre-trained-model-vgg-for-image-classification-8dd7c4a4a517), an ML model that can extract image features.

👉Source code：[zilliz-bootcamp/image_search](https://github.com/zilliz-bootcamp/image_search)

![2.jpeg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/2_09000e2e2e.jpeg)

#### How to use
1. Upload a zipped image dataset comprised of .jpg images only (other image file types are not accepted). Alternatively, a [sample dataset](https://www.zilliz.com/solutions/image-search) is available for download.
2. Upload an image to use as the search input for finding similar images.

👉Video: [[Demo] Image Search Powered by Milvus](https://www.youtube.com/watch?v=mTO8YdQObKY)

#### How it works
Images are converted into 512-dimensional feature vectors using the VGG model, then Milvus is used to manage and query the dataset.

**Data processing:**

1. The VGG model is used to convert the uploaded image dataset to feature vectors. The vectors are then imported to Milvus and assigned individual IDs.
2. Image feature vectors, and corresponding image file paths, are stored in CacheDB.

**Searching for similar images:**

1. VGG is used to convert a user’s uploaded image into feature vectors.
2. Vector IDs of images most similar to the input image are retrieved from Milvus.
3. The system looks up the corresponding image file paths in CacheDB.

 

### Audio search systems
Speech, music, sound effects, and other types of audio search makes it possible to quickly query massive volumes of audio data and surface similar sounds. Applications include identifying similar sound effects, minimizing IP infringement, and more. To demonstrate this application scenario, Zilliz built a highly efficient audio similarity search system by combining Milvus with [PANNs](https://arxiv.org/abs/1912.10211)—a large-scale pretrained audio neural networks built for audio pattern recognition.

👉Source code：[zilliz-bootcamp/audio_search](https://github.com/zilliz-bootcamp/audio_search)
![3.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/3_419bac3dd2.png)

#### How to use
1. Upload a zipped audio dataset comprised of .wav files only (other audio file types are not accepted). Alternatively, a [sample dataset](https://www.zilliz.com/solutions/audio-search) is available for download.
2. Upload a .wav file to use as the search input for finding similar audio.

👉Video: [[Demo] Audio Search Powered by Milvus](https://www.youtube.com/watch?v=0eQHeqriCXw)
#### How it works
Audio is converted into feature vectors using PANNs, large-scale pre-trained audio neural networks built for audio pattern recognition. Then Milvus is used to manage and query the dataset.

**Data processing:**

1. PANNs converts audio from the uploaded dataset to feature vectors. The vectors are then imported to Milvus and assigned individual IDs.
2. Audio feature vector IDs and their corresponding .wav file paths are stored in PostgreSQL.

**Searching for similar audio:**

1. PANNs is used to convert a user’s uploaded audio file into feature vectors.
2. Vector IDs of audio most similar to the uploaded file are retrieved from Milvus by calculating inner product (IP) distance.
3. The system looks up the corresponding audio file paths in MySQL.

 

### Video object detection (computer vision)
Video object detection has applications in computer vision, image retrieval, autonomous driving, and more. To demonstrate this application scenario, Zilliz built a video object detection system by combining Milvus with technologies and algorithms including [OpenCV](https://en.wikipedia.org/wiki/OpenCV), [YOLOv3](https://towardsdatascience.com/yolo-v3-object-detection-53fb7d3bfe6b), and [ResNet50](https://www.mathworks.com/help/deeplearning/ref/resnet50.html).

👉Source code: [zilliz-bootcamp/video_analysis](https://github.com/zilliz-bootcamp/video_analysis)

![4.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/4_54b4ceb2ad.png)

#### How to use
1. Upload a zipped image dataset comprised of .jpg files only (other image file types are not accepted). Ensure that each image file is named by the object it depicts. Alternatively, a [sample dataset](https://www.zilliz.com/solutions/video-obj-analysis) is available for download. 
2. Upload a video to use for analysis.
3. Click the play button to view the uploaded video with object detection results shown in real time.

👉Video: [[Demo] Video Object Detection System Powered by Milvus](https://www.youtube.com/watch?v=m9rosLClByc)

#### How it works
Object images are converted into 2048-dimensional feature vectors using ResNet50. Then Milvus is used to manage and query the dataset.

**Data processing:**

1. ResNet50 converts object images to 2048-dimensional feature vectors. The vectors are then imported to Milvus and assigned individual IDs.
2. Audio feature vector IDs and their corresponding image file paths are stored in MySQL.

**Detecting objects in video:**

1. OpenCV is used to trim the video.
2. YOLOv3 is used to detect objects in the video.
3. ResNet50 converts detected object images into 2048-dimensional feature vectors.

Milvus searches for the most similar object images in the uploaded dataset. Corresponding object names and image file paths are retrieved from MySQL.

small_blog cover.png

thumbnail_blog cover.png

blog cover.png

How to Make 4 Popular AI Applications with Milvus

Ensuring investments in artificial intelligence are successful requires effectively managing the life cycle of AI models and the applications they are used in. This article explains the concept of machine learning operations, or MLOps, and how the open-source vector database Milvus can be used to advance MLOps programs of varying complexity.

Building machine learning (ML) applications is a complex and iterative process. As more companies realize the untapped potential of unstructured data, the demand for [AI-powered data processing and analytics](https://zilliz.com/blog/Thanks-to-Milvus-Anyone-Can-Build-a-Vector-Database-for-1-Billion-Images) will continue to rise. Without effective machine learning operations, or MLOps, most ML application investments will wither on the vine. Research has found that [as little as 5%](https://www.forbes.com/sites/cognitiveworld/2020/03/31/modelops-is-the-key-to-enterprise-ai/?sh=44c0f5066f5a) of the AI adoptions companies plan to deploy actually reach deployment. Many organizations incur "model debt," where changes in market conditions, and failure to adapt to them, result in unrealized investments in models that linger unrefreshed (or worse, never get deployed at all).

This article explains MLOps, a systemic approach to AI model life cycle management, and how the open-source vector data management platform [Milvus](https://milvus.io/) can be used to operationalize AI at scale.

 

### What is MLOps?

Machine learning operations (MLOps), also known as model operations (ModelOps) or AI model operationalization, is necessary to build, maintain, and deploy AI applications at scale. As companies seek to apply the AI models they develop to hundreds of different scenarios, it is mission critical that models in use, and those under development, are operationalized across the entire organization. MLOps involves monitoring a machine learning model throughout its lifecycle, and governing everything from underlying data to the effectiveness of a production system that relies on a particular model.

![01.jpg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/01_362a07d156.jpg)

###### *MLOps governs the life cycle of a machine learning model.*

Gartner [defines](https://www.gartner.com/en/information-technology/glossary/modelops) ModelOps as the governance and life cycle management of a wide range of operationalized artificial intelligence and decision models. The core functionality of MLOps can be broken down as follows:

- **Continuous integration/continuous delivery (CI/CD):** A set of best practices borrowed from developer operations (DevOps), CI/CD is a method for delivering code changes more frequently and more reliably. [Continuous integration](https://www.gartner.com/en/information-technology/glossary/continuous-integration-ci) promotes implementing code changes in small batches while monitoring them with strict version control. [Continuous delivery](https://www.gartner.com/smarterwithgartner/5-steps-to-master-continuous-delivery/) automates the delivery of applications to various environments (e.g., testing and development environments). 

- **Model development environments (MDE):** A complex process for building, reviewing, documenting, and examining models, MDE help ensure models are created iteratively, documented as they are developed, trusted, and reproducible. Effective MDE ensures models can be explored, researched, and experimented on in a controlled manner.

- **Champion-challenger testing:** Similar to A/B testing methodology used by marketers, [champion-challenger testing](https://medium.com/decision-automation/what-is-champion-challenger-and-how-does-it-enable-choosing-the-right-decision-f57b8b653149) involves experimenting with different solutions to aid the decision-making process that proceeds committing to a single approach. This technique involves monitoring and measuring performance in real time to identify which deviation works best.

- **Model versioning:** As with any complex system, machine learning models are developed in steps by many different people—resulting in data management questions around versions of data and ML models. Model versioning helps manage and govern the iterative process of ML development where data, models, and code may evolve at different rates.

- **Model store and rollback:** When a model is deployed, its corresponding image file must be stored. Rollback and recovery abilities allow MLOps teams to revert to a previous model version if needed.

Using just one model in a production application presents a number of difficult challenges. MLOps is a structured, repeatable method that relies on tools, technology, and best practices to overcome technical or business problems that arise during a machine learning model’s life cycle. Successful MLOps maintains efficiency across teams working to build, deploy, monitor, retrain, and govern AI models and their use in production systems. 

 

### Why is MLOps necessary?

As depicted in the ML model life cycle above, building a machine learning model is an iterative process that involves incorporating new data, retraining models, and dealing with general model decay over time. These are all issues that traditional developer operations, or DevOps, does not address or provide solutions for. MLOps has become necessary as a way to manage investment in AI models and ensure a productive model life cycle. Because machine learning models will be leveraged by a variety of different production systems, MLOps becomes integral to making sure requirements can be met across different environments and amid varying scenarios.

![02.jpg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/02_403e7f2fe2.jpg)

###### *A machine learning model being deployed in a cloud environment that feeds into an application.*

 

The simple illustration above depicts a machine learning model being deployed in a cloud environment that feeds into an application. In this basic scenario, a number of problems could arise that MLOps helps overcome. Because the production application relies on a specific cloud environment, there are latency requirements that the data scientists who developed the ML model don’t have access to. Operationalizing the model life cycle would make it possible for data scientists or engineers with deep knowledge of the model to identify and troubleshoot problems that arise in specific production environments. 

Not only are machine learning models trained in different environments from the production applications they are used in, but they also often rely on historical datasets that differ from the data used in production applications. With MLOps the entire data science team, from those developing the model to people working at the application level, have a means of sharing and requesting information and assistance. The rate at which data and markets change make it imperative that there is as little friction as possible between all key stakeholders and contributors that will come to depend on a given machine learning model. 

### Supporting the transition to Software 2.0

[Software 2.0](https://karpathy.medium.com/software-2-0-a64152b37c35) is the idea that software development will experience a paradigm shift as artificial intelligence increasingly plays a central role in writing AI models that power software applications. Under Software 1.0, development involves programmers writing explicit instructions using a specific programming language (e.g., Python, C++). Software 2.0 is far more abstract. Although people provide input data and set parameters, neural networks are difficult for humans to understand due to their sheer complexity— with typical networks containing millions of weights that impact outcomes (and sometimes billions or trillions). 

DevOps was built around Software 1.0’s reliance on specific instructions dictated by programmers using languages, but never considered the life cycle of a machine learning model that powers a variety of different applications. MLOps addresses the need for the process of managing software development to change alongside the software under development. As Software 2.0 becomes the new standard for computer-based problem solving, having the right tools and processes for managing model life cycles will make or break investments in new technology. Milvus is an open-source vector similarity search engine built to support the transition to Software 2.0 and manage model life cycles with MLOps.

![03.jpg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/03_c63c501995.jpg)

###### *How Milvus supports the transition to Software 2.0.*

 

### Operationalizing AI at scale with Milvus

Milvus is a vector data management platform that was made specifically for storing, querying, updating, and maintaining massive, trillion-scale vector datasets. The platform powers vector similarity search and can integrate with widely adopted index libraries, including Faiss, NMSLIB, and Annoy. By pairing AI models that convert unstructured data to vectors with Milvus, applications spanning new drug development, biometric analysis, recommendation systems and much more can be created. 

[Vector similarity search](https://blog.milvus.io/vector-similarity-search-hides-in-plain-view-654f8152f8ab) is the go-to solution for unstructured data data processing and analytics, and vector data is quickly emerging as a core data type. A comprehensive data management system like Milvus facilitates operationalizing AI in many ways, including:

- Providing an environment for model training that ensures more aspects of development are done in one place, facilitating cross team collaboration, model governance, and more. 

- Offering a comprehensive set of APIs that support popular frameworks such as Python, Java, and Go, making it easy to integrate a common set of ML models. 

- Compatibility with Google Colaboratory, a Jupyter notebook environment that runs in a browser, simplifies the process of compiling Milvus from source code and running basic Python operations.

- Automated machine learning (AutoML) functionality makes it possible to automate the tasks associated with applying machine learning to real world problems. Not only does AutoML lead to efficiency improvements, but it makes it possible for non-experts to take advantage of machine learning models and techniques.


Regardless of the machine learning applications you’re building today, or the plans you have for applications in the future, Milvus is a flexible data management platform created with Software 2.0 and MLOps in mind. To learn more about Milvus or make contributions, find the project on [Github](https://github.com/milvus-io). To get involved with the community or ask questions, join our [Slack](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ) channel. Hungry for more content? Check out the following resources:

- [Milvus Is an Open-Source Scalable Vector Database](https://zilliz.com/blog/Milvus-Is-an-Open-Source-Scalable-Vector-Database)
- [Milvus Was Built for Massive-Scale (Think Trillion) Vector Similarity Search](https://zilliz.com/blog/Milvus-Was-Built-for-Massive-Scale-Think-Trillion-Vector-Similarity-Search)
- [Set Up Milvus in Google Colaboratory for Easy ML Application Building](https://zilliz.com/blog/Set-Up-Milvus-in-Google-Colaboratory-for-Easy-ML-Application-Building)

large_milvus.jpg

medium_milvus.jpg

small_milvus.jpg

thumbnail_milvus.jpg

milvus.jpg

MLOps is superseding DevOps as we make the transition to Software 2.0. Find  what model operations is, and how the open-source vector database Milvus supports it. 

Operationalize AI at Scale with Software 2.0, MLOps, and Milvus

Beike wanted to create a personalized home search experience. Discover how they leveraged Milvus to build an intelligent real estate search platform capable of providing tailored recommendations in near real-time. 

Artificial intelligence (AI) has [powerful applications](https://medium.com/unstructured-data-service/the-easiest-way-to-search-among-1-billion-image-vectors-d6faf72e361f#d62b) in real estate that are transforming the home search process. Tech savvy real estate professionals have been taking advantage of AI for years, recognizing its ability to help clients find the right home faster and simplify the process of purchasing property. The coronavirus pandemic has [accelerated](https://www.pwc.com/ca/en/industries/real-estate/emerging-trends-in-real-estate-2021/shifting-landscape-proptech.html) interest, adoption, and investement in property technology (or proptech) worldwide, suggesting it will play an increasingly greater role in the real estate industry moving forward. 

This article explores how Beike used vector similarity search to build a house hunting platform that provides personalized results and recommends listings in near real-time.

**Jump to:**

- [What is vector similarity search?](#what-is-vector-similarity-search)

- [How does Milvus manage vector data?](#how-does-milvus-manage-vector-data)

- [How does Beike use Milvus to make house hunting smarter?](#how-does-beike-use-milvus-to-make-house-hunting-smarter)

 

### What is vector similarity search?

[Vector similarity search](https://medium.com/unstructured-data-service/vector-similarity-search-hides-in-plain-view-654f8152f8ab) has applications spanning a wide variety of artificial intelligence, deep learning, and traditional vector calculation scenarios. The proliferation of AI technology is in part attributed to vector search and its ability to make sense of unstructured data, which includes things like images, video, audio, behavior data, documents, and much more.

Unstructured data makes up an estimated 80-90% of all data, and extracting insights from it is quickly becoming a requrement for businesses that want to remain competitive in an ever-changing world. Increasing demand for unstructured data analytics, rising compute power, and declining compute costs have made AI-enabled vector search more accessible than ever.

![beike-blog-img1.jpg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/beike_blog_img1_2dc95cac08.jpg)

Traditionally, unstructured data has been a challenge to process and analyze at scale because it doesn't follow a predefined model or organizational structure. Neural networks (e.g., CNN, RNN, and BERT) make it possible to convert unstructured data into feature vectors, a numerical data format that can be easily interpreted by computers. Algorithms are then used to calculate similarity between vectors using metrics like cosine similarity or Euclidean distance.

Ultimately, vector similarity search is a broad term that desribes techniques for identifying similar things in massive datasets. Beike uses this technology to power an intelligent home search engine that automatically recommends listings based on individual user preferences, search history, and property criteria—accelerating the real estate search and purchase process. Milvus is an open-source vector database that connects information with algorithms, enabling Beike to develop and manage its AI real estate platform.

 

### How does Milvus manage vector data?

Milvus was built specifically for large-scale vector data management, and has applications spanning image and video search, chemical similarity analysis, personalized recommendation systems, conversational AI, and much more. Vector datasets stored in Milvus can be efficiently queried, with most implementations following this general process:

![beike-blog-img2.jpg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/beike_blog_img2_d5abb58f95.jpg)

 

### How does Beike use Milvus to make house hunting smarter?

Commonly described as China's answer to Zillow, [Beike](https://bj.ke.com/) is an online platform that allows real estate agents to list properties for rent or sale. To help improve the home search experience for house hunters, and to help agents close deals faster, the company built an AI-powered search engine for its listing database. Beike’s real estate listing database was converted into feature vectors then fed into Milvus for indexing and storage. Milvus is then used to conduct similarity search based on an input listing, search criteria, user profile, or other criteria.

For example, when searching for more homes similar to a given listing, features such as floor plan, size, orientation, interior finishings, paint colors, and more are extracted. Since the original database of property listings listing data has been [indexed](https://medium.com/unstructured-data-service/how-to-choose-an-index-in-milvus-4f3d15259212), searches can be conducted in mere milliseconds. Beike’s final product had an average query time of 113 milliseconds on a dataset containing over 3 million vectors. However, Milvus is capable of maintaining efficient speeds on trillion-scale datasets—making light work of this relatively small real estate database. In general, the system follows the following process:

1. Deep learning models (e.g., CNN, RNN, or BERT) convert unstructured data to feature vectors, which are then imported to Milvus.

2. Milvus stores and indexes the feature vectors.

3. Milvus returns similarity search results based on user queries. 

![milvus-overview-diagram.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/milvus_overview_diagram_d17cda0e47.png)

###### *An overview of Milvus.*

 

Beike’s intelligent real estate search platform is powered by a recommendation algorithm that calculates vector similarity using cosine distance. The system finds similar homes based on favorite listings and search criteria. At a high level, it works as follows:

1. Based on an input listing, characteristics such as floor plan, size, and orientation are used to extract 4 collections of feature vectors.

2. The extracted feature collections are used to perform similarity search in Milvus. Results of the query for each collection of vectors are a measure of similarity between the input listing and other similar listings. 

3. The search results from each of the 4 vector collections are compared then used to recommend similar homes. 

![beike-intelligent-house-platform-diagram.jpg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/beike_intelligent_house_platform_diagram_6e278da118.jpg)

###### *An overview of Beike’s intelligent house hunting platform.*

 

As the figure above shows, the system implements an A/B table switching mechanism for updating data. Milvus stores the data for the first T days in table A, on day T+1 it starts to store data in table B, on day 2T+1, it starts to rewrite table A, and so forth. 

 

### To learn more about making things with Milvus, check out the following resources:

- [Building an AI-Powered Writing Assistant for WPS Office](https://www.zilliz.com/blog/Building-an-AI-Powered-Writing-Assistant-with-WPS-Office)

- [Making with Milvus: AI-Powered News Recommendation Inside Xiaomi's Mobile Browser](https://www.zilliz.com/blog/Making-with-Milvus-AI-Powered-News-Recommendation-Inside-Xiaomi-Mobile-Browser)

large_beike.png

medium_beike.png

small_beike.png

thumbnail_beike.png

beike.png

AI is transforming the real estate industry, discover how intelligent proptech accelerates the home search and purchase process. 

Making With Milvus: AI-Infused Proptech for Personalized Real Estate Search

Artificial intelligence influences what we read, the things we buy, who we become friends with, and more everyday human behavior. A huge part of what makes many modern AI applications possible is vector similarity search. Understanding how this underlying technology works can help demystify AI, providing insight into what computer intelligence is really made of. 

[Artificial intelligence (AI)](https://medium.com/unstructured-data-service/the-easiest-way-to-search-among-1-billion-image-vectors-d6faf72e361f#a291) has the potential to change the way even the most obscure things are done. For example, every year (before COVID, anyway) over 73,000 people congregate to compete in the Hong Kong Marathon. In order to properly sense and record finishing times for all race participants, organizers distribute 73,000 RFID chip timers to attach to each runner. Chip timing is a complex undertaking with obvious downsides. Materials (chips and electronic reading devices) must be purchased or rented from timing companies, and a registration area must be staffed for runners to collect chips on race day. Additionally, if sensors are installed only at the start and finish lines, it’s possible for unscrupulous runners to cut the course. 

![blog-1.jpeg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/blog_1_e55c133e05.jpeg)

###### Timing marathon runners is a logistical challenge few think about.

Now imagine a [video AI](https://cloud.google.com/video-intelligence) application capable of automatically identifying individual runners from footage captured at the finish line using a single photo. Rather than attach timing chips to each participant, runners simply upload a photo of themselves via an app after they cross the finish line. Instantly, a personalized highlight reel, race stats, and other relevant information is provided. Cameras installed at various points throughout the race can capture additional footage of participants and ensure each runner traverses the entire course. Which solution seems easier and more cost-effective to implement?

Although the Hong Kong Marathon doesn’t leverage machine learning to replace timing chips (yet), this example illustrates the potential AI has to drastically alter everything around us. For race timing, it reduces tens of thousands of chips to a few cameras paired with machine learning algorithms. But video AI is just one of many applications for vector similarity search, a process that uses artificial intelligence to analyze massive, trillion-scale unstructured datasets. This article provides an overview of vector search technology including what it is, how it can be used, as well as the open-source software and resources making it more accessible than ever before.





**Jump to:**

- [What is vector similarity search?](#what-is-vector-similarity-search)

- [What are some applications of vector similarity search?](#what-are-some-applications-of-vector-similarity-search)

- [Open-source vector similarity search software and resources.](#open-source-vector-similarity-search-software-and-resources)

### What is vector similarity search?

Video data is incredibly detailed and increasingly common, so logically it seems like it would be a great unsupervised learning signal for building video AI. In reality, this is not the case. Processing and analyzing video data, especially in large volumes, remains a [challenge for artificial intelligence](https://arxiv.org/pdf/1905.11954.pdf). Recent progress in this field, like much of the progress made in unstructured data analytics, is owed in large part to vector similarity search.

The problem with video, like all unstructured data, is that it doesn’t follow a predefined model or organizational structure, making it difficult to process and analyze at scale. Unstructured data includes things like images, audio, social media behavior, and documents, collectively accounting for an estimated 80-90%+ of all data. Companies are increasingly aware of the business-critical insights buried in massive, enigmatic unstructured datasets, driving demand for AI applications that can tap into this unrealized potential.

Using [neural networks](https://en.wikipedia.org/wiki/Neural_network) such as CNN, RNN, and BERT, unstructured data can be converted into feature vectors (aka embeddings), a machine-readable numerical data format. Algorithms are then used to calculate the similarity between vectors using measures like cosine similarity or Euclidean distance. Vector embedding and similarity search make it possible to analyze and build machine learning applications using previously indiscernible datasets.

Vector similarity is calculated using established algorithms however, unstructured datasets are typically massive. This means efficient and accurate search requires vast storage and compute power. To [accelerate similarity search](https://medium.com/unstructured-data-service/how-to-choose-an-index-in-milvus-4f3d15259212#7a9a) and reduce resource requirements, approximate nearest neighbor (ANN) search algorithms are used. By clustering similar vectors together, ANN algorithms make it possible to send queries to the clusters of vectors most likely to contain similar vectors, rather than searching the entire dataset. Although this approach is faster, it sacrifices some degree of accuracy. Leveraging ANN algorithms allows vector search to comb through billions of deep learning model insights in millisecond.


### What are some applications of vector similarity search?

Vector similarity search has applications spanning a wide variety of artificial intelligence, deep learning, and traditional vector calculation scenarios. The following provides a high-level overview of various vector similarity search applications:

**E-commerce:** Vector similarity search has broad applicability in e-commerce, including reverse image search engines that allow shoppers to search for products using an image captured on their smartphone or found online. Additionally, personalized recommendations based on user behavior, interests, purchase history, and more can be served by specialized recommender systems that rely on vector search.

**Physical & Cyber Security:** Video AI is just one of many applications for vector similarity search in the security field. Other scenarios include facial recognition, behavior tracing, identity authentication, intelligent access control, and more. Additionally, vector similarity search plays an important role in thwarting increasingly common and [sophisticated cyberattacks](https://www.wsj.com/articles/hack-suggests-new-scope-sophistication-for-cyberattacks-11608251360). For example, [code similarity search](https://medium.com/gsi-technology/application-of-ai-to-cybersecurity-part-3-19659bdb3422) can be used to identify security risks by comparing a piece of software to a database of known vulnerabilities or malware.

**Recommendation Engines:** Recommendation engines are systems that use machine learning and data analysis to suggest products, services, content, and information to users. User behavior, the behavior of similar users, and other data is processed using deep learning methods to generate recommendations. With enough data, algorithms can be trained to understand relationships between entities and invent ways to represent them autonomously. Recommendation systems have broad applicability and are something people already interact with every day, including content recommendations on Netflix, shopping recommendations on Amazon, and news feeds on Facebook.

**Chatbots:** Traditionally, chatbots are built using a regular knowledge graph that requires a large training dataset. However, chatbots built using deep learning models don’t need to preprocess data—instead, a map between frequent questions and answers is created. Using a pre-trained natural language processing (NLP) model, feature vectors can be extracted from the questions and then stored and queried using a [vector data management platform](https://medium.com/unstructured-data-service/the-easiest-way-to-search-among-1-billion-image-vectors-d6faf72e361f#92e0).

**Image or Video Search:** Deep learning networks have been used to recognize visual patterns since the late 1970s, and modern technology trends have made image and video search more powerful and accessible than ever before. 

**Chemical Similarity Search:** Chemical similarity is key to predicting the properties of chemical compounds and finding chemicals with specific attributes, making it indispensable to the development of new drugs. Fingerprints represented by feature vectors are created for each molecule, and then the distances between vectors are used to measure similarity. Using AI for new drug discovery is gaining momentum in the tech industry, with ByteDance (TikTok’s Chinese parent company) starting to [hire talent in the field](https://techcrunch.com/2020/12/23/bytedance-ai-drug/).


### Open-source vector similarity search software and resources.

Moore’s law, cloud computing, and declining resource costs are macro trends that have made artificial intelligence more accessible than ever. Thanks to open-source software and other publicly available resources, building AI/ML applications isn’t just for big tech companies. Below we provide a brief overview of Milvus, an open-source vector data management platform, and also highlight some publicly available datasets that help put AI within everyone’s reach.

#### Milvus, an open-source vector data management platform

[Milvus](https://milvus.io/) is an open-source vector data management platform built specifically for massive-scale vector data. Powered by Facebook AI Similarity Search (Faiss), Non-Metric Space Library (NMSLIB), and Annoy, Milvus brings a variety of powerful tools together under a single platform while extending their standalone functionality. The system was purpose built for storing, processing, and analyzing large vector datasets, and can be used to build all the AI applications (and more) mentioned above. 

More information about Milvus can be found on its [website](https://milvus.io/). Tutorials, instructions for setting up Milvus, benchmark testing, and information on building a variety of different applications is available in the [Milvus bootcamp](https://github.com/milvus-io/bootcamp). Developers interested in making contributions to the project can join Milvus' open-source community on [GitHub](https://github.com/milvus-io).

#### Public datasets for artificial intelligence and machine learning

It is no secret that technology giants like Google and Facebook have a data advantage over the little guys, with some pundits even advocating for a “[progressive data-sharing mandate](https://www.technologyreview.com/2019/06/06/135067/making-big-tech-companies-share-data-could-do-more-good-than-breaking-them-up/)” that would force companies that exceed a certain size to share some anonymized data with smaller rivals. Fortunately, there are thousands of publicly available datasets that can be used for AL/ML projects: 

- **The People’s Speech Dataset:** This [dataset from ML Commons](https://mlcommons.org/en/peoples-speech/) offers the largest speech dataset in the world, with over 87,000 hours of transcribed speech in 59 different languages. 

- **UC Irvine Machine Learning Repository:** The University of California at Irvine maintains [hundreds of public datasets](https://archive.ics.uci.edu/ml/index.php) in an effort to help the machine learning community. 

- **Data.gov:** The U.S. government offers [hundreds of thousands of open datasets](https://www.data.gov/) that span education, climate, COVID-19, and more. 

- **Eurostat:** The European Union’s statistical office provides [open datasets](https://ec.europa.eu/eurostat/data/database) spanning a variety of industries from economy and finance to population and social conditions. 

- **Harvard Dataverse:** [The Harvard Dataverse Repository](https://dataverse.harvard.edu/) is a free data repository open to researchers across disciplines. Many datasets are public, while others come with more restricted terms of use.

Although this list is by no means exhaustive, it is a good starting point for discovering the surprisingly wide variety of open datasets. For more information on public datasets as well as choosing the right data for your next ML or data science project, check out this [Medium post](https://altexsoft.medium.com/best-public-datasets-for-machine-learning-and-data-science-sources-and-advice-on-the-choice-636a0e754052).


## To learn more about vector similarity search, check out the following resources:
- [Thanks to AI, Anyone Can Build a Search Engine for 1+ Billion Images](https://zilliz.com/blogs/Thanks-to-AI-Anyone-Can-Build-a-Search-Engine-for-1-Billion-Images)
- [Milvus Was Built for Massive-Scale (Think Trillion) Vector Similarity Search](https://zilliz.com/blogs/Milvus-Was-Built-for-Massive-Scale-Think-Trillion-Vector-Similarity-Search)
- [Accelerating Search on Really Big Data with Vector Indexing](https://zilliz.com/blogs/Accelerating-Similarity-Search-on-Really-Big-Data-with-Vector-Indexing) 



large_plainview.jpg

medium_plainview.jpg

small_plainview.jpg

thumbnail_plainview.jpg

plainview.jpg

Discover what vector similarity search is, its various applications, and the public resources making artificial intelligence more accessible than ever.

Products

Resources

Open Source

About

Contribute

Blog

All

Product

Company

Technology

Case Study

Release

Load More