VOVA, an emerging e-commerce platform, prioritizes convenient and positive user experiences. Learn how the company used Milvus to build an image search tool to help shoppers find items they encounter in real life.

Jump to:
 
- [How does image search work?](#how-does-image-search-work)
- [Target detection using the YOLO model](#target-detection-using-the-yolo-model)
- [Image feature vector extraction with ResNet](#image-feature-vector-extraction-with-resnet)
- [Vector similarity search powered by Milvus](#vector-similarity-search-powered-by-milvus)
- [VOVA's shop by image tool](#vovas-shop-by-image-tool)

Online shopping surged in 2020, [up 44%](https://www.digitalcommerce360.com/2021/02/15/ecommerce-during-coronavirus-pandemic-in-charts/) in large part due to the coronavirus pandemic. As people sought to socially distance and avoid contact with strangers, no-contact delivery became an incredibly desirable option for many consumers. This popularity has also led to people buying a greater variety of goods online, including niche items that can be hard to describe using a traditional keyword search.

To help users overcome the limitations of keyword-based queries, companies can build image search engines that allow users to use images instead of words for search. Not only does this allow users to find items that are difficult to describe, but it also helps them shop for things they encounter in real life. This functionality helps build a unique user experience and offers general convenience that customers appreciate.

VOVA is an emerging e-commerce platform that focuses on affordability and offering a positive shopping experience to its users, with listings covering millions of products and support for 20 languages and 35 major currencies. To enhance the shopping experience for its users, the company used Milvus to build image search functionality into its e-commerce platform. The article explores how VOVA successfully built an image search engine with Milvus. 

 

### How does image search work?

VOVA's shop by image system searches the company's inventory for product images that are similar to user uploads. The following chart shows the two stages of the system process, the data import stage (blue) and the query stage (orange):

1. Use the YOLO model to detect targets from uploaded photos;
2. Use ResNet to extract feature vectors from the detected targets; 
3. Use Milvus for vector similarity search.

![Vova-1.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Vova_1_47ee6f2da9.png)

 

### Target detection using the YOLO model

VOVA's mobile apps on Android and iOS currently support image search. The company uses a state-of-the-art, real-time object detection system called YOLO (You only look once) to detect objects in user uploaded images. The YOLO model is currently in its fifth iteration. 

YOLO is a one-stage model, using only one convolutional neural network (CNN) to predict categories and positions of different targets. It is small, compact, and well suited for mobile use. 

YOLO uses convolutional layers to extract features and fully-connected layers to obtain predicted values. Drawing inspiration from the GooLeNet model, YOLO’s CNN includes 24 convolutional layers and two fully-connected layers.

As the following illustration shows, a 448 &times; 448 input image is converted by a number of convolutional layers and pooling layers to a 7 &times; 7 &times; 1024-dimensional tensor (depicted in the third to last cube below), and then converted by two fully-connected layers to a 7 &times; 7 &times; 30-dimensional tensor output.

The predicted output of YOLO P is a two-dimensional tensor, whose shape is [batch,7 &times;7 &times;30]. Using slicing, P[:,0:7&times;7&times;20] is the category probability, P[:,7&times;7&times;20:7&times;7&times;(20+2)] is the confidence, and P[:,7&times;7&times;(20+2)]:] is the predicted result of the bounding box.

![vova-2.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/vova_2_1ccf38f721.png)

 

### Image feature vector extraction with ResNet

VOVA adopted the residual neural network (ResNet) model to extract feature vectors from an extensive product image library and user uploaded photos. ResNet is limited because as the depth of a learning network increases, the accuracy of the network decreases. The image below depicts ResNet running the VGG19 model (a variant of the VGG model) modified to include a residual unit through the short circuit mechanism. VGG was proposed in 2014 and includes just 14 layers, while ResNet came out a year later and can have up to 152.

The ResNet structure is easy to modify and scale. By changing the number of channels in the block and the number of stacked blocks, the width and depth of the network can be easily adjusted to obtain networks with different expressive capabilities. This effectively solves the network degeneration effect, where accuracy declines as the depth of learning increases. With sufficient training data, a model with improving expressive performance can be obtained while gradually deepening the network. Through model training, features are extracted for each picture and converted to 256-dimensional floating point vectors.

![vova-3.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/vova_3_df4b810281.png)

 

### Vector similarity search powered by Milvus
 
VOVA's product image database includes 30 million pictures and is growing rapidly. To quickly retrieve the most similar product images from this massive dataset, Milvus is used to conduct vector similarity search. Thanks to a number of optimizations, Milvus offers a fast and streamlined approach to managing vector data and building machine learning applications. Milvus offers integration with popular index libraries (e.g., Faiss, Annoy), supports multiple index types and distance metrics, has SDKs in multiple languages, and provides rich APIs for managing vector data.

Milvus can conduct similarity search on trillion-vector datasets in milliseconds, with a query time under 1.5 seconds when nq=1 and an average batch query time under 0.08 seconds. To build its image search engine, VOVA referred to the design of Mishards, Mivus' sharding middleware solution (see the chart below for its system design), to implement a highly available server cluster. By leveraging the horizontal scalability of a Milvus cluster, the project requirement for high query performance on massive datasets was met.

![vova-4.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/vova_4_e305f1955c.png)


### VOVA's shop by image tool
 
The screenshots below show the VOVA search by image shopping tool on the company's Android app.

![vova-5.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/vova_5_c4c25a3bae.png)
 
As more users search for products and upload photos, VOVA will continue to optimize the models that power the system. Additionally, the company will incorporate new Milvus functionality that can further enhance the online shopping experience of its users.

### Reference

**YOLO:**

https://arxiv.org/pdf/1506.02640.pdf

https://arxiv.org/pdf/1612.08242.pdf

**ResNet:**

https://arxiv.org/abs/1512.03385

**Milvus:**

https://milvus.io/docs/overview.md

large_vova-thumbnail.jpg

medium_vova-thumbnail.jpg

small_vova-thumbnail.jpg

thumbnail_vova-thumbnail.jpg

vova-thumbnail.jpg

Discover how Milvus, an open-source vector database, was used by e-commerce platform VOVA to power shopping by image. 

Building a Search by Image Shopping Experience with VOVA and Milvus

We are proud to announce Milvus v1.0, a stable, long-term support version of  the world’s most popular open-source vector database. Discover the highlight features that Milvus v1.0 has to offer.

![Milvus.jpeg](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Milvus_510cf50aee.jpeg)

Milvus is an open-source vector database designed to manage massive million, billion, or even trillion vector datasets. Milvus has broad applications spanning new drug discovery, computer vision, autonomous driving, recommendation engines, chatbots, and much more. 

In March, 2021 Zilliz, the company behind Milvus, released the platform's first long-term support version—Milvus v1.0. After months of extensive testing, a stable, production ready version of the world's most popular vector database is ready for prime time. This blog article covers some Milvus fundamentals as well as key features of v1.0.

 

### Milvus distributions

Milvus is available in CPU-only and GPU-enabled distributions. The former relies exclusively on CPU for index building and search; the latter enables CPU and GPU hybrid search and index building that further accelerates Milvus. For example, using the hybrid distribution, CPU can be used for search and GPU for index building, further improving query efficiency.

Both Milvus distributions are available in Docker. You can either compile Milvus from Docker (if your operating system supports it) or compile Milvus from source code on Linux (other operating systems are not supported). 

 

### Embedding vectors

Vectors are stored in Milvus as entities. Each entity has one vector ID field and one vector field. Milvus v1.0 supports integer vector IDs only. When creating a collection within Milvus, vector IDs can be automatically generated or manually defined. Milvus ensures auto-generated vector IDs are unique however, manually defined IDs can be duplicated within Milvus. If manually defining IDs, users are responsible for making sure all IDs are unique.

 

### Partitions

Milvus supports creating partitions in a collection. In situations where data is inserted regularly and historical data isn't significant (e.g., streaming data), partitions can be used to accelerate vector similarity search. One collection can have up to 4,096 partitions. Specifying a vector search within a specific partition narrows the search and may significantly reduce query time, particularly for collections that contain more than a trillion vectors.

 

### Index algorithm optimizations
Milvus is built on top of multiple widely-adopted index libraries, including Faiss, NMSLIB, and Annoy. Milvus is far more than a basic wrapper for these index libraries. Here are some of the major enhancements that have been made to the underlying libraries:

- IVF index performance optimizations using the Elkan k-means algorithm.
- FLAT search optimizations.
- IVF_SQ8H hybrid index support, which can reduce index file sizes by up to 75% without sacrificing data accuracy. IVF_SQ8H is built upon IVF_SQ8, with identical recall but much faster query speed. It was designed specifically for Milvus to harnesses the parallel processing capacity of GPUs, and the potential for synergy between CPU/GPU co-processing.
- Dynamic instruction set compatibility.

 

### Search, index building, and other Milvus optimizations

The following optimizations have been made to Milvus to improve search and index building performance. 

- Search performance is optimized in situations when the number of queries (nq) is less than the number of CPU threads. 
- Milvus combines search requests from a client that take the same topK and search parameters. 
- Index building is suspended when search requests come in. 
- Milvus automatically preloads collections to memory at start.
- Multiple GPU devices can be assigned to accelerate vector similarity search.

 

### Distance metrics

Milvus is a vector database built to power vector similarity search. The platform was built with MLOps and production level AI applications in mind. Milvus supports a wide range of distance metrics for calculating similarity, such as Euclidean distance (L2), inner product (IP), Jaccard distance, Tanimoto, Hamming distance, superstructure, and substructure. The last two metrics are commonly used in molecular search and AI-powered new drug discovery.

 

### Logging

Milvus supports log rotation. In the system configuration file, milvus.yaml, you can set the size of a single log file, the number of log files, and log output to stdout.

 

### Distributed solution

Mishards, a Milvus sharding middleware, is the distributed solution for Milvus With one write node and an unlimited number of read nodes, Mishards unleashes the computational potential of server cluster. Its features include request forwarding, read/write splitting, dynamic/horizontal scaling, and more.

 

### Monitoring

Milvus is compatible with Prometheus, an open-source system monitoring and alerts toolkit. Milvus adds support for Pushgateway in Prometheus, making it possible for Prometheus to acquire short-lived batch metrics. The monitoring and alerts system works as follows:

- The Milvus server pushes customized metrics data to Pushgateway.
- Pushgateway ensures ephemeral metric data is safely sent to Prometheus.
- Prometheus continues pulling data from Pushgateway.
- Alertmanager is used to set the alert threshold for different indicators and send alerts via email or message.

 

### Metadata management

Milvus uses SQLite for metadata management by default. SQLite is implemented in Milvus and does not require configuration. In a production environment, it is recommended that you use MySQL for metadata management. 

 

### Engage with our open-source community:

- Find or contribute to Milvus on [GitHub](https://github.com/milvus-io/milvus/).
- Interact with the community via [Slack](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ).
- Connect with us on [Twitter](https://twitter.com/milvusio).

large_Milvus.jpeg

medium_Milvus.jpeg

small_Milvus.jpeg

thumbnail_Milvus.jpeg

Milvus.jpeg

	 Milvus v1.0 is available now. This stable, long-term support version powers image/video search, chatbots, and many more AI applications.

What's Inside Milvus 1.0?

Learn how Milvus is used to mitigate threats to critical data and strengthen cybersecurity with real-time virus detection.

Cybersecurity remains a persistent threat to both individuals and businesses, with data privacy concerns increasing for [86% of companies](https://www.getapp.com/resources/annual-data-security-report/) in 2020 and just [23% of consumers](https://www.firstdata.com/downloads/pdf/digital-commerce-cybersecurity-ebook.pdf) believing their personal data is very secure. As malware becomes steadily more omnipresent and sophisticated, a proactive approach to threat detection has become essential. [Trend Micro](https://www.trendmicro.com/en_us/business.html) is a global leader in hybrid cloud security, network defense, small business security, and endpoint security. To protect Android devices from viruses, the company built Trend Micro Mobile Security—a mobile app that compares APKs (Android Application Package) from the Google Play Store to a database of known malware. The virus detection system works as follows:

- External APKs (Android application package) from the Google Play Store are crawled.
- Known malware is converted into vectors and stored in [Milvus](https://www.milvus.io/docs/v1.0.0/overview.md).
- New APKs are also converted into vectors, then compared to the malware database using similarity search.
- If an APK vector is similar to any of the malware vectors, the app provides users with detailed information about the virus and its threat level.

To work, the system has to perform highly efficient similarity search on massive vector datasets in real time. Initially, Trend Micro used [MySQL](https://www.mysql.com/). However, as its business expanded so did the number of APKs with nefarious code stored in its database. The company’s algorithm team began searching for alternative vector similarity search solutions after quickly outgrowing MySQL.

 

### Comparing vector similarity search solutions

There are a number of vector similarity search solutions available, many of which are open source. Although the circumstances vary from project to project, most users benefit from leveraging a vector database built for unstructured data processing and analytics rather than a simple library that requires extensive configuration. Below we compare some popular vector similarity search solutions and explain why Trend Micro chose Milvus.

#### Faiss
[Faiss](https://ai.facebook.com/tools/faiss/) is a library developed by Facebook AI Research that enables efficient similarity search and clustering of dense vectors. The algorithms it contains search vectors of any size in sets. Faiss is written in C++ with wrappers for Python/numpy, and supports a number of indexes including IndexFlatL2, IndexFlatIP, HNSW, and IVF.

Although Faiss is an incredibly useful tool, it has limitations. It only works as a basic algorithm library, not a database for managing vector datasets. Additionally, it does not offer a distributed version, monitoring services, SDKs, or high availability, which are the key features of most cloud-based services. 

#### Plug-ins based on Faiss & other ANN search libraries
There are several plug-ins built on top of Faiss, NMSLIB, and other ANN search libraries that are designed to enhance the basic functionality of the underlying tool that powers them. Elasticsearch (ES) is a search engine based on the Lucene library with a number of such plugins. Below is an architecture diagram of an ES plug-in:

![1.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/1_3ce4e516c3.png)

###### *Architecture diagram of an Elasticsearch plug-in.*

Built in support for distributed systems is a major advantage of an ES solution. This saves developers time and companies money thanks to code that doesn’t have to be written. ES plug-ins are technically advanced and prevalent. Elasticsearch provides a QueryDSL (domain-specific language), which defines queries based on JSON and is easy to grasp. A full set of ES services makes it possible to conduct vector/text search and filter scalar data simultaneously.

Amazon, Alibaba, and Netease are a few large tech companies that currently rely on Elasticsearch plug-ins for vector similarity search. The primary downsides with this solution are high memory consumption and no support for performance tuning. In contrast, [JD.com](http://jd.com/) has developed its own distributed solution based on Faiss called [Vearch](https://github.com/vearch/vearch). However, Vearch is still an incubation-stage project and its open-source community is relatively inactive.

#### Milvus 
[Milvus](https://www.milvus.io/) is an open-source vector database created by [Zilliz](https://zilliz.com). It is highly flexible, reliable, and blazing fast. By encapsulating multiple widely adopted index libraries, such as Faiss, NMSLIB, and Annoy, Milvus provides a comprehensive set of intuitive APIs, allowing developers to choose the ideal index type for their scenario. It also provides distributed solutions and monitoring services. Milvus has a highly active open-source community and over 5.5K stars on [Github](https://github.com/milvus-io/milvus).

#### Milvus bests the competition
We compiled a number of different test results from the various vector similarity search solutions mentioned above. As we can see in the following comparison table, Milvus was significantly faster than the competition despite being tested on a dataset of 1 billion 128-dimensional vectors.

| **Engine** | **Performance (ms)** | **Dataset Size (million)** |
| :---- | :---- | :---- |
| ES | 600 | 1 |
| ES + Alibaba Cloud | 900 | 20 |
| Milvus | 27 | 1000+ |
| SPTAG | Not good | |
| ES + nmslib, faiss | 90 | 150 |

###### *A comparison of vector similarity search solutions.*

After weighing the pros and cons of each solution, Trend Micro settled on Milvus for its vector retrieval model. With exceptional performance on massive, billion-scale datasets, it's obvious why the company chose Milvus for a mobile security service that requires real-time vector similarity search.

 

### Designing a system for real-time virus detection
Trend Micro has more than 10 million malicious APKs stored in its MySQL database, with 100k new APKs added each day. The system works by extracting and calculating Thash values of different components of an APK file, then uses the Sha256 algorithm to transform it into binary files and generate 256-bit Sha256 values that differentiate the APK from others. Since Sha256 values vary with APK files, one APK can have one combined Thash value and one unique Sha256 value.

Sha256 values are only used to differentiate APKs, and Thash values are used for vector similarity retrieval. Similar APKs may have the same Thash values but different Sha256 values.

To detect APKs with nefarious code, Trend Micro developed its own system for retrieving similar Thash values and corresponding Sha256 values. Trend Micro chose Milvus to conduct instantaneous vector similarity search on massive vector datasets converted from Thash values. After similarity search is run, the corresponding Sha256 values are queried in MySQL. A Redis caching layer is also added to the architecture to map Thash values to Sha256 values, significantly reducing query time.

Below is the architecture diagram of Trend Micro’s mobile security system.

![image-20210118-022039.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/image_20210118_022039_ae824b663c.png)

###### *Architecture diagram for Trend Micro Mobile Security.*

 

Choosing an appropriate distance metric helps improve vector classification and clustering performance. The following table shows the [distance metrics](https://www.milvus.io/docs/v1.0.0/metric.md#binary) and the corresponding indexes that work with binary vectors. 

| **Distance Metrics** | **Index Types** |
| ---- | ---- |
| - Jaccard - Tanimoto - Hamming | - FLAT - IVF_FLAT |
| - Superstructure - Substructure | FLAT |

###### *Distance metrics and indexes for binary vectors.*

 

Trend Micro converts Thash values into binary vectors and stores them in Milvus. For this scenario, Trend Micro is using Hamming distance to compare vectors.

Milvus will soon support string vector ID, and integer IDs won’t have to be mapped to the corresponding name in string format. This makes the Redis caching layer unnecessary and the system architecture less bulky.

Trend Micro adopts a cloud-based solution and deploys many tasks on [Kubernetes](https://kubernetes.io/). To achieve high availability, Trend Micro uses [Mishards](https://www.milvus.io/docs/v1.0.0/mishards.md), a Milvus cluster sharding middleware developed in Python. 

![image-20210118-022104.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/image_20210118_022104_3001950ee8.png)

###### *Mishards architecture in Milvus.*

 

Trend Micro separates storage and distance calculation by storing all vectors in the [EFS](https://aws.amazon.com/efs/) (Elastic File System) provided by [AWS](https://aws.amazon.com/). This practice is a popular trend in the industry. Kubernetes is used to start multiple reading nodes, and develops LoadBalancer services on these reading nodes to ensure high availability.

To maintain data consistency Mishards supports just one writing node. However, a distributed version of Milvus with support for multiple writing nodes will be available in the coming months. 

 

### Monitoring and Alert Functions
Milvus is compatible with monitoring systems built on [Prometheus](https://prometheus.io/), and uses [Grafana](https://grafana.com/), an open-source platform for time-series analytics, to visualize various performance metrics.

Prometheus monitors and stores the following metrics:

- Milvus performance metrics including insertion speed, query speed, and Milvus uptime. 
- System performance metrics including CPU/GPU usage, network traffic, and disk access speed.
- Hardware storage metrics including data size and total file number.

The monitoring and alert system works as follows: 

- A Milvus client pushes customized metrics data to Pushgateway.
- The Pushgateway ensures short-lived, ephemeral metric data is safely sent to Prometheus.
- Prometheus keeps pulling data from Pushgateway. 
- Alertmanager sets the alert threshold for different metrics and raises alarms through emails or messages.

 

### System Performance
A couple months have passed since the ThashSearch service built on Milvus was first launched. The graph below shows that end-to-end query latency is less than 95 milliseconds.

![image-20210118-022116.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/image_20210118_022116_a0c735ce20.png)

###### *Query latency for Thash search service built on Milvus.*

 

Insertion is also fast. It takes around 10 seconds to insert 3 million 192-dimensional vectors. With help from Milvus, the system performance was able to meet the performance criteria set by Trend Micro. 

 

### Don’t be a stranger
- Find or contribute to Milvus on [GitHub](https://github.com/milvus-io/milvus/).
- Interact with the community via [Slack](https://join.slack.com/t/milvusio/shared_invite/zt-e0u4qu3k-bI2GDNys3ZqX1YCJ9OM~GQ).
- Connect with us on [Twitter](https://twitter.com/milvusio).

large_trend-blog-cover.png

medium_trend-blog-cover.png

small_trend-blog-cover.png

thumbnail_trend-blog-cover.png

trend-blog-cover.png

Making with Milvus: Detecting Android Viruses in Real Time for Trend Micro

Everything as a vector. That’s the future that we see for neural search. It’s already working remarkably well in image, natural language and sound, and research is constantly driving us closer to building a single embedding space for all modalities.


Those of you who follow our GitHub repository will know that there has been a lot of activity on [Haystack](https://github.com/deepset-ai/haystack) recently! Not only have we been working hard on a set of new features, but we’ve received a lot of great code contributions and had a lot of interesting and productive conversations with our community. The sum of all this interaction is [our latest release v0.8.0](https://github.com/deepset-ai/haystack/releases/tag/v0.8.0), which features more changes than we could list here. Nonetheless, we wanted to walk you through the most exciting new features that will help you build your own semantic search pipeline.

### Milvus

Everything as a vector. That’s the future that we see for neural search. It’s already working remarkably well in image, natural language and sound, and research is constantly driving us closer to building a single embedding space for all modalities.
With this vision comes the need for vector optimized databases that can perform indexing and similarity calculations fast and so we are very excited to be introducing the [MilvusDocumentStore](https://github.com/deepset-ai/haystack/blob/master/haystack/document_store/milvus.py) in Haystack. Milvus encapsulates multiple Approximate Nearest Neighbours (ANN) libraries such as FAISS and ANNOY but is also built with the robustness and reliability required in production systems. It allows for dynamic data management, it can also run as a separate service via Docker and we found their library very smooth to work with. And of course, they are open source!

A MilvusDocumentStore is the perfect complement to a DPRRetriever or an EmbeddingRetriever and in our experimental runs, we have already seen great performance from it. You can expect to see a blog from us soon with benchmarks of all our retrievers and document stores.

Thanks go to our Github contributor lalitpagaria who was the driving force behind [this new feature](https://github.com/deepset-ai/haystack/pull/771)!

 

### Knowledge Graph QA

Our ambition with Haystack is to build the most comprehensive and practical framework for anything in the field of search or information retrieval and as such, we could not ignore Knowledge Graphs. Many companies build large, well maintained knowledge graphs to accurately capture the interrelations between different concepts and entities in their domain. Following strong interest from our community, we challenged ourselves to build a system that could interpret natural language questions and fetch structured data from such a graph.

![Deepsetai-1.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Deepsetai_1_12f42ea866.png)

###### *From [Yashu Seth](https://yashuseth.blog/2019/10/08/introduction-question-answering-knowledge-graphs-kgqa/)*

 

In the latest release, you will see the first small steps we’ve made in this direction. Haystack now has a [GraphDBKnowledgeGraph](https://github.com/deepset-ai/haystack/blob/master/haystack/knowledge_graph/graphdb.py) class which stores your triples and executes SPARQL queries. There is also a [Text2SparqlRetriever](https://github.com/deepset-ai/haystack/blob/master/haystack/graph_retriever/text_to_sparql.py) class which connects to a GraphDBKnowledgeGraph and functions as a semantic parser. It turns natural language questions into SPARQL queries using a BART sequence-to-sequence model. Thanks must go to the [Transformers](https://github.com/huggingface/transformers) library maintained by Huggingface which has made it incredibly easy to integrate new transformers models into Haystack!

Knowledge graph QA systems have a key advantage over extractive QA systems in that they can handle questions that require counting or operations like taking the maximum or minimum. However, the extraction of relational triples and the training of the semantic parser is by no means straightforward. But when done right, we see a lot of potential in the ensembling of the two methods.

If you want to see how to work with Knowledge Graphs in Haystack, please check out [our end-to-end tutorial](https://github.com/deepset-ai/haystack/blob/master/tutorials/Tutorial10_Knowledge_Graph.ipynb)! This will give you a sense of the first small successes we’ve had with the technique and we would love to hear from you if you want to drive Knowledge Graph QA forward with us!

### Pipeline Configs
The flexibility of Pipelines in Haystack has proven to be very popular with our community and we ourselves have enjoyed experimenting with different configurations. For example, we created Pipelines where a BM25 retriever is ensembled with a DPR retriever to get the best of both worlds and it is now possible in Haystack to build retrieval systems that return not only the full document text, but also their summaries and translations.

Now, with the introduction of Pipeline config files, it is easier than ever to define, save and load your custom Pipeline. Rather than writing lines of code to tweak Pipelines, everything that configures a Pipeline can be written in a YAML file allowing for easier and quicker experimentation cycles and also simpler maintenance of production systems.

This is an example of what a YAML config might look like.

![deepsetai-2.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/deepsetai_2_4a04c0ad31.png)

These few lines are all you need to load that config and run a Pipeline.

![deepsetai-3.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/deepsetai_3_8d18335643.png)

These configs are here to lower the barriers to entry, and allow you to get creative with your Pipeline designs.

 

### Confidence Scores
Apart from seeing your model’s top predictions, it is often very helpful to know how confident the model is in its predictions. For example, you might be building a system where you expect 90% accuracy and would rather route a request elsewhere if the model is unsure. The Softmaxed scores coming from a Question Answering head do not necessarily capture the model’s confidence of an answer being correct and can create many problems in practice.

The latest release takes us a step in the right direction as each prediction is now accompanied by a `probability` score in the range of 0 to 1, with values near 0 meaning very low confidence in the prediction and values near 1 meaning very high confidence. Using our transfer learning framework FARM, you can even go one step further and tune this `probability` score to be aligned with your metrics so that you can take confidence=0.9 to mean that the model will be right 90% of the time. If this is something that you’re interested in, have a look at [this tutorial](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering_confidence.py).

There are many different approaches and complexities to calculating model confidence but you can expect further support and new tools to help you in tuning your system.

 

### Web Crawler

Finally, it is central to Haystack’s design that it connects easily to other components and data sources. For this reason, we want to highlight a very nice community contribution from DIVYA-19 that has added a [web crawler](https://github.com/deepset-ai/haystack/pull/775) to Haystack! Simply provide URLs of pages you’d like to search over and have the Crawler convert them into Haystack documents.

We anticipate that more use cases will pop up where there is a need to connect data from different sources, formats and streams. If you are working with a certain application, data source or platform that is missing a connection to Haystack, please feel free to bring this to our attention in our GitHub repository or [through Slack](https://haystack.deepset.ai/community/join)! Haystack is designed to be an end-to-end search system but it is also our goal to make sure it integrates seamlessly into your tech stack.

 

### Conclusion

This latest release is packed full of new features and improvements, so much so that we didn’t even find space in this blog to talk about Evaluation Nodes or SQuAD to DPR data scripts. Haystack v0.8.0 is available now and you can see a full list of changes in [our release notes](https://github.com/deepset-ai/haystack/releases/tag/v0.8.0).

 

This blog article is reposted from: https://medium.com/deepset-ai/semantic-search-with-milvus-knowledge-graph-qa-web-crawlers-and-more-837451eae9fa

small_Deepsetai-1.png

thumbnail_Deepsetai-1.png

Deepsetai-1.png

Build a fully featured pipeline using the latest features from Haystack v0.8.0

Semantic Search with Milvus, Knowledge Graph QA, Web Crawlers and more!

Learn more about using semantic machine learning methodologies to power more relevant search results across your organization.

[Semantic search](https://lucidworks.com/post/what-is-semantic-search/) is a great tool to help your customers—or your employees—find the right products or information. It can even surface difficult-to-index information for better results. That said, if your semantic methodologies aren’t being deployed to work fast, they won’t do you any good. The customer or employee isn’t just going to sit around while the system takes its time responding to their query—and a thousand others are likely being ingested at the same time.

How can you make semantic search fast? Slow semantic search isn’t going to cut it.

Fortunately, this is the kind of problem Lucidworks loves to solve. We recently tested a modest-sized cluster—read on for more details—that resulted in 1500 RPS (requests per second) against a collection of over one million documents, with an average response time of roughly 40 milliseconds. Now that’s some serious speed.

 

### Implementing Semantic Search
To make lightning-fast, machine learning magic happen, Lucidworks has implemented semantic search using the semantic vector search approach. There are two critical parts.

 

#### Part One: The Machine Learning Model

First, you need a way to encode text into a numerical vector. The text could be a product description, a user search query, a question, or even an answer to a question. A semantic search model is trained to encode text such that text that is semantically similar to other text is encoded into vectors that are numerically “close” to one another. This encoding step needs to be fast in order to support the thousand or more possible customer searches or user queries coming in every second.

 

#### Part Two: The Vector Search Engine
Second, you need a way to quickly find the best matches to the customer search or user query. The model will have encoded that text into a numerical vector. From there, you need to compare that to all the numerical vectors in your catalog or lists of questions and answers to find those best matches—the vectors that are “closest” to the query vector. For that, you will need a vector engine that can handle all of that information effectively and at lightning speed. The engine could contain millions of vectors and you really just want the best twenty or so matches to your query. And of course, it needs to handle a thousand or so such queries every second.

To tackle these challenges, we added the vector search engine [Milvus](https://doc.lucidworks.com/fusion/5.3/8821/milvus) in our [Fusion 5.3 release](https://lucidworks.com/post/enhance-personalization-efforts-with-new-features-in-fusion/). Milvus is open-source software and it is fast. Milvus uses FAISS ([Facebook AI Similarity Search](https://ai.facebook.com/tools/faiss/)), the same technology Facebook uses in production for its own machine learning initiatives. When needed, it can run even faster on [GPU](https://en.wikipedia.org/wiki/Graphics_processing_unit). When Fusion 5.3 (or higher) is installed with the machine learning component, Milvus is automatically installed as part of that component so you can turn on all of these capabilities with ease.

The size of the vectors in a given collection, specified when the collection is created, depends on the model that produces those vectors. For example, a given collection could store the vectors created from encoding (via a model) all of the product descriptions in a product catalog. Without a vector search engine like Milvus, similarity searches would not be feasible across the entire vector space. So, similarity searches would have to be limited to pre-selected candidates from the vector space (for example, 500) and would have both slower performance and lower quality results. Milvus can store hundreds of billions of vectors across multiple collections of vectors to ensure that search is fast and results are relevant.

 

### Using Semantic Search

Let’s get back to the semantic search workflow, now that we’ve learned a little about why Milvus might be so important. Semantic search has three stages. During the first stage, the machine learning model is loaded and/or trained. Afterwards, data is indexed into Milvus and Solr. The final stage is the query stage, when the actual search occurs. We’ll focus on those last two stages below.

 

### Indexing into Milvus
![Lucidworks-1.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_1_47a9221723.png)

As shown in the above diagram, the query stage begins similarly to the indexing stage, just with queries coming in instead of documents. For each query:

1. The query is sent to the [Smart Answers](https://lucidworks.com/products/smart-answers/) index pipeline.
2. The query is then sent to the ML model.
3. The ML model returns a numeric vector (encrypted from the query). Again, the type of model determines the size of the vector.
4. The vector is sent to Milvus, which then determines which vectors, in the specified Milvus collection, best match the provided vector.
5. Milvus returns a list of unique IDs and distances corresponding to the vectors determined in step four.
6. A query containing those IDs and distances is sent to Solr.
7. Solr then returns an ordered list of the documents associated with those IDs.

 

### Scale Testing
In order to prove that our semantic search flows are running at the efficiency we require for our customers, we run scale tests using Gatling scripts on the Google Cloud Platform using a Fusion cluster with eight replicas of the ML model, eight replicas of the query service, and a single instance of Milvus. Tests were run using the Milvus FLAT and HNSW indexes. The FLAT index has 100% recall, but is less efficient – except when the datasets are small. The HNSW (Hierarchical Small World Graph) index still has high quality results and it has improved performance on larger datasets.

Let’s jump into some numbers from a recent example we ran:

![Lucidworks-2.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_2_3162113560.png)

![Lucidworks-3.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_3_3dc17f0ed8.png)

![Lucidworks-4.png](https://zillizstorage.blob.core.windows.net/zilliz-assets/zilliz-assets/assets/Lucidworks_4_8a6edd2f59.png)

 

### Getting Started

The [Smart Answers](https://lucidworks.com/products/smart-answers/) pipelines are designed to be easy-to-use. Lucidworks has [pre-trained models that are easy-to-deploy](https://doc.lucidworks.com/how-to/734/set-up-a-pre-trained-cold-start-model-for-smart-answers) and generally have good results—though training your own models, in tandem with pre-trained models, will offer the best results. Contact us today to learn how you can implement these initiatives into your search tools to power more effective and delightful results.


> This blog is reposted from: https://lucidworks.com/post/how-to-build-fast-semantic-search/?utm_campaign=Oktopost-Blog+Posts&utm_medium=organic_social&utm_source=linkedin

Products

Resources

Open Source

About

Contribute

Blog

All

Product

Company

Technology

Case Study

Release

Load More