What is FiftyOne
FiftyOne is an open-source solution for computer vision model development. As a machine learning toolkit, FiftyOne empowers data science teams to elevate the efficacy of their computer vision models through various functionalities. These include the ability to curate top-tier datasets meticulously, conduct thorough model evaluations, pinpoint errors, visualize embeddings, and streamline the path to production.
Why FiftyOne and Zilliz Cloud (Milvus)
Vector search engines, such as Milvus, streamline the process of storing, indexing, and searching vector embeddings for the growing number of AI applications benign built every day.
For those integrating vector search into image or video workflows, challenges like implementing cross-modal retrieval or incorporating traditional search filters can arise. FiftyOne offers a solution as the computer vision interface for vector search. With native integrations with Milvus, FiftyOne allows users to seamlessly leverage their preferred vector search engine for efficient visual data search with just a single line of code. Whether it's finding similar images or searching for specific objects within images, FiftyOne simplifies the process, enabling users to achieve their search objectives effortlessly.
When integrating vector search into image or video data workflows, several challenges may arise:
- Implementing cross-modal retrieval, such as searching for images based on associated text.
- Incorporating traditional search filters like confidence thresholds or class labels.
- Searching over specific objects within images, such as people, cats, dogs, cars, bikes, and more.
How to use FiftyOne and Zilliz Cloud
To use Milvus with FiftyOne, to create a similarity index on your FiftyOne datasets and use this to query your data is as follows:
- Load your dataset into FiftyOne
- Specify the model you want to use to generate the embedding vectors, and configure whether you want to use Milvus or Zilliz Cloud to store these vector embeddings
- Use the
compute_similarity()method to generate a Milvus similarity index for the samples or object patches in a dataset by setting the parameter backend="milvus" and specifying a brain_key of your choice - Use this Milvus similarity index to query your data with
sort_by_similarity()
Setup Milvus
## setup milvus ## download and start docker container + pip install pymilvus ## generate a similarity index ## with CLIP model embeddings ## using Milvus backend fob.compute_similarity( dataset, brain_key="milvus_clip_index", backend="milvus", metric="dotproduct"Query Milvus
## get ID of first sample query = dataset.skip(1).first().id ## find 25 most similar images with Milvus backend sim_view = dataset.sort_by_similarity( query, k=25, brain_key="milvus_clip_index" ) ## display results session = fo.launch_app(sim_view)Learn More
Check out these tutorials to learn how to use the FiftyOne and Milvus / Zilliz Cloud.