Opera’s Intelligent Video Deduplication System
Opera is a Norwegian software company that creates some of the world's most popular web browsers, Opera News, and other services. The company is a pioneer in leveraging artificial intelligence to make digital content recommendations. To ensure users are served a wide variety of videos, and to avoid recommending the same content multiple times, Opera built a smart video reduplication system inside its video content recommendation platform.
Milvus, an open-source vector database built by Zilliz, is used to power Opera’s smart video deduplication system.
Objective
Build a video content deduplication system that can remove similar contents from video recommendations and ensure users discover a wide variety of unique content.
Challenges
- Even at a low extraction rate, video content could generate enough key frames to crash the computational system.
Why Milvus
- By optimizing the existing high-performance computing (HPC) frameworks, Milvus supports and accelerates most computation-demanding application scenarios.
- Milvus is created with Software 2.0 and MLOps in mind, and is compatible with popular machine learning frameworks.
Results
- A video deduplication system was built at minimal cost and without burdening the underlying system.
- Duplicate content was removed making more videos discoverable. This improved user experience by providing more varied and considerate recommendations.
"The Milvus community is very active and has an abundance of resources available for its users. I drew much inspiration from Milvus' demos and bootcamp. I didn’t have to start from scratch, but rather built the deduplication system based on the reverse image search demo the community provides. This helped us quickly developed a minimum viable product (MVP) and significantly lowered the development costs."
