Shaping Tomorrow: How Milvus Powers Shopee's Multimedia Ambition
In the world of e-commerce, staying ahead of the game requires more than just a user-friendly interface and a vast product range. Shopee, a leading platform in Southeast Asia and Latin America, understood the evolving landscape and embarked on a groundbreaking journey to change its Multimedia Understanding (MMU) business. To compete with short video giants like TikTok and safeguard its e-commerce market share, Shopee ventured into the world of short video services. In this blog, we’ll delve into the challenges facing Shoppee during this process and how how Milvus helped Shopee’s multimedia business.
Shopee’s data dilemma
As Shopee delved into the multimedia world, it faced a significant challenge – handling vast amounts of unstructured data, including videos, images, audio, and text. The relational databases Shopee used previously struggled with the complexity of all the unstructured data, prompting the team to explore a robust tech stack that could efficiently store, process, search, and make the best use of such data. vector database are one of the most important parts of their exploration.
Adding to the complexity were Shopee's internal systems – video recall, video deduplication, and video recommendations – each crafted with different technologies and relying heavily on vector search capabilities. Shopee needed a solution seamlessly integrated with these systems and various technological stacks.
Milvus emerges as the perfect fit
After thoroughly exploring available options, Milvus stood out for its ability to handle billions of vectors, scalability, and seamless integration with Shopee's internal ecosystem. Milvus offered a cloud-native architecture, making it easy to set up vector retrieval systems from scratch.
The Shopee team also loved Milvus’s rich feature offerings, including distributed processing, GPU support, incremental updates, and scalar support, making it an ideal choice for building a highly scalable and performant vector search engine.
The migration of Milvus 1.x to Milvus 2.x
Shopee's journey with Milvus saw a migration from version 1.x to 2.x. Initially efficient, the 1.x version faced challenges as Shopee's business scaled. Latency issues surfaced due to uneven segment distribution among read-only nodes. The migration to Milvus 2.x marked a turning point.
Enhanced stability, scalability, and multi-replica capabilities in Milvus 2.x versions resulted in low-latency and high-availability retrieval services. The cloud-native architecture of Milvus 2.x versions introduced cost-effective logging and monitoring features, streamlining Shopee's operations.
The use of Milvus has also elevated Shopee's real-time search capabilities to unprecedented levels. A standout illustration of this enhancement is evident in the video recall system. Milvus seamlessly integrates instant video recall into Shopee's video recommendation systems, contributing to a better user experience for millions worldwide. Furthermore, Milvus has significantly streamlined offline data retrieval, a critical aspect for copyright video matching and video deduplication processes. Its role in recognizing original content and identifying duplicate videos ensures the content retains its freshness and authenticity, ultimately enhancing user satisfaction.
Now, Shopee is using Milvus 2.2 in production and is considering upgrading to the latest versions for enhanced performance and richer features such as Mmap, GPU indexing, and range search.
Shaping the future of e-commerce with Milvus
Equipped with Milvus, Shopee envisions a future where multimedia understanding seamlessly integrates with user experience, paving new paths in e-commerce. As Shopee continues to evolve, Milvus remains a close partner, ready to meet increasingly sophisticated AI demands.
Fendy FengFendy Feng is the Technical Marketing Writer at Zilliz. She has extensive experience developing and enhancing the impact of open-source projects in various global markets by producing high-quality, tailored content. Before joining Zilliz, Fendy worked as a Content Strategist at PingCAP, a fast-growing E-Series startup renowned for its open-source distributed SQL database.
- Shopee’s data dilemma
- Milvus emerges as the perfect fit
- The migration of Milvus 1.x to Milvus 2.x
- Shaping the future of e-commerce with Milvus
Content
Start Free, Scale Easily
Try the fully-managed vector database built for your GenAI applications.
Try Zilliz Cloud for Free