Blog
Unlocking Next-Level APK Security: Trend Micro's Journey with Milvus

Unlocking Next-Level APK Security: Trend Micro's Journey with Milvus

Dec 21, 20234 min read

Trend Micro has long been a trailblazer in cybersecurity, consistently delivering cutting-edge solutions to address evolving threats. With a dedicated focus on mobile security, Trend Micro has embarked on a journey to enhance APK (Android application package) security. At the heart of this quest for efficiency lies the integration of Milvus, a robust open-source vector database, into their security infrastructure.

In this blog, we'll uncover the challenges Trend Micro faced and how exactly Milvus came to the rescue.

Trend Micro’s mission to safeguard the Android ecosystem

As our digitally connected world increasingly relies on mobile applications, securing the Android ecosystem has become paramount. Trend Micro, a global leader in security software, recognized this challenge and assembled a specialized team tasked with developing mobile security apps and the underlying infrastructure. Their mission is to detect and neutralize APKs carrying viruses by employing Trend Micro's proprietary algorithms.

The rise and fall of database solutions: MySQL and Faiss

Trend Micro turned to MySQL for APK similarity search in this project's nascent stages. This relational database served them well initially, allowing for SQL queries and efficient handling of the growing dataset. However, MySQL faced scalability issues as the dataset swelled into the tens of millions, with daily increments reaching hundreds of thousands. Query latency soared, creating bottlenecks that hindered the system's responsiveness.

To overcome these challenges, Trend Micro shifted its focus to Faiss, a specialized library for similarity searches released by Facebook. While Faiss excelled in speed, it lacked the critical features required for a production environment. The absence of data management capabilities and high availability features made it unsuitable for the mission-critical task of APK security. Faiss's non-distributed nature further impeded horizontal scalability, compelling the team to seek a more robust solution.

Milvus: a comprehensive vector search engine

Trend Micro found themselves at a crossroads when the limitations of MySQL and Faiss became apparent. Their quest for a robust and scalable solution led them to the game-changing Milvus – a comprehensive vector search engine crafted in C++, addressing the pitfalls of Trend Micro’s previous approaches.

Evaluated Solution Comparison Summary

Seamless Integration with leading index libraries

Milvus distinguished itself with a standout feature: seamless integration with mainstream vector index libraries, including Faiss, NMSLIB, and Annoy. This integration became a strategic advantage for Trend Micro, allowing them to harness Faiss's speed while capitalizing on the additional features offered by Milvus.

Simple and intuitive APIs and horizontal scalability

Milvus offers simple and intuitive APIs, giving the development team unprecedented flexibility in choosing indexing types based on specific use cases.

Unlike Faiss, Milvus embraced a distributed architecture, allowing seamless horizontal scaling without an upper limit. As Trend Micro's dataset grew exponentially, Milvus's design ensured they could effortlessly add more nodes to handle increased loads, future-proofing their APK analysis infrastructure.

Elevated monitoring capabilities

Monitoring, a critical aspect of any mission-critical application, was another arena where Milvus stood out. Equipped with built-in support for Prometheus – a leading open-source monitoring tool– Milvus enabled Trend Micro to track performance metrics closely. From query latency to data import speeds, this monitoring capability empowered proactive issue resolution, ensuring a resilient APK security system.

As the digital landscape continues to evolve, Trend Micro's journey with Milvus underscores the importance of adopting innovative solutions. Milvus addressed existing challenges and propelled Trend Micro into a future-ready state, equipped to combat emerging threats in mobile security.

Realizing tangible impact: low latency and high data import speed

The implementation of Milvus in Trend Micro's ThashSearch service has translated into tangible results. The ThashSearch service, which has been live for several months, consistently achieves an average query latency of under 95 milliseconds. This low latency ensures that Trend Micro promptly alerts corporate and individual users about potentially harmful APKs, significantly enhancing overall security posture.

The dashboard shows that the average latency of end-to-end queries is within 95 ms

But it's not just about low latency; in the data-intensive world of APK analysis, the speed of new data ingestion is equally critical. Milvus has showcased impressive data import speed, capable of ingesting 3 million 192-dimensional vector data in approximately 10 seconds. This rapid data import capability is pivotal for Trend Micro, ensuring the database is always up-to-date and facilitating the most accurate and current similarity searches.

Looking to the future

As Trend Micro looks toward the future, they are keenly interested in Milvus's roadmap and how its upcoming features can further optimize APK analysis operations. The introduction of string-type IDs in Milvus holds promise, with Trend Micro planning to streamline its architecture by eliminating the need for Redis caching.

Moreover, Trend Micro is excited about Milvus's plans to evolve into a fully distributed system. Currently, Milvus supports only one write node, but future versions aim to remove this limitation. TrendMicro sees this as an opportunity to scale its operations horizontally, enhancing its ability to handle even larger datasets and higher query volumes. Buoyed by the success of the ThashSearch service, Trend Micro is considering expanding the footprint of Milvus within their organization, exploring integration into other projects and workflows that could benefit from efficient vector similarity search.

Fendy Feng
Fendy Feng is the Technical Marketing Writer at Zilliz. She has extensive experience developing and enhancing the impact of open-source projects in various global markets by producing high-quality, tailored content. Before joining Zilliz, Fendy worked as a Content Strategist at PingCAP, a fast-growing E-Series startup renowned for its open-source distributed SQL database.