How SmartNews Transforms Real-Time Ad Recommendations with Milvus
< 10ms
P99 latency to enable real-time data retrieval
100+ Million
Daily vector updates without performance degradation
Increase revenue
Serving more relevant ads led to improved click-through rate
We've gained so much from the Milvus community that we decided to contribute features like "hot reload," which have also benefited our internal operations.
Dennis Zhao
About SmartNews
SmartNews, headquartered in Tokyo, Japan, has been a dominant force in the news app sector since its inception in 2012. In the U.S., it's particularly noted for its high user engagement. Dennis Zhao, AI Infrastructure Lead at SmartNews, elaborates, "As of July 2021, SmartNews had the highest average monthly user engagement among all news apps, even surpassing the combined metrics of AppleNews and Google News." This high level of user engagement naturally led to increased opportunities for ad placements. Still, it also posed challenges in ensuring the ads reached the right users at the right time.
The Technical Hurdles of Ad Recommendation Algorithms
With its leadership position in user engagement, SmartNews had a unique challenge: optimizing ad recommendations to a highly engaged user base. Shu, an AI infrastructure engineer at SmartNews, details these challenges: "We were wrestling with high-dimensional sparse features and the need for rapid unstructured data retrieval in our large-scale machine learning systems."
These challenges were not just theoretical; they had real-world implications. The high-dimensional sparse features meant traditional database solutions were inefficient for storing and querying the data. The unstructured data made it difficult to perform quick and accurate searches, a critical requirement for real-time ad recommendations.
Before turning to Milvus, the team had been using Faiss, a library for similarity search and clustering of dense vectors. However, Faiss came with its own set of limitations. "It was cumbersome to maintain and didn't scale well with our growing machine learning applications," Shu adds. These limitations led the team to explore other solutions that could offer both the scalability and performance they needed, eventually leading them to Milvus.
Technical Solutions: From Vector Search to Community Contributions
When SmartNews realized the limitations of its existing solutions, the team turned to Milvus for its capabilities in handling large-scale vector data and similarity search. "After two months of rigorous research, Milvus emerged as the clear winner," says Shu. Milvus's ability to handle high-throughput and low-latency queries was a game-changer. "It met our strict requirement of P99 latency < 10ms, crucial for our real-time ad recommendation system."
Milvus's vector similarity search capabilities were initially applied to optimize SmartNews's dynamic ad vector recall. "Milvus allowed us to map incomparable data types into vectors, thereby enabling efficient similarity search to link users with the most relevant ads," explains Shu. This performant search capability was significant for SmartNews, as the platform needed to sift through millions of vectors to make real-time ad recommendations.
Beyond ad recommendations, Milvus proved versatile enough for other machine-learning applications. "We expanded its use to real-time data and index updates, which was vital for keeping our recommendation models current," Shu adds.
SmartNews also recognized the value of community-driven development. "We've gained so much from the Milvus community that we decided to contribute features like 'hot reload,' which have also been beneficial for our internal operations," says Dennis Zhao, AI Infrastructure Lead at SmartNews. This feature allows developers to change code on the fly, significantly speeding up the development cycle.
Business Results: Achieving Scalability and Preparing for the Future
The adoption of Milvus wasn't just a technical upgrade; it had significant business implications. Yusup, another engineer at SmartNews, discusses one of the initial challenges: "The original mechanism for updating vectors was cumbersome and time-consuming. However, Milvus's 'collection aliasing' feature allowed us to make smoother transitions between old and new data sets, streamlining the update process."
This technical improvement translated into business results. The optimized ad recommendation system led to more relevant ads being served, increasing click-through rates and ultimately driving up ad revenue. Moreover, the real-time data and index updates enabled SmartNews to adapt quickly to changing user behavior, keeping the platform's recommendations consistently relevant.
SmartNews is preparing for the next phase of its technological evolution. "With the upcoming release of Milvus 2.0, we're planning a migration," says Dennis Zhao. "The new version promises even better performance and scalability, and we're excited about leveraging these features to build even more real-time and reliable systems."