Apple researchers propose BayesCNS: a unified Bayesian approach that addresses cold starts and non-stationarity in large-scale search systems

Information retrieval (IR) systems for search and recommendations often use learning-to-rank (LTR) solutions to prioritize items relevant to user queries. These models rely heavily on user interaction features such as clicks and engagement data, which are very effective for ranking. However, this dependency presents significant challenges. User interaction data can be noisy and sparse, especially for newer or less popular items, leading to cold start issues where these items are ranked poorly and receive no attention. Scanning for item recommendations may solve cold start issues, but it negatively impacts key business metrics and user trust.

Existing methods to address cold start in recommender systems rely on heuristics to improve item ranking or use additional information to compensate for the lack of interaction data. Non-stationary distribution changes are then handled by periodic retraining of the model, which is costly and unstable due to varying data quality. Finally, there is Bayesian modeling which offers a principled approach to handling the dynamic nature of user interaction features, allowing for real-time updates as new data is observed. However, Bayesian methods require a large amount of computation, since exact estimation of the posterior distribution is intractable. Furthermore, recent advances in variational inference that use neural networks to simultaneously address cold start and nonstationarity in scaled recommender systems remain unexplored.

To this end, researchers at Apple have proposed BayesCNS, a unified Bayesian approach that comprehensively addresses the challenges of cold start and non-stationarity in search systems at scale. The method is formulated as a Bayesian online learning problem, using an empirical Bayesian framework to learn expressive prior distributions of user-item interactions based on contextual features. The approach interacts with a classification model, providing online learning guided by classifications to explore relevant elements based on contextual information efficiently. The effectiveness of BayesCNS in comprehensive online and offline experiments, including an A/B test, shows an overall improvement of 10.60% in overall interactions with new elements and a 1.05% increase in success rate overall compared to baseline.

BayesCNS uses a Thompson sampling algorithm for online learning under non-stationary conditions, allowing continuous updates of previous estimates and learning from new data to maximize cumulative reward. BayesCNS is evaluated on three diverse benchmark datasets that address cold starts in recommender systems: CiteULike, LastFM, and XING. These data sets cover users' preferences for scientific articles, music artists, and job recommendations, respectively. For comparison, five state-of-the-art cold start recommendation algorithms are KNN, LinMap, NLinMap, DropoutNet, and Heater. These algorithms use different techniques such as nearest neighbor algorithms, linear transformations, deep neural networks, dropout methods, and a combination of experts to generate recommendations and solve cold start problems.

The performance of BayesCNS is evaluated using metrics such as Recall@k, Precision@k, and NDCG@k for k values ​​of 20, 50, and 100. The results show that BayesCNS had competitive performance compared to other state-of-the-art methods. in all data sets. An online A/B test introduces millions of new items, representing 22.81% of the original item index size. The test was conducted for one month, comparing BayesCNS to a baseline that introduced new elements without considering cold start or non-stationary effects. BayesCNS consistently outperformed the baseline, showing statistically significant improvements in hit rate and new item surfacing rate across most cohorts.

In conclusion, Apple researchers have introduced BayesCNS, a Bayesian online learning approach that effectively addresses the challenges of cold start and non-stationarity in large-scale search systems. This method predicts prior distributions of user-item interaction using contextual features of items, using a novel deep neural network parameterization to learn expressive priors while allowing for efficient subsequent updates. The effectiveness of BayesCNS has been demonstrated through a comprehensive evaluation showing significant improvements in critical metrics such as click-through rates, new article impression rates, and overall user success metrics. These findings utilize the potential of BayesCNS to improve the performance of search and recommendation systems in dynamic real-world environments.

look at the Paper. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter and join our Telegram channel and LinkedIn Grabove. If you like our work, you will love our information sheet.. Don't forget to join our SubReddit over 50,000ml

(Next Event: Oct 17, 202) RetrieveX – The GenAI Data Recovery Conference (Promoted)

Sajjad Ansari is a final year student of IIT Kharagpur. As a technology enthusiast, he delves into the practical applications of ai with a focus on understanding the impact of ai technologies and their real-world implications. Its goal is to articulate complex ai concepts in a clear and accessible way.