Information retrieval (IR) systems used in search and recommendation platforms frequently employ learning-to-rank (LTR) models to rank items in response to user queries. These models rely heavily on features derived from user interactions, such as clicks and engagement data. This dependency introduces cold start problems for elements that lack user input and poses challenges in adapting to non-stationary changes in user behavior over time. We address both challenges comprehensively as an online learning problem and propose BayesCNS, a Bayesian approach designed to handle cold start and non-stationary distribution changes in search systems at scale. BayesCNS achieves this by estimating prior distributions for user-item interactions, which are continually updated with new user interactions collected online. This online learning procedure is guided by a classification model, which allows efficient exploration of relevant elements using contextual information provided by the classifier. We successfully implemented BayesCNS in a large-scale search system and demonstrated its effectiveness through comprehensive online and offline experiments. In particular, an online A/B experiment showed a 10.60% increase in interactions with new items and a 1.05% improvement in overall success metrics over the existing production baseline.
