A deep dive into the latest technology used to bypass classification models
Recommender systems are among the most ubiquitous machine learning applications in the world today. However, the underlying classification models are plagued by numerous biases that can severely limit the quality of the resulting recommendations. The problem of creating unbiased classifiers, also known as unbiased classification learning, ULTR, remains one of the most important research problems within machine learning and is still far from being solved.
In this post, we will delve into a particular modeling approach that relatively recently has allowed the industry to control bias very effectively and thus build far superior recommender systems: the two-tower model, where one tower learns relevance and another learn relevance. The (shallow) rook learns biases.
While dual-tower models have likely been used in the industry for several years, the first document that formally introduced them to the broader ML community was Huawei’s 2019 PAL document.
PAL (Huawei, 2019): the OG two-tower model
Huawei’s role COMRADE (“Learning to rank based on position”) considers the problem of position bias within the context of the Huawei app store.
Position bias has been observed time and time again in industry-wide ranking models. It simply means that users are more likely to click on the items that are displayed first. This may be because they are in a hurry, blindly trusting the ranking algorithm, or other reasons. Here is a graph demonstrating the position bias in Huawei data:
Position bias in the Huawei app store. Items in higher positions get more clicks than those in lower positions.
Position bias is a problem because we simply can’t know whether users clicked on the first item because it was in fact the most relevant to them or because it was displayed first; and in recommender systems our goal is to solve the first learning objective, not the first. last.
