*= Equal taxpayers
Recommender systems in large-scale online marketplaces are essential to help users discover new content. However, state-of-the-art systems for item-to-item recommendation tasks often rely on a superficial level of contextual relevance, which may make the system insufficient for tasks where relationships between items are more nuanced. Contextually relevant item pairs can sometimes have problematic relationships that are confusing or even controversial to end users, and could degrade user experiences and brand perception when recommended to users. For example, recommending a book about a sports team to someone who reads a book about that team’s biggest rival could be a bad experience, despite supposed similarities between the books. In this paper, we propose a classifier to identify and prevent problematic item-by-item recommendations and improve overall user experiences. The proposed approach uses active learning to efficiently sample concrete examples into sensitive item categories and employs human evaluators to label the data. We also conducted offline experiments to demonstrate the effectiveness of this system in identifying and filtering problematic recommendations while maintaining recommendation quality.
