When I started out as a data scientist, I expected to use cutting-edge models. XGBoost, Neural Networks. These things are complex and interesting and would surely drive improvements. Little did I know, the models faced an obstacle: explaining it to other people.
Who would have thought that you need to understand the decisions your automated systems make?
To my delight, I stumbled down the rabbit hole of model-agnostic methods. With these, you could have the best of both worlds. You could train black box models and then explain them using methods like SHAP, LIME, PDP, ALE, and Friedman's H-stat. We no longer need to trade precision for interpretability!
Not so fast. That thinking is wrong.
In our quest for the best performance, we often miss the goal of machine learning: that is, making accurate predictions on new, unseen data. Let's discuss why complex models are not always the best way to achieve this. Even if we can explain them using other methods.
