Why making your model fail is the best thing to do
When developing a new model or algorithm, it is tempting to test it again and again with similar examples that work perfectly. While this can be fun, it doesn’t really help you understand and improve your model. You learn from mistakes, so make your model fail!
Imagine that your data science teammate comes up to you and tells you about the new model you just trained. It is so amazing that you can classify all kinds of baby animal images. Of course, your teammate will start to show you how well it works by allowing the model to correctly classify a bunch of images. Right now the best thing you can ask of your partner is the following:
Show me some cases where it fails
At first, this may seem counterintuitive. Sure, it’s fun to watch the model work and you don’t want to demotivate your teammate by making the model fail, but what gives you more information about how the model behaves: seeing the model work or seeing it fail?
If you ranked N images of super cute kittens correctly, chances are you’ll also rank image N+1 correctly, if it looks similar to the others. Are you surprised? No. Did you learn anything about the behavior of the model? No.
However, if you find the images that are causing the model to fail, you might finally have an idea of which images the model is still having trouble with. That’s worth a lot! Now you can start to understand the model and improve it further.
The theoretical background on fault finding has a long tradition. For hundreds of years, very intelligent people have thought and debated the question of how we can derive common rules from observations. The sun has risen this morning, and it also did yesterday and the day before yesterday. Does that mean he’ll be up again tomorrow? Well, not necessarily. Another example: I went to the park and all the swans I saw there were white, so I could formulate my hypothesis
All swans are white.
As a good scientist, I will test my hypothesis of course, so I go to the next park and look at the swans. They are white too. So, is my hypothesis proven? No, because if I wanted to try it, I would have to check all swans Well, I don’t have time for that, so what should I do? After I have seen N white swans, looking at the N+1 white swan will give me no additional information. It should rather:
Try to find a black swan.
Why is that? Wouldn’t that falsify my hypothesis? Yes, that is exactly what I would do. And that is what I should aspire to.
Formally speaking, if I look at N white swans and deduce that all swans are white, I make a logic induction. Logical induction, however, has a drawback: it is wrong.
The previous statement is an induction and we could read it as
My hypothesis H implies an observation B, and I observed this observation B. That implies that my hypothesis H is true.
However, the statement, as a whole, is incorrect. In our example, the declaration would be:
My hypothesis ‘all swans are white’ (H) implies that the next swan I see is white (B). The next swan I see is certainly white (B). That implies that my hypothesis is true.
Once again, that is not true. While the observation of a white swan is consistent with the hypothesis, it does not imply its truth. He’s just not speaking against the hypothesis. If you are not convinced, take the following example:
My hypothesis ‘all swans are painted white by a secret US government organization’ implies that the next swan I see is white (B). The next swan I see is certainly white (B). That implies that my hypothesis is true.
It’s just not true.
However, there is still hope. While the above statement was incorrect, the following is true:
We can read it as
My hypothesis H implies an observation B, and I did not observe B. This implies that my hypothesis H is not true.
or phrase your example like
My hypothesis ‘all swans are white’ (H) implies that the next swan I see is white (B). The next swan I see is not white (¬B). That means that my hypothesis is not true (¬H).
That’s a true statement (and for the formal logic nerds among you, it’s called a delete mode). If I find an example that speaks against the hypothesis, the hypothesis is shown to be incorrect, and is in fact new information that I didn’t have before.
Taken together, that means that hypotheses can never be proven or verified. They can simply be faked. On the other hand, if a hypothesis survives many of my attempts to falsify it, that speaks for the hypothesis.
So how does all of that relate to your teammate’s model? You don’t want to prove or falsify a simple hypothesis here, but the main idea was that information gain comes from cases that go wrong. In short, the cases in which the model works, don’t tell you anything you didn’t already know. To understand the behavior of the model, look at those cases where the model fails. For example:
An entity recognizer that recognizes names can detect the names ‘Alice’ and ‘Paula’.
- Can you also spot ‘Kathrin’? Yes. Did you learn anything from that? No.
- Can you also spot ‘Linh’? No. → You may have problems with Asian names.
An image classifier can detect the location where landscape images have been taken. Correctly detects that the photos of your last vacation were taken on a beach in Indonesia.
- Can you also detect the location of the photos you took last year in India? Yes. Did you learn anything from that? No.
- Does it also work for the photographs your grandfather took on his first trip to Italy in the 50s? No. → The data may lose old black and white photos.
A cool new robotic hand is so flexible and can be controlled in such fine detail that it can play a C major scale on a piano.
- Can you also play an F major scale? Yes. Did you learn anything from that? No.
- Can you also play a Mozart sonata? No. → Maybe her flexibility is still limited and a Mozart sonata is too difficult for her.
All those examples show how knowledge is gained from cases that fail, not from those that work. In the same way, you can learn more about your teammate’s model that classifies cute animals:
- Can you also classify rare animals like baby octopuses or tree kangaroos?
- Does it also work with different image backgrounds?
- Can you classify baby fish, which look exactly like their parents, only smaller?
- What happens if there are several different animals in one image?
Those are just a few examples that can help you understand the behavior of the model. When breaking the pattern, be creative!
I’ve just shown you why breaking your model is more useful than seeing it work. The cases in which it fails are those that carry useful information, just as trying to falsify a hypothesis is the key to strengthening it. Now is the time to put formal logic into action! The next time your teammate comes over to show you their new model, take a moment to appreciate it and celebrate the instances where it works well. But after that, start breaking it down and find the edge cases. Those are the ones that will help you further improve the model. On the hunt for black swans!
