In his book Bad Science, columnist and psychiatrist Ben Goldacre presents the following simple problem. (I have modified it a bit, but the basic idea is the same.)
I have a deck of 100 index cards. On one side of each card is a letter; on the other is a number between 1 and 10 (inclusive). That information you can take as true. I then deal out four cards as follows.
Hypothesis: Every card in the deck that has a vowel on the letter side has an even number on the number side.
Question: Which two cards do you flip?
I leave you to mull that over before you go on.
Most people start by flipping the A card. If there is an even number on the other side, that supports the hypothesis. If it is an odd number, the hypothesis is shown to be false. In fact, there is a 2 on the other side, so we have support (but not proof!) of the hypothesis.
Hardly anyone flips the B card. Since B is not a vowel, the parity (even or odd) of the other side will tell you nothing about the hypothesis, which says nothing about the other side of non-vowel cards. In fact, there is a 4 on the other side.
What about the 2 card? If you find a vowel on the other side, you will have gathered support (but not proof!) for the hypothesis. But if it’s a non-vowel, you have no evidence for or against the hypothesis. Still, a lot of people turn over the 2 card. And find an F.
On to the 3 card. If the other side is a non-vowel, then the situation is similar to flipping the B card. This result neither supports nor denies the hypothesis. But if the other side is a vowel, then the hypothesis goes down the drain. You have a vowel on one side and an odd number on the other. Definite proof that the hypothesis is untrue.
Do you see the point here? If the hypothesis true, you will have to go through the whole deck to confirm it. But it takes one exception (a non-vowel on the last card) to prove that the hypothesis is untrue.
In short, you should turn over the A and the 3 cards because they are the ones that will show the hypothesis to be untrue. Don’t bother turning the 2 card. Incidentally, here’s what you see when you turn over all the cards.
This conclusion leaves a lot of people baffled. For them, testing a hypothesis amounts to searching for evidence in favour of it. Looking for exceptions strikes them as a kind of betrayal. Deliberately avoiding evidence against the hypothesis is so common it has a name — confirmation bias.