The Test That Cried Wolf

Hold that fear for a second. As scary as it sounds, that “99%” doesn’t mean what your gut thinks it means. For a lot of real tests, a positive like this is more likely to be a false alarm than the real thing. Not because the test is bad — but because of one number nobody mentioned on the phone.

Say a disease affects 1 in 1,000 people, and the test is right 99% of the time. You test positive. What’s the chance you actually have it? Commit to a gut number before you peek.

Drag the two dials. How common the condition is, and how accurate the test is. Watch the big number — your real chance of being sick after a positive — and the bar of who actually gets flagged.

Notice the pattern: crank the test accuracy as high as you like and the positive result still can’t be trusted when the disease is rare. The dial that really moves the answer isn’t accuracy — it’s how common the thing is in the first place.

Picture 10,000 people and a disease that hits 1 in 1,000. That’s just 10 sick people and 9,990 healthy ones. A 99% test catches about 10 of the sick — good. But 1% of the 9,990 healthy people get flagged too, and that’s about 100 false alarms. So the positives are 10 real and 100 fake: your odds are 10 out of 110, roughly 9%.

This isn’t just a medical-school riddle. The same trap fires anywhere you screen a big population for something rare:

Spam filters flagging a real email, because almost no message is the rare thing they hunt for. Fraud alerts freezing your card on a normal purchase. Airport security buzzing on belt buckles a thousand times for every genuine threat. Each looks broken — but it may just be the base rate doing what it always does to a rare target.

The fix in real life is rarely panic. It’s a second, independent test. The first positive drags your odds up from “1 in 1,000” to “1 in 11”; a second positive starts from that new number and pushes it much higher. That’s why doctors confirm before they worry you.

Everything so far was the picture. Here’s the same idea with its proper names, so you’d recognize it on a whiteboard. Three numbers do all the work: sensitivity $P(+\mid \text{sick})$, the chance a sick person tests positive; specificity $P(-\mid \text{healthy})$, the chance a healthy person tests negative; and the base rate $P(\text{sick})$, how common the thing is before any test. The bar “$\mid$” just means “given”.

What you actually want isn’t any of those — it’s the flipped question: given a positive, am I sick? That’s the positive predictive value, $P(\text{sick}\mid +)$, and the tool that flips a conditional around is Bayes’ theorem.

And here’s the payoff for a second test. Bayes doesn’t replace your starting number — it updates it. The first positive turns 0.001 into ~0.09. Run a second, independent test and you do Bayes again, but now 0.09 is your new base rate instead of 0.001. Starting from 9% instead of 0.1%, a second positive pushes the answer well past 90%.