posted
Here's a quiz that introduces the importance of conditional (Bayesian probabilities). Let's see how many of you can find the correct answer (I will reveal later in the weekend).

The hypothetical problem is as follows: ----

There's a new cancer-detection techniques with the following characteristics. - In people suffering from cancer, it has a 80% detection rate. - In healthy people, it has a 5% false negatives rate. Medical science informs us that one out of every hundred people in the age of fifty have cancer, and therefore the government decides to require everyone in that age to be tested, using the new technique.

One particular person, John Smith, age 50, goes and get tested. The test comes back positive. John Smith's not a very educated man, and so he doesn't really know what all these false negatives and detection rates really mean. He pleads to the doctor to tell him straightforwardly: "Doctor, what does the test mean exactly? What's the probability I *actually* have cancer?"

---------------------------------------- I suggest people do the following: - First, write down your intuitive off-the-cuff approximation of what you believe the probability is like -- before making any calculations. - Then, if you have the time, try to do the math as well, calculate it exactly. - Post both your approximation and calculated result. Or just the approximation if you can't bother with the math. - If you've seen this same problem before, please don't spoil the answer for others. -----

I will post the answer later this weekend.
Posts: 3318 | Registered: Feb 2003
| IP: Logged |

posted
In a population of 1000 people: 8 people will be correctly identified. 2 people will go undiagnosed. ~50 people will be false positives. That means ~58 out of 1000 people will test positive for cancer, and 8 of them will actually have it.

John Smith, having tested positive, has a roughly 14% chance of having cancer.
Posts: 22935 | Registered: Nov 2000
| IP: Logged |

posted
Not being nitpicky -- The last probability question I saw on Ornery made less sense to me than a healthy person with a false negative, so I had to ask.

Off the cuff I'd say slightly over 95%, or to be precise, a percentage equal to 100 divided by 105. reduced to the fraction 20/21%

Seems to me that the only relevant numbers are the 5% false positive and the 1% age 50 rate.

Out of 10,000 50 year olds, there will be 50 folks with cancer, of which 40 will be correctly diagnosed and 10 cancerous with false negatives.

Of the healthy 9,500, 95 will be false positives.

40+95 = 135.

Therefore out of 135 folks who tests positive for cancer, 40 actually have cancer.

Therefore someone who tests positive has an 8/27 chance of actually having cancer.

If my calculations are right, then my first guess was way off.

[ November 13, 2010, 12:12 AM: Message edited by: Pete at Home ]
Posts: 44193 | Registered: Jun 2001
| IP: Logged |

posted
Gah, I can't do simple arithmetic. Or I confused the 5% with the 1%

Re-running the numbers:

100 will have cancer out of 10K: 80 correctly diagnosed positives, 20 false negatives.

9900 will be healthy, of which 95% test negative, and 5% (495) test falsely positive.

so 575 out of 10,000 test positive.

so 80 correct cancer diagnoses out of 575 total diagnoses. That reduces to 16/115, or approximately 13.9%
Posts: 44193 | Registered: Jun 2001
| IP: Logged |

posted
I think that's a bit easy- what really becomes fun is asking what the probability is that a second test will now come up positive and how well that second test indicates whether he's actually sick.
Posts: 11997 | Registered: Oct 2005
| IP: Logged |

posted
Yeah, it proved a bit easier than I had hoped. Everyone got the correct answer, at least once they did the math.

The thing that intrigued me about this problem was that the vast majority of *doctors* are supposed to have gotten the answer to the problem (well not *this* problem, but a very similar one) wrong. See here http://yudkowsky.net/rational/bayes

Anyway, I'll have to find a trickier problem. :-)
Posts: 3318 | Registered: Feb 2003
| IP: Logged |

posted
I agree that was much too logical for a statistics problem. Based on the last one we did, I expected something where the numbers change dramatically because he picked his nose last Thursday or something bizarre and irrelevant like that. I never did understand that.
Posts: 44193 | Registered: Jun 2001
| IP: Logged |

a doctor has to answer that type of question for the USMLE Step 1s, and it is one of the 'gimmee' questions. It might be 'old' doctors who are getting it wrong. (It is also the first thing covered in a famous medical instructors review session).

His 1995 reference only gives his other references.

Both the 1982 and 1978 sound like they are 'off the cuff' estimates based on what I can find of their methodology.
Posts: 8287 | Registered: Jan 2001
| IP: Logged |

posted
Pete, if you want someone to go nuts trying to convince you that those crazy-seeming problems make sense, I'm always here for you.
Posts: 2147 | Registered: Nov 2004
| IP: Logged |

quote:Originally posted by Jordan: Pete, if you want someone to go nuts trying to convince you that those crazy-seeming problems make sense, I'm always here for you.

Jordan, I believe that it makes sense to you, and that's what drives me nuts

Say the doctor tells the patient that he has a 13.91% chance of actually having cancer.

The patient says, "but Doctor, I was born on a Thursday."

The doctor says, "Damnit! Why'd you have to go and tell me that? That totally changes the calculations. Now your odds of actually having cancer are ______"

What I have a hard time believing isn't that the method of analysis from that earlier thread makes sense, but that it has some application to real life probability.
Posts: 44193 | Registered: Jun 2001
| IP: Logged |

posted
I shouldn't worry about his being born on a Thursday having an effect in this case!

quote:What I have a hard time believing isn't that the method of analysis from that earlier thread makes sense, but that it has some application to real life probability.

One of the big discussion points in that thread was that the problem statement itself was quite vague, and in fact the counter-intuitive probabilities only come into effect in very specific and carefully constructed scenarios. So while the effect is "real," it's only real in an artificially restricted range of scenarios.

If I recall correctly, Aris was the first to try and explain that, and ended up getting some stick because he'd assumed we were all bright enough to follow him.
Posts: 2147 | Registered: Nov 2004
| IP: Logged |

quote:Originally posted by Jordan: Pete, if you want someone to go nuts trying to convince you that those crazy-seeming problems make sense, I'm always here for you.

Jordan, I believe that it makes sense to you, and that's what drives me nuts

Say the doctor tells the patient that he has a 13.91% chance of actually having cancer.

The patient says, "but Doctor, I was born on a Thursday."

The doctor says, "Damnit! Why'd you have to go and tell me that? That totally changes the calculations. Now your odds of actually having cancer are ______"

What I have a hard time believing isn't that the method of analysis from that earlier thread makes sense, but that it has some application to real life probability.

In your exact scenario, the probability doesn't change.

It will change if the doctor only sees patients who are born on Thursday (i.e. the input is filtered based on this bit of information). The probability will not change if the input isn't filtered. If the man could've said "I was born on Tuesday", and he was just making conversation by telling you what day he was born on (whatever it happened to be), then the numbers don't change.