Ornery.org
  Front Page   |   About Ornery.org   |   World Watch   |   Guest Essays   |   Contact Us

The Ornery American Forum Post New Topic  Post A Reply
my profile login | register | search | faq | forum home

  next oldest topic   next newest topic
» The Ornery American Forum » General Comments » A probability quiz.

 - UBBFriend: Email this page to someone!    
Author Topic: A probability quiz.
Aris Katsaris
Member
Member # 888

 - posted      Profile for Aris Katsaris   Email Aris Katsaris   Send New Private Message       Edit/Delete Post   Reply With Quote 
Here's a quiz that introduces the importance of conditional (Bayesian probabilities). Let's see how many of you can find the correct answer (I will reveal later in the weekend).

The hypothetical problem is as follows:
----

There's a new cancer-detection techniques with the following characteristics.
- In people suffering from cancer, it has a 80% detection rate.
- In healthy people, it has a 5% false negatives rate.
Medical science informs us that one out of every hundred people in the age of fifty have cancer, and therefore the government decides to require everyone in that age to be tested, using the new technique.

One particular person, John Smith, age 50, goes and get tested. The test comes back positive. John Smith's not a very educated man, and so he doesn't really know what all these false negatives and detection rates really mean. He pleads to the doctor to tell him straightforwardly: "Doctor, what does the test mean exactly? What's the probability I *actually* have cancer?"

----------------------------------------
I suggest people do the following:
- First, write down your intuitive off-the-cuff approximation of what you believe the probability is like -- before making any calculations.
- Then, if you have the time, try to do the math as well, calculate it exactly.
- Post both your approximation and calculated result. Or just the approximation if you can't bother with the math.
- If you've seen this same problem before, please don't spoil the answer for others.
-----

I will post the answer later this weekend.

Posts: 3318 | Registered: Feb 2003  |  IP: Logged | Report this post to a Moderator
TomDavidson
Member
Member # 99

 - posted      Profile for TomDavidson   Email TomDavidson   Send New Private Message       Edit/Delete Post   Reply With Quote 
In a population of 1000 people:
8 people will be correctly identified.
2 people will go undiagnosed.
~50 people will be false positives.
That means ~58 out of 1000 people will test positive for cancer, and 8 of them will actually have it.

John Smith, having tested positive, has a roughly 14% chance of having cancer.

Posts: 22935 | Registered: Nov 2000  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
" In healthy people, it has a 5% false negatives rate."

How can a healthy person have a false negative for cancer? Do you mean false positive?

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Aris Katsaris
Member
Member # 888

 - posted      Profile for Aris Katsaris   Email Aris Katsaris   Send New Private Message       Edit/Delete Post   Reply With Quote 
False positive, my apologies.
Posts: 3318 | Registered: Feb 2003  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
Not being nitpicky -- The last probability question I saw on Ornery made less sense to me than a healthy person with a false negative, so I had to ask.

Off the cuff I'd say slightly over 95%, or to be precise, a percentage equal to 100 divided by 105. reduced to the fraction 20/21%


Seems to me that the only relevant numbers are the 5% false positive and the 1% age 50 rate.

Out of 10,000 50 year olds, there will be 50 folks with cancer, of which 40 will be correctly diagnosed and 10 cancerous with false negatives.

Of the healthy 9,500, 95 will be false positives.

40+95 = 135.

Therefore out of 135 folks who tests positive for cancer, 40 actually have cancer.

Therefore someone who tests positive has an 8/27 chance of actually having cancer.

If my calculations are right, then my first guess was way off.

[ November 13, 2010, 12:12 AM: Message edited by: Pete at Home ]

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
Gah, I can't do simple arithmetic. Or I confused the 5% with the 1%

Re-running the numbers:

100 will have cancer out of 10K: 80 correctly diagnosed positives, 20 false negatives.

9900 will be healthy, of which 95% test negative, and 5% (495) test falsely positive.

so 575 out of 10,000 test positive.

so 80 correct cancer diagnoses out of 575 total diagnoses. That reduces to 16/115, or approximately 13.9%

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
LetterRip
Member
Member # 310

 - posted      Profile for LetterRip   Email LetterRip   Send New Private Message       Edit/Delete Post   Reply With Quote 
Off the cuff I'd say 10%.

Running the numbers.

False positive 5%*99/100 = .0495
True positive 80%*1/100 = .008

.008/(.0495+.008) = 13.91% chance

So if he has a positive test he has a 13.91% chance of actually having the disease.

[ November 13, 2010, 04:31 AM: Message edited by: LetterRip ]

Posts: 8287 | Registered: Jan 2001  |  IP: Logged | Report this post to a Moderator
Pyrtolin
Member
Member # 2638

 - posted      Profile for Pyrtolin   Email Pyrtolin   Send New Private Message       Edit/Delete Post   Reply With Quote 
I think that's a bit easy- what really becomes fun is asking what the probability is that a second test will now come up positive and how well that second test indicates whether he's actually sick.
Posts: 11997 | Registered: Oct 2005  |  IP: Logged | Report this post to a Moderator
Aris Katsaris
Member
Member # 888

 - posted      Profile for Aris Katsaris   Email Aris Katsaris   Send New Private Message       Edit/Delete Post   Reply With Quote 
Yeah, it proved a bit easier than I had hoped. Everyone got the correct answer, at least once they did the math.

The thing that intrigued me about this problem was that the vast majority of *doctors* are supposed to have gotten the answer to the problem (well not *this* problem, but a very similar one) wrong. See here http://yudkowsky.net/rational/bayes

Anyway, I'll have to find a trickier problem. :-)

Posts: 3318 | Registered: Feb 2003  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
I agree that was much too logical for a statistics problem. Based on the last one we did, I expected something where the numbers change dramatically because he picked his nose last Thursday or something bizarre and irrelevant like that. I never did understand that.
Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
LetterRip
Member
Member # 310

 - posted      Profile for LetterRip   Email LetterRip   Send New Private Message       Edit/Delete Post   Reply With Quote 
Aris,

a doctor has to answer that type of question for the USMLE Step 1s, and it is one of the 'gimmee' questions. It might be 'old' doctors who are getting it wrong. (It is also the first thing covered in a famous medical instructors review session).

His 1995 reference only gives his other references.

Both the 1982 and 1978 sound like they are 'off the cuff' estimates based on what I can find of their methodology.

Posts: 8287 | Registered: Jan 2001  |  IP: Logged | Report this post to a Moderator
Jordan
Member
Member # 2159

 - posted      Profile for Jordan   Email Jordan   Send New Private Message       Edit/Delete Post   Reply With Quote 
Pete, if you want someone to go nuts trying to convince you that those crazy-seeming problems make sense, I'm always here for you. [Wink]
Posts: 2147 | Registered: Nov 2004  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
Originally posted by Jordan:
Pete, if you want someone to go nuts trying to convince you that those crazy-seeming problems make sense, I'm always here for you. [Wink]

Jordan, I believe that it makes sense to you, and that's what drives me nuts [Smile]

Say the doctor tells the patient that he has a 13.91% chance of actually having cancer.

The patient says, "but Doctor, I was born on a Thursday."

The doctor says, "Damnit! Why'd you have to go and tell me that? That totally changes the calculations. Now your odds of actually having cancer are ______"

What I have a hard time believing isn't that the method of analysis from that earlier thread makes sense, but that it has some application to real life probability.

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Jordan
Member
Member # 2159

 - posted      Profile for Jordan   Email Jordan   Send New Private Message       Edit/Delete Post   Reply With Quote 
I shouldn't worry about his being born on a Thursday having an effect in this case!

quote:
What I have a hard time believing isn't that the method of analysis from that earlier thread makes sense, but that it has some application to real life probability.
One of the big discussion points in that thread was that the problem statement itself was quite vague, and in fact the counter-intuitive probabilities only come into effect in very specific and carefully constructed scenarios. So while the effect is "real," it's only real in an artificially restricted range of scenarios.

If I recall correctly, Aris was the first to try and explain that, and ended up getting some stick because he'd assumed we were all bright enough to follow him. [Smile]

Posts: 2147 | Registered: Nov 2004  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
Yay! Thank you. The world seems a little less frightening and confusing.
Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
JoshuaD
Member
Member # 1420

 - posted      Profile for JoshuaD   Email JoshuaD   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
Originally posted by Pete at Home:
quote:
Originally posted by Jordan:
Pete, if you want someone to go nuts trying to convince you that those crazy-seeming problems make sense, I'm always here for you. [Wink]

Jordan, I believe that it makes sense to you, and that's what drives me nuts [Smile]

Say the doctor tells the patient that he has a 13.91% chance of actually having cancer.

The patient says, "but Doctor, I was born on a Thursday."

The doctor says, "Damnit! Why'd you have to go and tell me that? That totally changes the calculations. Now your odds of actually having cancer are ______"

What I have a hard time believing isn't that the method of analysis from that earlier thread makes sense, but that it has some application to real life probability.

In your exact scenario, the probability doesn't change.

It will change if the doctor only sees patients who are born on Thursday (i.e. the input is filtered based on this bit of information). The probability will not change if the input isn't filtered. If the man could've said "I was born on Tuesday", and he was just making conversation by telling you what day he was born on (whatever it happened to be), then the numbers don't change.

[ November 15, 2010, 02:05 AM: Message edited by: JoshuaD ]

Posts: 3742 | Registered: Dec 2003  |  IP: Logged | Report this post to a Moderator
   

Quick Reply
Message:

HTML is not enabled.
UBB Code™ is enabled.
UBB Code™ Images not permitted.
Instant Graemlins
   


Post New Topic  Post A Reply Close Topic   Feature Topic   Move Topic   Delete Topic next oldest topic   next newest topic
 - Printer-friendly view of this topic
Hop To:


Contact Us | Ornery.org Front Page

Powered by Infopop Corporation
UBB.classic™ 6.7.1