Ornery.org
  Front Page   |   About Ornery.org   |   World Watch   |   Guest Essays   |   Contact Us

The Ornery American Forum Post New Topic  Post A Reply
my profile login | register | search | faq | forum home

  next oldest topic   next newest topic
» The Ornery American Forum » General Comments » Remember, the next time you see, read or hear "Studies Prove..." (Page 1)

 - UBBFriend: Email this page to someone!   This topic comprises 2 pages: 1  2   
Author Topic: Remember, the next time you see, read or hear "Studies Prove..."
Daruma28
Member
Member # 1388

 - posted      Profile for Daruma28   Email Daruma28   Send New Private Message       Edit/Delete Post   Reply With Quote 
From Economist Thomas Sowell:

quote:

Studies Prove...

Whenever I hear the phrase "studies prove" this or that, it makes me think back to the beginning of my career as an economist at the Labor Department in Washington.

Secretary of Labor Arthur Goldberg was scheduled to appear before Congress to argue in favor of some policy that the Labor Department wanted enacted into law. Down at the bottom of the chain of command, I was given four sets of census data that had not yet been published and was told to analyze these data for a report to go to the Secretary of Labor.

Two of these sets of data seemed to support the Labor Department's position but the other two went counter to it. When I wrote up a paper explaining why this was so and concluded that the statistics overall were inconclusive, there was much dismay among those in the hierarchy between me and the Secretary.

They were also puzzled as to why anyone would write up such a paper, knowing what the Department's position was on the issues. They took my paper, edited and rewrote it before passing it up the chain of command.

Secretary Goldberg then made his usual confident presentation of the rewritten study to Congress, probably unaware of the contradictory data that had been left out.

It was a valuable experience so early in my career to learn that what "studies prove" is often whatever those who did the studies wanted to prove. Labor Department studies "prove" whatever serves the interest of the Labor Department, just as Agriculture Department studies "prove" whatever serves the Department of Agriculture's interests.

It is the same story on the other side of the Atlantic, where a new book about Britain's criminal justice system exposes the fraudulent methods used to generate statistics about the "success" of various programs of alternatives to imprisonment. The book is titled "A Land Fit for Criminals" by David Fraser.

The numbers may be accurate but the definition of "success" makes them meaningless. When a criminal is put on probation and the probation is not revoked for a violation, that is "success."

Unfortunately, the British criminal justice system does not automatically revoke probation when a criminal commits a new crime.

A criminal on two years' probation can commit a crime after six months, be convicted and sentenced -- and, after serving his sentence, go back to completing the remaining 18 months of his probation, producing statistical "success" for the probation program. That is the whole point of the "study."

On either side of the Atlantic, it is a terminal case of naivete to put statistical studies under the control of the same government agencies whose policies are being studied.

Nor will it do any good to let those agencies farm out these studies to "independent" researchers in academia or think tanks because they will obviously farm them out to people whose track record virtually guarantees that they will reach the conclusions that the agency wants.

Climate expert Richard S. Lindzen of M.I.T. has indicated that the vast amount of government research money available for studies of "global warming" can discourage skeptics from being vocal about their skepticism.

This is not peculiar to studies of "global warming." Many people who complain about the corrupting influence of money never seem to apply that to government money.

If high government officials were serious about wanting to know the facts, they could set up an independent statistical agency, along the lines of the General Accounting Office, to do studies of the effects of the policies of the operating agencies.

That would mean that the fox would no longer be in charge of the hen house, whether the fox was the Labor Department, the Commerce Department, or any of the other departments and agencies.

It would also mean that various bright ideas originating in Congress or the White House would now be exposed to the risk of being shown to be costly failures or even counterproductive. Whole careers could be ruined among both elected officials and bureaucrats.

Don't hold your breath waiting for it to happen. But do keep that in mind when someone says "studies prove . . . "


Posts: 7543 | Registered: Nov 2003  |  IP: Logged | Report this post to a Moderator
Larfoutloud
Member
Member # 2881

 - posted      Profile for Larfoutloud   Email Larfoutloud       Edit/Delete Post   Reply With Quote 
86.29% of statistics are made up on the spot.
Posts: 119 | Registered: May 2006  |  IP: Logged | Report this post to a Moderator
Daruma28
Member
Member # 1388

 - posted      Profile for Daruma28   Email Daruma28   Send New Private Message       Edit/Delete Post   Reply With Quote 
Where's your citations to prove that statistic?
Posts: 7543 | Registered: Nov 2003  |  IP: Logged | Report this post to a Moderator
hobsen
Member
Member # 2923

 - posted      Profile for hobsen   Email hobsen   Send New Private Message       Edit/Delete Post   Reply With Quote 
Once in a while Thomas Sowell writes a good column, and this is by far the best I have seen.

Long ago I took part in an Army study of the effect of freezing temperatures on a soldier's performance. They had us stand in our underwear in a wind tunnel with a 30 mph wind at 30 degrees, while we tried to hold a stylus within a small hole in a metal plate. An electrical contact reported every time we touched the side. Obviously this became harder as we started to shiver more from the cold.

After a few hours over several days, I discovered that despite severe shivering I could still learn to hold the stylus steady and not touch the side. When I mentioned this to the experimenter, he said others had found this out before; and he corrected by throwing out all such data. He wanted an experiment showing a linear relationship between increasing cold and inability to perform a simple manual task, so he threw out all data that conflicted with that desired result.

This left me feeling rather disappointed for standing around shivering for a week. Obviously he could just as well have made up the data without performing any measurements at all, as many researchers have done since. Ah well, it gave me a break from military routine; and we were only tested two hours a day. The other six I spent in the library of the Natick Environmental Test Station - or whatever it was called - catching up on back issues of scientific journals. But the Bush Administration could save a lot of money by simply faking all data for all scientific studies, since they seem to pay no attention to them anyway.

[ August 09, 2006, 04:46 PM: Message edited by: hobsen ]

Posts: 4387 | Registered: Jul 2006  |  IP: Logged | Report this post to a Moderator
TommySama
Member
Member # 2780

 - posted      Profile for TommySama   Email TommySama       Edit/Delete Post   Reply With Quote 
99.99% of people who claim statistics are made up on the spot, are making up those statistics on the spot.
Posts: 6396 | Registered: Feb 2006  |  IP: Logged | Report this post to a Moderator
LoverOfJoy
Member
Member # 157

 - posted      Profile for LoverOfJoy   Email LoverOfJoy   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
Where's your citations to prove that statistic?
A comment by larfoutloud proves that 86.29% of statistics are made up on the spot. However, a recent post by TommySama contradicts this claim and proves that global warming is NOT occurring and if it is, then it's only through naturally occuring means such as volcanoes. [Big Grin]
Posts: 3639 | Registered: Nov 2000  |  IP: Logged | Report this post to a Moderator
Daruma28
Member
Member # 1388

 - posted      Profile for Daruma28   Email Daruma28   Send New Private Message       Edit/Delete Post   Reply With Quote 
Actually LOJ, Global Warming IS happening...studies prove that 83.75% of Global Warming comes from overheated rhetoric from political debates on internet message boards....
Posts: 7543 | Registered: Nov 2003  |  IP: Logged | Report this post to a Moderator
LetterRip
Member
Member # 310

 - posted      Profile for LetterRip   Email LetterRip   Send New Private Message       Edit/Delete Post   Reply With Quote 
First, studies tend not to 'prove' anything. Second, this is why peer review is critical, and why results should be published and not subject to political revision or censorship. Third, his experience is during the 1960's. Fourth his failure to report that someone doctored his study appears to show a lack of scientific integrity on the behalf of the author.

quote:
It was a valuable experience so early in my career to learn that what "studies prove" is often whatever those who did the studies wanted to prove. Labor Department studies "prove" whatever serves the interest of the Labor Department, just as Agriculture Department studies "prove" whatever serves the Department of Agriculture's interests.
This is complete crap. This can only be so if the individuals doing the study have a complete lack of sense of scientific integrity. Fortunately the vast majority of scientists would resign rather than sacrifice their scientific integrity by publishing or allowing to be published in their name something that they know to be false.

quote:
Climate expert Richard S. Lindzen of M.I.T. has indicated that the vast amount of government research money available for studies of "global warming" can discourage skeptics from being vocal about their skepticism.
A credible 'skeptic' scientist can make more in 'speaking fees' than a highly respected scientist could earn doing research. Also his statement is a misrepresentation, there is probably little or no funding for 'global warming' research. The only area that would be specific about 'global warming' would be economic impact studies. Botanists, chemists, physicists, mathematicians, biologists, etc. are generally funded for research that isn't global warming specific (ie characterizing ice flow, historical percipitation and temperature records, climate modeling, etc.)

LetterRip

Posts: 8287 | Registered: Jan 2001  |  IP: Logged | Report this post to a Moderator
Daruma28
Member
Member # 1388

 - posted      Profile for Daruma28   Email Daruma28   Send New Private Message       Edit/Delete Post   Reply With Quote 
I know you don't like Dr. Sowell since you are pretty much a liberal that loves to cite research and studies to back up your arguments...but are you saying Dr. Sowell is a liar? Or that he should be discredited for not resigning at that point?

Also note that in his story, he said his report was re-written and then PRESENTED in a speech to congress - in other words, the Secretary most likely only made a passing reference such as "Our most recent Labor Department studies prove that..."

So if you were Dr. Sowell, at the beginning of your career, you'd resign over something like that?

Posts: 7543 | Registered: Nov 2003  |  IP: Logged | Report this post to a Moderator
Larfoutloud
Member
Member # 2881

 - posted      Profile for Larfoutloud   Email Larfoutloud       Edit/Delete Post   Reply With Quote 
quote:
Originally posted by LoverOfJoy:
quote:
Where's your citations to prove that statistic?
A comment by larfoutloud proves that 86.29% of statistics are made up on the spot. However, a recent post by TommySama contradicts this claim and proves that global warming is NOT occurring and if it is, then it's only through naturally occuring means such as volcanoes. [Big Grin]
I plead the fifth and rest my case at the same time.
Posts: 119 | Registered: May 2006  |  IP: Logged | Report this post to a Moderator
Mariner
Member
Member # 1618

 - posted      Profile for Mariner   Email Mariner       Edit/Delete Post   Reply With Quote 
Fortunately the vast majority of scientists would resign rather than sacrifice their scientific integrity by publishing or allowing to be published in their name something that they know to be false.

What's your source on that, LetterRip? I found that "studies prove" that a great deal of scientists have momentary lapses in ethical judgements.

http://www.washingtonpost.com/wp-srv/nation/daily/graphics/misconduct_060905.html

In short:
15.5% of those surveyed had changed their studies due to pressure from a funding source
15.3% dropped data from analysis
12.3% overlooked flawed data
10.8% withheld details
6% failed to prevent data contradicting their own work

Sowell's lack of scientific integrity seems rather mild by comparison. Or should we also criticize hobsen for failing to report his scientist's mishap?

Posts: 538 | Registered: Mar 2004  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
This can only be so if the individuals doing the study have a complete lack of sense of scientific integrity. Fortunately the vast majority of scientists would resign rather than sacrifice their scientific integrity by publishing or allowing to be published in their name something that they know to be false.
This is doubtlessly true in at least one respect, LR, but does a self-defining statements like that really show what it appears to show? I mean, if they don't feel that way, they aren't really scientists in the ideal sense, just as I can say that Christians don't commit genoicide, in spite of the evidence of history. It's a perscriptive statement, and true, because Scientist and Christian describe certain ideals.

OTOH, I could not say that members of Christian churches don't commit X, Y, or Z, because there I've used descriptive phrases rather than perscriptive. Could you say that the vast majority of persons with science degrees would resign rather than sacrifice their scientific integrity by publishing or allowing to be published in their name something that they know to be false? Looking from the outside rather than looking at the heart, how would you characterize the group that you just described as scientists?

See, I'd be the first to agree that a lot of the scientists that get up and testify to whatever for money, aren't really scientists. The trick is, how do I convince the judge that they aren't really scientists?

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Mormegil
Member
Member # 2439

 - posted      Profile for Mormegil         Edit/Delete Post   Reply With Quote 
quote:
Fortunately the vast majority of scientists would resign rather than sacrifice their scientific integrity by publishing or allowing to be published in their name something that they know to be false.
Integrity is not something I see a great deal of any more in this society. I don't see why so-called scientists would be magically exempt.
Posts: 800 | Registered: May 2005  |  IP: Logged | Report this post to a Moderator
rightleft22
Member
Member # 2832

 - posted      Profile for rightleft22   Email rightleft22   Send New Private Message       Edit/Delete Post   Reply With Quote 
Studies and statistics are dangerous, if we just got rid of studies the world would be a better place.
Posts: 935 | Registered: Mar 2006  |  IP: Logged | Report this post to a Moderator
canadian
Member
Member # 1809

 - posted      Profile for canadian   Email canadian       Edit/Delete Post   Reply With Quote 
but keep the statistics.
Posts: 5362 | Registered: Jun 2004  |  IP: Logged | Report this post to a Moderator
canadian
Member
Member # 1809

 - posted      Profile for canadian   Email canadian       Edit/Delete Post   Reply With Quote 
(we need a little danger)
Posts: 5362 | Registered: Jun 2004  |  IP: Logged | Report this post to a Moderator
rightleft22
Member
Member # 2832

 - posted      Profile for rightleft22   Email rightleft22   Send New Private Message       Edit/Delete Post   Reply With Quote 
Shouldn’t make life to comfortable!

[ August 10, 2006, 12:17 PM: Message edited by: rightleft22 ]

Posts: 935 | Registered: Mar 2006  |  IP: Logged | Report this post to a Moderator
canadian
Member
Member # 1809

 - posted      Profile for canadian   Email canadian       Edit/Delete Post   Reply With Quote 
[Wink]
Posts: 5362 | Registered: Jun 2004  |  IP: Logged | Report this post to a Moderator
hobsen
Member
Member # 2923

 - posted      Profile for hobsen   Email hobsen   Send New Private Message       Edit/Delete Post   Reply With Quote 
Mormegil: "Integrity is not something I see a great deal of any more in this society. I don't see why so-called scientists would be magically exempt."

While scientists are little better than others, that seems overly pessimistic to me. Most people try to do a good job most of the time, while a few are characteristically scum. But most will compromise if you hold a gun to their children's heads.

While Christians celebrate the martyrs who suffered in the first persecutions, I vaguely remember a modern study showing about 90% of Christians at the time renounced their faith rather than get tortured to death. When the persecution eased, they went back to it. I do not really think worse of them for that.

Posts: 4387 | Registered: Jul 2006  |  IP: Logged | Report this post to a Moderator
Colin JM0397
Member
Member # 916

 - posted      Profile for Colin JM0397   Email Colin JM0397   Send New Private Message       Edit/Delete Post   Reply With Quote 
The problem isn't the outright "evil". It's from good people with good hearts and good souls but misguided motives that most "evil" things happen.

Allegedly, only about 5% of the population is disposed to sociopathic behavior, and less than that actually act on it. Meaning most evil done is not perpetrated by an evil or morally blank person.

The so called experts are held up not just as experts in their field, but as "smart" people who are wise in everything. Therein, IMO, is our trouble with "studies prove." In my experience, the more letters you have after your name, the more you know about one thing and the less you know about everything else.

For example, what does a climatologist know about what makes the petro-based economy tick? It’s one thing to come out in favor of, say, reduced carbon emissions. It’s quite another to push the Kyoto treaty because you want to reduce carbon emissions.

What's that saying about the road to hell?

[ August 10, 2006, 02:43 PM: Message edited by: jm0397 ]

Posts: 4738 | Registered: Mar 2003  |  IP: Logged | Report this post to a Moderator
Everard
unregistered


 - posted            Edit/Delete Post   Reply With Quote 
I've been tempted to totally avoid this thread, because the article in question appears to me to be only another in a long line of attempts to get people to think the scientific method is a fraudulent epistomology, and that we can't learn anything from research scientists.

However, what the problem really is are people who aren't doing the science that is being reported on. The people in the labor department? Not doing science, rewrote the report for political reasons. "Studies prove?" Usually uttered in reporting situations, not usually found in scientific papers.

The real lesson to learn is this:
1) Scientific papers need to have abstracts written by people who know how to write for communication with non-scientists, which can be diseminated to news media.
2) Scientific papers should not be re-written by politicians or other non-specialists in the area.
3) More people need to learn the scientific method better, so that they can go back to research and understand how the research was done, and what conclusions can be drawn from the data.

Scientists can, by and large, be trusted to do their job honestly, just as, by and large, the cashier at your grocery store can be trusted to do his job honestly. There are checks and mechanisms to deal with dishonest science and scientists. More could be put into place, and probably should be. For example, peer reviewers should be given all the data, including the ignored data points, to make sure what is tossed is tossed for good reason. This doesn't always happen (though often it does).

The ultimate question, though, is how many flawed studies are there, and how flawed are they? Given the nature of the scientific communities, the answers almost have to be "not very many that are deeply flawed," because deeply flawed papers that get written up, and found out, get totally trashed by other scientists in the field. One of the defining characterists of publishing scientists is "massive ego," and if they can tear into a collegue who screwed up, they will.

What we really need is for scientists and media outlets to work together to improve scientific reporting, so that we STOP seeing "studies prove," and start seeing more "studies indicate that X has a Y% chance of doing Z"

IP: Logged | Report this post to a Moderator
hobsen
Member
Member # 2923

 - posted      Profile for hobsen   Email hobsen   Send New Private Message       Edit/Delete Post   Reply With Quote 
Better than I could have said it, Everard. Very good.
Posts: 4387 | Registered: Jul 2006  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
What we really need is for scientists and media outlets to work together to improve scientific reporting, so that we STOP seeing "studies prove," and start seeing more "studies indicate that X has a Y% chance of doing Z"
I absolutely agree. That's why I always chime in with my complaints about how our culture sets up scientists to be televangelists rather than scientists. Unfortunately you take some of my statements as an attack on science itself or on the scientific ethic, rather than on the way that the media and the law treat science and other 'expert' opinion.

Science thrives on hard questions and free speech. I don't think that it thrives well in an environment where scientists are rewarded for reaching politically expedient conclusions, and where no one is allowed to question or argue those conclusions. Autharitarian science and authoritarian religion meet at the extremes.

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Everard
unregistered


 - posted            Edit/Delete Post   Reply With Quote 
" don't think that it thrives well in an environment where scientists are rewarded for reaching politically expedient conclusions, and where no one is allowed to question or argue those conclusions."

I just don't think that environment exists to the degree that you do. You should talk to The Rabbit on hatrack. Or possibly I should find the link I'm thinking of, but I'm making a fresh blackberry pie at the moment, and am therefore not feeling like running a search [Smile]

IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
LR, there's a physician researcher right now who has shed some light on Autism, and is under discipline by British medical boards who are saying that he did studies on people without their consent.

But he has the consent forms!
And no one who signed those forms is complaining!
He had the consent forms verified by the ethics board before he proceeded with the project.

The medical :ethics: board makes its argument like this:

You did not tell us that your evidence would show that certain vaccines are linked to autism.

If we'd known that you were going to investigate that connection, we would not have approved your consent forms.

Therefore you committed malpractive by doing research on patients without their consent.

I could not believe my eyes -- I saw this on THEIR boards, not on some crackpot anti-vaccine board (and there are lots of unreliable ones there). Can a group of prestigious scientists and doctors get so buggered up in their own authority and community and politics that they really forget what it's all about?

I think that Shaw was right. We really are all human, and the inquisition was more about how human authority works, rather than how religion in particular works.

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Everard
unregistered


 - posted            Edit/Delete Post   Reply With Quote 
http://www.hatrack.com/cgi-bin/ubbmain/ultimatebb.cgi?ubb=get_topic;f=2;t=044114;p=0&r=nfx#000011

Here's the thread I'm thinking of from most recently

IP: Logged | Report this post to a Moderator
Everard
unregistered


 - posted            Edit/Delete Post   Reply With Quote 
Can you link to that please, pete?
IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
Originally posted by Everard:
I just don't think that environment exists to the degree that you do. You should talk to The Rabbit on hatrack. Or possibly I should find the link I'm thinking of, but I'm making a fresh blackberry pie at the moment, and am therefore not feeling like running a search [Smile]

I don't speculate to what degree it exists. I just know that I keep running into this problem, in science and in other forms of authority. People mistake the means for the ends. Lawyers and judges start talking as if the purpose of the law was to fill up the prisons and to keep people respecting us, rather than to keep the peace and to help foster a just society. Government agencies end up having no other purpose other than to expand and maintain themselves.

A society that fails to question authority is on a collision course with history. The political left used to understand this back in the 1960s, or at least said so, but then they got institutional power, and they became like any other group. When did "question authority" turn into "smile and say yes"?

[ August 10, 2006, 04:27 PM: Message edited by: Pete at Home ]

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
Later. I've got to finish this damned paper before 4pm, and then get some sleep.

/break over

Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
Eric
Member
Member # 2699

 - posted      Profile for Eric   Email Eric       Edit/Delete Post   Reply With Quote 
My own informal study proves that swallowing saliva in small amounts over a long period of time ends in death in 100% of the cases!
Posts: 448 | Registered: Dec 2005  |  IP: Logged | Report this post to a Moderator
Mormegil
Member
Member # 2439

 - posted      Profile for Mormegil         Edit/Delete Post   Reply With Quote 
quote:
While scientists are little better than others, that seems overly pessimistic to me. Most people try to do a good job most of the time, while a few are characteristically scum. But most will compromise if you hold a gun to their children's heads.
Oh, people sell their integrity for a lot less than their children's lives.

Some sell it so they won't get an F on their homework.

Just try this little experiment: any time someone asks you to do anything that might compromise your integrity, just say "No, that would be dishonourable."

See how much respect that gets you. That'll tell you how much people respect integrity. And a society that doesn't respect it is one that doesn't have it.

Posts: 800 | Registered: May 2005  |  IP: Logged | Report this post to a Moderator
hobsen
Member
Member # 2923

 - posted      Profile for hobsen   Email hobsen   Send New Private Message       Edit/Delete Post   Reply With Quote 
Nice one, Eric! Telling the difference between causation and correlation is a genuine scientific problem.
Posts: 4387 | Registered: Jul 2006  |  IP: Logged | Report this post to a Moderator
Daruma28
Member
Member # 1388

 - posted      Profile for Daruma28   Email Daruma28   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
I've been tempted to totally avoid this thread, because the article in question appears to me to be only another in a long line of attempts to get people to think the scientific method is a fraudulent epistomology, and that we can't learn anything from research scientists.

However, what the problem really is are people who aren't doing the science that is being reported on. The people in the labor department? Not doing science, rewrote the report for political reasons. "Studies prove?" Usually uttered in reporting situations, not usually found in scientific papers.

Ev, I'm impressed. From past experience with you, you would generally take the first approach and immediately run with it to criticize or argue with the point I posted. You got it exactly right this time...good job.

You will NEVER find me posting anything about the scientific method as being a fradulent epistemology. But a lot of frauds and/or disingenous politically driven people can and do distort, misrepresent or mis-report studies to suit their ideological purposes...which is exactly what Dr. Sowell is trying to point out.

Glad you got it.

Anyhow, Dr. Sowell has written a part II on this topic...


quote:

Studies show...Part II
By Dr. Thomas Sowell

My late mentor, Nobel Prize-winning economist George Stigler, used to say that it could be very instructive to spend a few hours in a library checking up on studies that had been cited. When I began doing that, I found it not only instructive but disillusioning.

A footnote in a textbook on labor economics cited six studies to back up a conclusion it reached. But, after I went to the library and looked at those six studies, it turned out that they each cited some other study -- the same other study in all six cases.

Now that the six studies had shrunk to one, I got that one study -- and found that it was a study of a very different situation from the one discussed in the labor economics textbook.

Some years back, there was a great flurry in the liberal media because a study showed that (1) black pregnant women received prenatal care less often than white pregnant women and that (2) infant mortality rates were higher among blacks.

There were indignant editorials in the New York Times and the Washington Post blaming the government for not providing greater access to prenatal care in order to stop preventable deaths of infants.

After getting a copy of the original study, I discovered that in the same study -- on the very same page -- statistics showed that (1) Mexican American women received even less prenatal care than black women and that (2) infant mortality rates among Mexican Americans were no higher than among whites.

A few pages further on, statistics showed that American women of Chinese, Japanese and Filipino ancestry also received less prenatal care than white women -- and had lower infant mortality rates than whites.

Apparently prenatal care was not the answer, though it was the kind of answer that suited the mindset of the liberal media and provided an occasion for them to wax indignant.

More recently, the National Academy of Sciences came out with a study that supposedly proved beyond a doubt that human activities were responsible for "global warming." A chorus of voices in the media, in politics and in academia proclaimed that this was no longer an issue but a scientific fact, proven with hard data.

The NAS report not had only statistics, it had an impressive list of scientists, which supposedly put the icing on the cake.

The only problem was that the scientists had not written the report and in fact had not even seen it before it was published, even though they had some affiliation with the National Academy of Sciences.

At least one of those scientists, meteorologist Richard S. Lindzen of M.I.T., publicly opposed the conclusion and has continued to do so. But that fact was largely lost in the midst of the media hoopla.

Besides, what is a mere meteorologist at M.I.T. compared to Al Gore and his movie?

Nobody can afford the time to check out every claim of what "studies prove." Even with the help of outstanding research assistants, I can only check out some.

However, the big television and print media have ample financial resources to check out claims before they present them to the public as "news." But when "60 Minutes" didn't bother before basing a story about President Bush's national guard service on a forged document, do not look for a lot of zeal for facts when that could kill a juicy story or the political spin accompanying it.

Let's face it. There is not much pay-off to checking original sources.

Once a minister was explaining to me the structure of his funeral orations. He said, "At this point, you are expected to say something good about the deceased. Now, Tom, if I were preaching your funeral, what would I say good about you at that point?"

He thought and thought -- for an embarrassingly long time. Finally, he said gravely: "In his research, he always used original sources."

I'll take that.

If anyone has anything to respond to this latest...please let's not get sidetracked about Dr. Sowell's use of the term "liberal media."
Posts: 7543 | Registered: Nov 2003  |  IP: Logged | Report this post to a Moderator
LetterRip
Member
Member # 310

 - posted      Profile for LetterRip   Email LetterRip   Send New Private Message       Edit/Delete Post   Reply With Quote 
arrrgh... had a response that i'd been typing on off and on all day and it crashed... oh well will try again tomorrow.

Pete,

quote:
I mean, if they don't feel that way, they aren't really scientists in the ideal sense, just as I can say that Christians don't commit genoicide, in spite of the evidence of history.
I said the vast majority (possibly an overstatement though, probably depends a lot on the field and other factors, for instance), I realize that there are individuals who are 'scientists' and who don't feel particularly strongly about the need for integrity in research.

The research that Mariner is citing is a sample primarily consiting of clinicians (unfortunately the integrity research is almost exclusively focused on problems with clinical research reports - and in particular the influence that corporate sponsorship has - since questionable research practices do occur far too frequently in clinical trials research).

Regarding the quoted numbers, after some digging I found the original source, but the paper itself isn't freely available online.

There is a letter to nature regarding the paper that suggests that due to lack of clarity in methodology the interpretation of the results are in question.

http://www.asm.org/ASM/files/ccLibraryFiles/FILENAME/000000001723/znw00805000347.pdf

It would be interesting to reconduct the study with the above errors addressed.

Also as noted above, clinical research, and particularly corporate sponsorship of clinical research is strongly linked to engaging in questionable research practices (most common is suppressing or delaying of research that is unfavorable to the corporation; along with questionable sampling and interpretation practices - this is why many journals are now requiring that research not have gag contracts attached - that is the corporation cannot have stipulated that they get to decide whether the results are published etc), and the sample for that study consisted primarily of individuals involved in clinical research.

LetterRip

Posts: 8287 | Registered: Jan 2001  |  IP: Logged | Report this post to a Moderator
LetterRip
Member
Member # 310

 - posted      Profile for LetterRip   Email LetterRip   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
My late mentor, Nobel Prize-winning economist George Stigler, used to say that it could be very instructive to spend a few hours in a library checking up on studies that had been cited. When I began doing that, I found it not only instructive but disillusioning.
I certainly recommend checking cites, especially of books, since they are generally not peer reviewed. Checking cites of peer reviewed articles is much less likely to turn up sloppy work though (although it certainly happens).

quote:
A footnote in a textbook on labor economics cited six studies to back up a conclusion it reached. But, after I went to the library and looked at those six studies, it turned out that they each cited some other study -- the same other study in all six cases.
That is clearly citing inappropriately on behalf of the textbook author, one is supposed to cite primary sources, not secondary sources. That is a strong arguement that textbooks should go through a peer review process, and something that I'd absolutely advocate. The social sciences are known to have lower quality citing and peer review than the hard sciences, so the fact that this occurred in a economics textbook isn't too surprising.

quote:
There were indignant editorials in the New York Times and the Washington Post blaming the government for not providing greater access to prenatal care in order to stop preventable deaths of infants.
Yep, I never trust newspapers to properly report the findings of research, I'm sure you've all seen me rip apart such articles here.

quote:
A few pages further on, statistics showed that American women of Chinese, Japanese and Filipino ancestry also received less prenatal care than white women -- and had lower infant mortality rates than whites.

Apparently prenatal care was not the answer, though it was the kind of answer that suited the mindset of the liberal media and provided an occasion for them to wax indignant.

This is very sloppy on his part - he is drawing conclusions not in evidence - all that can be drawn from the above is that there are alternatives to pre natal care that also have healthy outcomes.

quote:
The only problem was that the scientists had not written the report and in fact had not even seen it before it was published, even though they had some affiliation with the National Academy of Sciences.
Not sure what report he is refering to, anyone have a reference?

I thought perhaps this one, which reaffirms Mann et als findings,

http://fermat.nap.edu/catalog/11676.html

but perhaps it is another?

quote:
But when "60 Minutes" didn't bother before basing a story about President Bush's national guard service on a forged document
They'd probably spent many 100 man hours doing so, the multiple forgery experts they hired failed to identify the document as a forgery, nor did they identify even aspects that would have suggested that the document might not be authentic. Similarly the individuals they contacted who were relations of the individual and might have had direct knowledge of the document did not suggest that it was a forgery.

Essentially they had a document that while from a politically motivated source, was vetted by both experts in forger who did not identify any reason for thm to believe it was a forgery, and by individuals who might have had direct knowledge.

quote:
do not look for a lot of zeal for facts when that could kill a juicy story or the political spin accompanying it.
Of course this flys in the face of the facts of their actual actions, but why should that stop him from libeling them.

quote:
Let's face it. There is not much pay-off to checking original sources.
There is the prevention of the embarassment and loss of prestige from being wrong, which is a potentially huge 'payoff'. Unfortunately inaccurate reporting of science is held to a much lower standard then other innaccurate reporting.

LetterRip

Posts: 8287 | Registered: Jan 2001  |  IP: Logged | Report this post to a Moderator
Pete at Home
Member
Member # 429

 - posted      Profile for Pete at Home   Email Pete at Home   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:
A few pages further on, statistics showed that American women of Chinese, Japanese and Filipino ancestry also received less prenatal care than white women -- and had lower infant mortality rates than whites.
Interesting. Do those numbers include or discount the ones they actually kill on purpose, e.g. for being disabled or being female?
Posts: 44193 | Registered: Jun 2001  |  IP: Logged | Report this post to a Moderator
LetterRip
Member
Member # 310

 - posted      Profile for LetterRip   Email LetterRip   Send New Private Message       Edit/Delete Post   Reply With Quote 
Pete,

quote:
LR, there's a physician researcher right now who has shed some light on Autism, and is under discipline by British medical boards who are saying that he did studies on people without their consent.
If the situation is as you represent it, then they are almost surely in the wrong. Can you provide a link?

LetterRip

Posts: 8287 | Registered: Jan 2001  |  IP: Logged | Report this post to a Moderator
Daruma28
Member
Member # 1388

 - posted      Profile for Daruma28   Email Daruma28   Send New Private Message       Edit/Delete Post   Reply With Quote 
After reading all your responses, and a little more consideration, I am surprised LR that your initial reaction to the first post was negative to Dr. Sowell, because the overall point he was making was exactly what you yourself take great pains to do here on OA...never trust or take at face value any particular interpretation or casual citation of a study, as often times they are skewed or distorted to fit a particular ideological agenda.
Posts: 7543 | Registered: Nov 2003  |  IP: Logged | Report this post to a Moderator
LoverOfJoy
Member
Member # 157

 - posted      Profile for LoverOfJoy   Email LoverOfJoy   Send New Private Message       Edit/Delete Post   Reply With Quote 
quote:


quote:

A footnote in a textbook on labor economics cited six studies to back up a conclusion it reached. But, after I went to the library and looked at those six studies, it turned out that they each cited some other study -- the same other study in all six cases.

That is clearly citing inappropriately on behalf of the textbook author, one is supposed to cite primary sources, not secondary sources. That is a strong arguement that textbooks should go through a peer review process, and something that I'd absolutely advocate. The social sciences are known to have lower quality citing and peer review than the hard sciences, so the fact that this occurred in a economics textbook isn't too surprising.
Does the peer review process include checking every source to make sure they are primary sources? I got the impression that while some may go the extra mile in that regard, most people (including peer reviewers and researchers) are lazy and the reviewers check claims made vs. results found much more than actually looking up accurate-sounding sources.
Posts: 3639 | Registered: Nov 2000  |  IP: Logged | Report this post to a Moderator
TommySama
Member
Member # 2780

 - posted      Profile for TommySama   Email TommySama       Edit/Delete Post   Reply With Quote 
quote:
Originally posted by hobsen:
Nice one, Eric! Telling the difference between causation and correlation is a genuine scientific problem.

Even if you have to use a George Carlin quote to do it. ;-)
Posts: 6396 | Registered: Feb 2006  |  IP: Logged | Report this post to a Moderator
  This topic comprises 2 pages: 1  2   

Quick Reply
Message:

HTML is not enabled.
UBB Code™ is enabled.
UBB Code™ Images not permitted.
Instant Graemlins
   


Post New Topic  Post A Reply Close Topic   Feature Topic   Move Topic   Delete Topic next oldest topic   next newest topic
 - Printer-friendly view of this topic
Hop To:


Contact Us | Ornery.org Front Page

Powered by Infopop Corporation
UBB.classic™ 6.7.1