PDA

View Full Version : Seven signs of bogus science


Flaherty
31st December 2003, 02:04 PM
Excellent essay in case you have not read it:

http://www.quackwatch.org/01QuackeryRelatedTopics/signs.html

The National Aeronautics and Space Administration is investing close to a million dollars in an obscure Russian scientist's antigravity machine, although it has failed every test and would violate the most fundamental laws of nature. The Patent and Trademark Office recently issued Patent 6,362,718 for a physically impossible motionless electromagnetic generator, which is supposed to snatch free energy from a vacuum. And major power companies have sunk tens of millions of dollars into a scheme to produce energy by putting hydrogen atoms into a state below their ground state, a feat equivalent to mounting an expedition to explore the region south of the South Pole.

There is, alas, no scientific claim so preposterous that a scientist cannot be found to vouch for it. And many such claims end up in a court of law after they have cost some gullible person or corporation a lot of money. How are juries to evaluate them?

Before 1993, court cases that hinged on the validity of scientific claims were usually decided simply by which expert witness the jury found more credible. Expert testimony often consisted of tortured theoretical speculation with little or no supporting evidence. Jurors were bamboozled by technical gibberish they could not hope to follow, delivered by experts whose credentials they could not evaluate.

In 1993, however, with the Supreme Court's landmark decision in Daubert v. Merrell Dow Pharmaceuticals, Inc. the situation began to change. The case involved Bendectin, the only morning-sickness medication ever approved by the Food and Drug Administration. It had been used by millions of women, and more than 30 published studies had found no evidence that it caused birth defects. Yet eight so-called experts were willing to testify, in exchange for a fee from the Daubert family, that Bendectin might indeed cause birth defects.

In ruling that such testimony was not credible because of lack of supporting evidence, the court instructed federal judges to serve as "gatekeepers," screening juries from testimony based on scientific nonsense. Recognizing that judges are not scientists, the court invited judges to experiment with ways to fulfill their gatekeeper responsibility.

Justice Stephen G. Breyer encouraged trial judges to appoint independent experts to help them. He noted that courts can turn to scientific organizations, like the National Academy of Sciences and the American Association for the Advancement of Science, to identify neutral experts who could preview questionable scientific testimony and advise a judge on whether a jury should be exposed to it. Judges are still concerned about meeting their responsibilities under the Daubert decision, and a group of them asked me how to recognize questionable scientific claims. What are the warning signs?

I have identified seven indicators that a scientific claim lies well outside the bounds of rational scientific discourse. Of course, they are only warning signs -- even a claim with several of the signs could be legitimate...

RabbiSatan
31st December 2003, 02:09 PM
Thanks very much Flaherty, good read http://www.randi.org/vbulletin/images/icons/icon14.gif

RCNelson
31st December 2003, 03:59 PM
Seven Warning Signs of Bogus Science - Robert L. Park, Ph.D.

1. The discoverer pitches the claim directly to the media.
2. The discoverer says that a powerful establishment is trying to suppress his or her work.
3. The scientific effect involved is always at the very limit of detection.
4. Evidence for a discovery is anecdotal.
5. The discoverer says a belief is credible because it has endured for centuries.
6. The discoverer has worked in isolation.
7. The discoverer must propose new laws of nature to explain an observation.

Suddenly
31st December 2003, 07:50 PM
Some large problems with the essay...

Before 1993, court cases that hinged on the validity of scientific claims were usually decided simply by which expert witness the jury found more credible. Expert testimony often consisted of tortured theoretical speculation with little or no supporting evidence. Jurors were bamboozled by technical gibberish they could not hope to follow, delivered by experts whose credentials they could not evaluate.

<shakes head mournfully>

Daubert simply noted that the "new" Federal Rules of Evidence superceeded the old standard for scientific evidence contained in Frye. Frye applied a "generally accepted in the scientific community" test. Daubert eliminated that test in favor of one where the science must be "relevant and reliable," with the reliable part focusing on methodology. Judges have always had a gatekeeping role w/r/t scientific evidence, all Daubert does is make it more flexible, and also seek to allow more scientific testimony, not less.

In ruling that such testimony was not credible because of lack of supporting evidence, the court instructed federal judges to serve as "gatekeepers," screening juries from testimony based on scientific nonsense. Recognizing that judges are not scientists, the court invited judges to experiment with ways to fulfill their gatekeeper responsibility.

I'm shocked by this; it is simply an incorrect statement of what happened in the case. He has the part about the case dealing with alleged birth defects right, but he has the ruling flat wrong.

Both sides presented experts. Those on the defense testified that "epidemiological" tests showed the drug did not cause birth defects. Those on the plaintiff's side argued that many animal studies showed that the drug caused birth defects. These plaintiff's experts were eight in number all of which had credentials the Supreme Court termed "impressive."

Yes, the experts the article calls "so called experts" were actually eight separate experts said by the Supreme Court to have "impressive credentials." Here's a typical one at random from footnote 2 in the opinion:

"Stewart A. Newman, who received his master's and a doctorate in chemistry from Columbia University and the University of Chicago, respectively, is a professor at New York Medical College and has spent over a decade studying the effect of chemicals on limb development. "

The opinion is here: http://supct.law.cornell.edu:8080/supct/html/92-102.ZO.html


(note the birth defect in question involved limb development)

The trial court ruled that these eight expert's opinions were inadmissable based on the Frye standard, and threw the plaintiffs out of court, holding that only "epidemiological" testing enjoyed general acceptance and that Frye deemed inadmissable all novel scientific theories.

The Supreme Court reversed, saying that the trial court used the incorrect standard and should revisit the issue using the new "relevance and reliabilty" standard that included four main considerations:

1) Can the core principle the evidence is based on be tested
2) Is the theory subject to peer review
3) What is the error rate
4) Is the principle generally accepted.

These are not hard and fast requirements, rather a non-exclusive set of considerations.

The idea being that a sound, but "novel" expert opinion should be allowed even if it has not yet reached "general acceptance."

The article is correct in that judges are encouraged by Daubert to be more "scientific" in that opinions are to be judged on their soundness and methodology rather that their standing in the scientific community.

The irony of the article is that Daubert actually makes crap science more of a problem, by taking the decision of what is valid away from the "scientific community" and giving it to judges, some of whom have no real scientific background. The problems he identifies are more of a problem now than before 1993.

There is one current interesting battleground created by Daubert, the status of forensic identification sciences, such as fingerprinting and tool mark evidence. For years the central principle that no two fingerprints (tool marks) are identical was simply assumed to be true. Now, under Daubert it is becoming obvious that this central principle lacks a scientific basis. Furthermore, the question of whether a match occurs is left to the subjective judgment of the examiner as there are presently no objective criteria for deciding when a match exists. These doctrines are largely under attack (the reason I'm so up to date on this is that at least the "tool mark" side is about to be under attack by me).

Mr Manifesto
1st January 2004, 02:52 AM
Damn, I'm impressed, Suddenly. You could almost make me like lawyers.

!Xx+-Rational-+xX!
1st January 2004, 03:12 AM
If you think about it NDEs must be just fantasies because almost all of them include seeing fields of magical bunnies skipping around looking happy! I don’t have to back this up and show where I have seen this because I’m a skeptic and skeptics are right by default!