criticism of PEAR

[davidsmith73]

In May 20th weeks Swift, Randi says this about the PEAR research program:

"They dump all data into one huge record, thus obtaining a tiny margin of what Any scraps of positive data that might have gotten in by error or design, would skew the total data. "


This is quite a claim made by Randi. Error that might be present in the experimental design and data analysis (and it could be there after all, we never know) is the cause of the positive findings. But hang on, if we disect what is being said here we can see that its extremely easy to dismiss the results of an experiment without actually giving a reasoned argument, but instead rely on unsubstantiated claims. All you have to declare is that the data set is too large, the effect too small and, hey presto, error can explain the results. What we don't see in Randi's commentry is any suggestion of what errors could be responsible, and how they could skew the results. Such an explanation is the duty of a commentry that claims to be based on critical thinking.

[davidsmith73]

Randi also wrote:

"Note, too, that Ms. Dunne diverts the intent of the investigation to appear as if she would have to "persuade" us of something or other; that's obviously not true,..."


Thats a bizzare comment. What else would the investigations by Randi's team be for other than to persuade them as to the validity of the results?

[DrMatt]

Originally posted by davidsmith73
In May 20th weeks Swift, Randi says this about the PEAR research program:

"They dump all data into one huge record, thus obtaining a tiny margin of what Any scraps of positive data that might have gotten in by error or design, would skew the total data. "


This is quite a claim made by Randi. Error that might be present in the experimental design and data analysis (and it could be there after all, we never know) is the cause of the positive findings. But hang on, if we disect what is being said here we can see that its extremely easy to dismiss the results of an experiment without actually giving a reasoned argument, but instead rely on unsubstantiated claims. All you have to declare is that the data set is too large, the effect too small and, hey presto, error can explain the results. What we don't see in Randi's commentry is any suggestion of what errors could be responsible, and how they could skew the results. Such an explanation is the duty of a commentry that claims to be based on critical thinking.

Randi consults with professionals in the field of statistics before saying stuff like this. The goofy procedures of PEAR are well-known to be statistically useless.

[DrMatt]

Originally posted by davidsmith73
Randi also wrote:

"Note, too, that Ms. Dunne diverts the intent of the investigation to appear as if she would have to "persuade" us of something or other; that's obviously not true,..."


Thats a bizzare comment. What else would the investigations by Randi's team be for other than to persuade them as to the validity of the results?

You could try reading up about the challenge rather than simply calling it bizarre. Among other things, all parties to the challenge agree upon specific things which will or will not occur under properly secure circumstances, and which are configured to be obvious and not require a judge. Nobody needs to be pursuaded to embrace any paradigm or model of how the alleged paranormal feat occurs. The claimant simply must show that the paranormal feat does actually occur, within the agreed-upon time-frame, without cheating, faking, lying, etc.

[CFLarsen]

Originally posted by davidsmith73
Error that might be present in the experimental design and data analysis (and it could be there after all, we never know) is the cause of the positive findings. But hang on, if we disect what is being said here we can see that its extremely easy to dismiss the results of an experiment without actually giving a reasoned argument, but instead rely on unsubstantiated claims. All you have to declare is that the data set is too large, the effect too small and, hey presto, error can explain the results. What we don't see in Randi's commentry is any suggestion of what errors could be responsible, and how they could skew the results. Such an explanation is the duty of a commentry that claims to be based on critical thinking.

Randi is right: Error can easily explain the minuscule results.

Perhaps you are saying that the PEAR results can not be explained by this?

[Jeff Corey]

I'm looking into something that I heard back when this PEAR stuff started to get ink. Unfortunately, I thought the whole issue was just silly and didn't follow up.
What I heard was that Ms. Dunne started out as a lab assistant/subject and got to take the probability generator home with her and record the results of tens of thousands of trials.
So the question is, were any of the PEAR data recorded in a blinded fashion?
Even a small proportion of trials without this elementary control could account for the statistically significant, but miniscule, effect.
50.052 percent, at one time.
Look at how easily Soal and Levy were able to diddle data.

[davidsmith73]

Originally posted by DrMatt
Randi consults with professionals in the field of statistics before saying stuff like this. The goofy procedures of PEAR are well-known to be statistically useless.

How? Is there a paper available on it? Or is are you basing your view on what you have heard?

[davidsmith73]

Originally posted by DrMatt
Nobody needs to be pursuaded to embrace any paradigm or model of how the alleged paranormal feat occurs. The claimant simply must show that the paranormal feat does actually occur, within the agreed-upon time-frame, without cheating, faking, lying, etc.

Right, so by taking the challenge the claimant is persuading Randi that the feat does occur then. That is what I think Dunne was saying. I find it bizzare the Randi must interpret her words as a diversion. This is Randi's personal subjective interpretation of something rather open to interpretation. Ho hum.

[davidsmith73]

Originally posted by CFLarsen
Randi is right: Error can easily explain the minuscule results.

How?


Perhaps you are saying that the PEAR results can not be explained by this?

No, I'm just after a bit of an explanation as to how Randi thinks error and a large data set explain the results of the PEAR program on micro-PK.

[jmercer]

Originally posted by davidsmith73

No, I'm just after a bit of an explanation as to how Randi thinks error and a large data set explain the results of the PEAR program on micro-PK.

The best thread I've seen on PEAR (http://www.skepticforum.com/viewtopic.php?t=207).

The above thread pretty much gives you a complete explanation of the multiple flaws in PEAR and should answer your questions. However, be advised - some of the posts are technically dense. You'll need to take some time to read and parse the concepts - just a "browse" won't do it.

[davidsmith73]

Originally posted by jmercer
The best thread I've seen on PEAR (http://www.skepticforum.com/viewtopic.php?t=207).

The above thread pretty much gives you a complete explanation of the multiple flaws in PEAR and should answer your questions. However, be advised - some of the posts are technically dense. You'll need to take some time to read and parse the concepts - just a "browse" won't do it.

That's a thread on the GCP. Although they use similar techniques, the experimental design is very different to the original PEAR program on micro-PK.

[jmercer]

Oh - my mistake... I thought they were one and the same. Now I have to read up on PEAR. :)

[davidsmith73]

Originally posted by jmercer
Oh - my mistake... I thought they were one and the same. Now I have to read up on PEAR. :)

some of the same people are involved though. But a discussion of the science shouldn't really concern that, apart from issues of replication.

[fls]

No, I'm just after a bit of an explanation as to how Randi thinks error and a large data set explain the results of the PEAR program on micro-PK.

Two things jumped out at me when I first read the PEAR analysis. One was that they used one-tailed testing (which would not be justified for the purposes of convincing proof) which changes the standard to 1 in 10 instead of 1 in 20. Second, they used a calculated variation rather than the measured baseline variation. Since the calculated variation was smaller than the measured variation, more numbers would be found outside of the "normal" variation.

Also, apparently half of the "positive" results were due to one operator who may have been a PEAR staffer. It is worth questioning whether systematic error in the machine used, or some influence (even inadvertent) as an employee may have led to a slight bias of the results. It wouldn't take much since the effect size is "for every thousand electronic tosses, the random event generator is producing about one more head or tail than it should by chance alone". This information is from skepdic.com. As well as the information that other researchers have been unable to duplicate the results.

Linda

[Jeff Corey]

A few points.
One tail testing is fine if your experimental hypotheis is directional. Theirs was.
Doing a one tailed test does not turn .05 into .10. That is not how it works.
And as I mentioned earlier here, Ms. Dunne was apparently the person who contributed the bulk of signifcant results, some of which she recorded at home with no checking for Soal-type "errors".
It would be even easier to cheat than in Soal's case, where he had to alter data where people could see him.

[fls]

Originally posted by Jeff Corey
A few points.
One tail testing is fine if your experimental hypotheis is directional. Theirs was.
Doing a one tailed test does not turn .05 into .10. That is not how it works.
And as I mentioned earlier here, Ms. Dunne was apparently the person who contributed the bulk of signifcant results, some of which she recorded at home with no checking for Soal-type "errors".
It would be even easier to cheat than in Soal's case, where he had to alter data where people could see him.

One-tailed testing requires more than wishful thinking, otherwise we could justify its use in most situations. It isn't just that the hypothesis is directional, but that logic dictates that it is directional. We don't know how this supposed effect works. Thinking "produce more pluses" may lead to the machine producing minuses, for all we know.

Okay, my suspicion is probably unjustified, but here's what I was thinking. If I decide to do a one-tailed test, I get to use an easier cutoff, and I have a 10 percent chance of getting a result outside of that cutoff (if I don't specify a direction). So before I specify a direction, I take a machine and run a bunch of trials. Due to chance alone the results will fall on one or the other side of the mean. I choose whatever side the results fall on as the intended direction of my experimental hypothesis and declare that I am doing one-tailed testing.

Linda