Misleading Sceptic Report article

[davidsmith73]

Claus,
I have been browsing your Sceptic Report site and although I admire all efforts in bringing to light the rising tides of nonsense that's out there, I think that the article "Analysis of a telepathy test" by Per Johan Rasmark paints an incomplete picture of research into telepathy. Perhaps this distortion is inadvertant but I feel I should express my concerns. The author rightly states that the test described in the article is not scientific, nor was it intended to be. However this makes the test entirely unrepresentative of parapsychological research. What concerns me is the fact that no mention is made in the article of the fact that experiments by serious parapsychologists do not employ such sloppy methodology. Indeed, the author goes as far as to suggest the design of a better experiment than that described. An uninformed reader would not know that better experiments of the type the author suggests have been carried out for many decades now. Do you not think that it is your duty as a sceptic to inform the reader of this?


http://www.skepticreport.com/psychics/telepathy.htm

[Ashles]

[Ashles gets popcorn, settles into comfy seat and awaits the fun]

[Lothian]

Originally posted by davidsmith73
Claus,
I have been browsing your Sceptic Report site and although I admire all efforts in bringing to light the rising tides of nonsense that's out there, I think that the article "Analysis of a telepathy test" by Per Johan Rasmark paints an incomplete picture of research into telepathy. Perhaps this distortion is inadvertant but I feel I should express my concerns. The author rightly states that the test described in the article is not scientific, nor was it intended to be. However this makes the test entirely unrepresentative of parapsychological research. What concerns me is the fact that no mention is made in the article of the fact that experiments by serious parapsychologists do not employ such sloppy methodology. Indeed, the author goes as far as to suggest the design of a better experiment than that described. An uninformed reader would not know that better experiments of the type the author suggests have been carried out for many decades now. Do you not think that it is your duty as a sceptic to inform the reader of this?


http://www.skepticreport.com/psychics/telepathy.htm And I was also remiss to point out that incredibly despite the fact that better experiments of the type the author suggests have been carried out for many decades now we still have no proof that the supposed effect exists.

[davidsmith73]

Originally posted by Lothian
And I was also remiss to point out that incredibly despite the fact that better experiments of the type the author suggests have been carried out for many decades now we still have no proof that the supposed effect exists.

That's a different issue entirely

[Lothian]

Originally posted by davidsmith73
That's a different issue entirely Ummmm why ?

Surely the point you make is that this poor experiment which showed no effect is not typical of the many other good experiments which also show no effect.

[davidsmith73]

Originally posted by Lothian
Ummmm why ?

Surely the point you make is that this poor experiment which showed no effect is not typical of the many other good experiments which also show no effect.

The results of such experiments are not the issue of my concern here. Its the representation of the methodology used in telepathy research. In fact, no mention is made in the article of the fact that the methodological problems in such a non-scientific test have been adressed thoroughly in scientific tests of telepathy. I would not be surprised if readers not familiar with parapsychological research conclude otherwise after reading this article.

[Lothian]

Originally posted by davidsmith73
The results of such experiments are not the issue of my concern here. Its the representation of the methodology used in telepathy research. In fact, no mention is made in the article of the fact that the methodological problems in such a non-scientific test have been adressed thoroughly in scientific tests of telepathy. I would not be surprised if readers not familiar with parapsychological research conclude otherwise after reading this article. But surely it would be cruel to point out that the methodological problems in such a non-scientific test have been addressed thoroughly in scientific tests without also giving the results of those tests.

I am not disagreeing with your proposed addition but this isn’t a TV program where we leave the viewer with a teaser. This is about education and we need to let people know how far we have progresses in both the methodology of tests and in the results those tests achieved in the search for telepathic powers.

So far we have devised some really sound experiments that don’t allow room for error or cheating and they all show no effect.

[davidsmith73]

Originally posted by Lothian
But surely it would be cruel to point out that the methodological problems in such a non-scientific test have been addressed thoroughly in scientific tests without also giving the results of those tests.

I am not disagreeing with your proposed addition but this isn’t a TV program where we leave the viewer with a teaser. This is about education and we need to let people know how far we have progresses in both the methodology of tests and in the results those tests achieved in the search for telepathic powers.

So far we have devised some really sound experiments that don’t allow room for error or cheating and they all show no effect.

By all means include the results of proper scientific tests, but that issue deserves a separate article (as there further down in the Sceptic Report). Also, I completely disagree with you there. The results do show effects, just not enough to convince some people. I don't want to get into another debate about results. I just wanted to point out that its important to fully inform readers and leave no room for biased representation.

[Dr Adequate]

Originally posted by davidsmith73
By all means include the results of proper scientific tests, but that issue deserves a separate article
Er, yes.

So what do you want changing?

Suggest an extra paragraph or so that would make it un-"biased" according to you.

[davidsmith73]

Originally posted by Dr Adequate
Er, yes.

So what do you want changing?

Suggest an extra paragraph or so that would make it un-"biased" according to you.

Just a few lines would do. Something along the lines of - "although this test fails miserable on methodological grounds, the majority of scientific research into telepathy employs adequate controls in its experimental design"

Perhaps a link to the excellent article further down in the Sceptic Report - "The Current State of Parapsychology Research" by Scott Teresi

[Rob Lister]

Originally posted by davidsmith73
Just a few lines would do. Something along the lines of - "although this test fails miserable on methodological grounds, the majority of scientific research into telepathy employs adequate controls in its experimental design"

Perhaps a link to the excellent article further down in the Sceptic Report - "The Current State of Parapsychology Research" by Scott Teresi

Hey! Here's an idea. Why don't YOU write your own article and submit it to CFL for publication on his site. I think he [willing/happily] accepts well-written pieces from guest bloggers. Or, you could start your own site.

[CFLarsen]

Originally posted by davidsmith73
The results of such experiments are not the issue of my concern here. Its the representation of the methodology used in telepathy research. In fact, no mention is made in the article of the fact that the methodological problems in such a non-scientific test have been adressed thoroughly in scientific tests of telepathy. I would not be surprised if readers not familiar with parapsychological research conclude otherwise after reading this article.

First of all, readers not familiar with parapsychological research should immediately be informed that the whole field of parapsychology is not only littered with crooks, charlatans and con men, but also with an abundance of incompetent research. While brutal and uncomfortable to some, that is the harsh truth.

Second, it would be most incorrect to give readers not familiar with parapsychological research the impression that the "majority" (as you call it) of research is being done with sound methodology without also stating that this has resulted in rapidly disappearing effect sizes.

Third, why don't you write an article of your own and submit it to SkepticReport? Put your money where your mouth is, and see if your arguments hold up to scrutiny.

Originally posted by davidsmith73
Just a few lines would do. Something along the lines of - "although this test fails miserable on methodological grounds, the majority of scientific research into telepathy employs adequate controls in its experimental design"

Back this up with evidence, please.

Originally posted by davidsmith73
Perhaps a link to the excellent article further down in the Sceptic Report - "The Current State of Parapsychology Research" by Scott Teresi

Which was criticized for not being "current" enough... There's no pleasing some people! ;)

[Rob Lister]

Originally posted by CFLarsen
Third, why don't you write an article of your own and submit it to SkepticReport? Put your money where your mouth is, and see if your arguments hold up to scrutiny.

Gosh, I beat you to it again, sorta.

I must be on a run.

[davidsmith73]

Originally posted by CFLarsen
First of all, readers not familiar with parapsychological research should immediately be informed that the whole field of parapsychology is not only littered with crooks, charlatans and con men, but also with an abundance of incompetent research. While brutal and uncomfortable to some, that is the harsh truth.

"an abundance of incompetent research" is a claim that doesn't really cut it with me. If you're interested in the harsh truth then some kind of justification of this is needed. Peer reviewed articles with incompetent research please.


Second, it would be most incorrect to give readers not familiar with parapsychological research the impression that the "majority" (as you call it) of research is being done with sound methodology without also stating that this has resulted in rapidly disappearing effect sizes.

By all means state that, but also give proof of this claim. Prove that there is a correlation between improved study quality and disappearing effect sizes.


Third, why don't you write an article of your own and submit it to SkepticReport? Put your money where your mouth is, and see if your arguments hold up to scrutiny.

Firstly, it is you who is claiming that there is a correlation between the disappearance of effect sizes and methodology. My argument was simply that the article in your Sceptic Report does not inform the reader of the methodological quality of modern telepathy research, which it should in my opinion.


Secondly I don't think I need to write an article about your claim. Since the original article was on telepathy research we can quickly look at ganzfeld experiments, which I think are representative of the majority of telepathy experiments performed over the last 30 or so years.

(and here's me saying I don't want to get into a debate about the results ;) )

Pre-autoganzfeld experiments were reviewed by Hyman and Honorton. Hyman claimed he found a correlation between study quality and effect sizes while Honortan did not (1) This was probably down to their different criteria applied to the study quality (interestingly the independent reviewed agreed with Honorton). Because of this discrepency, Hyman and Honorton came up with an agreed set of improved protocols for the autoganzfeld studies. According to your claim this should have resulted in an immediate disappearance of the effect. This did not happen. The results were consistent with the pre-ganzfeld studies (2) More recently, a meta-analysis revealed that the overall effect disappeared with the addition of all most recent experiments at the time of writing (well almost all, Wiseman and Milton left out 10 experiments that, if they had been included, would have resulted in a significantly replicated effect size) (3). However this late disappearance was found to correlate with the degree of standardness of the ganzfeld protocol rather than the degree of study quality, since all the recent studies follow the same autoganzfeld procedures as recommended by Hyman and Honortan (4)

1. Honorton, C. (1985). Meta-analysis of psi ganzfeld re search: A response to Hyman. Journal of Para psychology, 49, 51–91.

2. Bem, D. J., & Honorton, C. (1994). Does psi exist? Replicable evidence for an anomalous process of information transfer. Psy chological Bulletin, 115, 4–18.

3. Milton, J. & Wiseman, R. (2001). Does Psi Exist? Reply to Storm and Ertel (2001). Psychological Bulletin, 127, 434-438.

4. Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218.



Back this up with evidence, please.

Of course. Just look at all experiments published in peer reviewed parapsychology journals. I could post all the methods sections here but it might take up a tad too much space

[Open Mind]

Originally posted by Lothian

So far we have devised some really sound experiments that don’t allow room for error or cheating and they all show no effect.

'All show no effect' ...... Yes if you dismiss every positive controlled trial result as fraud, error, lucky fluctuations, etc. ..... it also helps to have a large pile of nearby sand, a long neck and pointed head too. ;)

Personally I have a fairly pointed head, fairly long neck but I'm all out of sand .... so I think telepathy exists ... uselessly weak perhaps from practical purposes ... ... but it still exists IMO :) ....

.... and David is correct, that article is misleading, modern PSI trials have possibly the tightest controls of any area of research .....

[Stitch]

Originally posted by davidsmith73
Just a few lines would do. Something along the lines of - "although this test fails miserable on methodological grounds, the majority of scientific research into telepathy employs adequate controls in its experimental design"


Let me add:

"although this test fails miserable on methodological grounds, the <strike>majority of</strike> scientific research into telepathy that employs adequate controls in its experimental design shows no evidence of telepathy either"

[CFLarsen]

Originally posted by davidsmith73
"an abundance of incompetent research" is a claim that doesn't really cut it with me. If you're interested in the harsh truth then some kind of justification of this is needed. Peer reviewed articles with incompetent research please.

Look at the history of paranormal research: The experiments generally are full of design flaws. Radin, Schwartz, Rhine, Soal, Targ, Puthoff...need I go on?

Originally posted by davidsmith73
By all means state that, but also give proof of this claim. Prove that there is a correlation between improved study quality and disappearing effect sizes.

Excuse me? You claim to be knowledgable about paranormal research, and you are not aware of this?

Originally posted by davidsmith73
Firstly, it is you who is claiming that there is a correlation between the disappearance of effect sizes and methodology. My argument was simply that the article in your Sceptic Report does not inform the reader of the methodological quality of modern telepathy research, which it should in my opinion.

No, no, no...you are trying to turn the tables here. You show your evidence.

Originally posted by davidsmith73
Secondly I don't think I need to write an article about your claim. Since the original article was on telepathy research we can quickly look at ganzfeld experiments, which I think are representative of the majority of telepathy experiments performed over the last 30 or so years.

You "think"? :rolleyes:

Originally posted by davidsmith73
Pre-autoganzfeld experiments were reviewed by Hyman and Honorton. Hyman claimed he found a correlation between study quality and effect sizes while Honortan did not (1) This was probably down to their different criteria applied to the study quality (interestingly the independent reviewed agreed with Honorton). Because of this discrepency, Hyman and Honorton came up with an agreed set of improved protocols for the autoganzfeld studies. According to your claim this should have resulted in an immediate disappearance of the effect.

I see you misunderstand me, and I can understand why: When I said "disappearing", I should have said "diminishing".

Originally posted by davidsmith73
This did not happen. The results were consistent with the pre-ganzfeld studies (2)

Who, among skeptics, reviewed this?

Originally posted by davidsmith73
More recently, a meta-analysis revealed that the overall effect disappeared with the addition of all most recent experiments at the time of writing (well almost all, Wiseman and Milton left out 10 experiments that, if they had been included, would have resulted in a significantly replicated effect size) (3). However this late disappearance was found to correlate with the degree of standardness of the ganzfeld protocol rather than the degree of study quality, since all the recent studies follow the same autoganzfeld procedures as recommended by Hyman and Honortan (4)

Let me get this straight, OK? Are you saying that we can, today, perform a paranormal experiment that produces the same effect size, and that is not due to experimental flaws?

Because that's what I am reading into your statement here.

Originally posted by davidsmith73
Of course. Just look at all experiments published in peer reviewed parapsychology journals. I could post all the methods sections here but it might take up a tad too much space

No evidence, then.

Your complaint is therefore dismissed.

[T'ai Chi]

Originally posted by davidsmith73
Perhaps a link to the excellent article further down in the Sceptic Report - "The Current State of Parapsychology Research" by Scott Teresi

Isn't it odd that a paper that hasn't been updated since Jan. 2000 would be put in the Jan. 2004 issue?

The paper omits the RNG experiment results, things which Sagan even said deserve study:

"At the time of writing, there are three claims in the ESP field which, in my opinion, deserve serious study: (1) that by thought alone humans can (barely) affect random number generators in computers;..."
(Sagan, The Demon-Haunted World, 1995, p. 302)

[TLN]

Originally posted by jzs
"At the time of writing, there are three claims in the ESP field which, in my opinion, deserve serious study: (1) that by thought alone humans can (barely) affect random number generators in computers;..."
(Sagan, The Demon-Haunted World, 1995, p. 302)

You left this part out:

I pick these claims not because I think they're likely to be valid (I don't), but as examples of contentions that might be true.

[T'ai Chi]

TLN, I'm not sure what your point is.

Sagan, while personally thinking that they aren't real, realizes that they "deserve serious study".

To get back to the contents of the article, one that purports to show the current state in 2000, to omit these results is something of a mystery.

[TLN]

Originally posted by jzs
To get back to the contents of the article, one that purports to show the current state in 2000, to omit these results is something of a mystery.

Sure. Let me sum this up for everyone involved. You and others claim that there are serious parapsychological studies that show a reproducible effect. May I see one please, complete with testing protocol and raw data? Please don't throw out random names and places and tell me to "go look" or to "Google" such-and-such.

You say there are good studies. Show one.

[CFLarsen]

Originally posted by jzs
Isn't it odd that a paper that hasn't been updated since Jan. 2000 would be put in the Jan. 2004 issue?

The paper omits the RNG experiment results, things which Sagan even said deserve study:

"At the time of writing, there are three claims in the ESP field which, in my opinion, deserve serious study: (1) that by thought alone humans can (barely) affect random number generators in computers;..."
(Sagan, The Demon-Haunted World, 1995, p. 302)

Yes, T'ai Chi, we've been through this before. Sorry, you don't have a case.

And you selecting only the part of Sagan's quote....you really thought you would get away with that? :rolleyes:

BTW: How come you can take a neutral stance on paranormal matters (or at least claim that you do), when you demanded that people expressed either a pro-psi or con-psi stance, before they were allowed to sign up for your (botched) experiment?

[The Don]

If it is at all possible, studies cited should also have been successfully repeated by an independent, unbiased, set of researchers

[Lothian]

Originally posted by Open Mind
'All show no effect' ...... Yes if you dismiss every positive controlled trial result as fraud, error, lucky fluctuations, etc. ..... So far but I am sure you can point me in the direction of one that wasn't.

[TLN]

Originally posted by Lothian
So far but I am sure you can point me in the direction of one that wasn't.

Yeah, asking to actually see one of these mythical studies has really shut this tread up quick, huh?

[Dancing David]

Originally posted by davidsmith73
Just a few lines would do. Something along the lines of - "although this test fails miserable on methodological grounds, the majority of scientific research into telepathy employs adequate controls in its experimental design"

Perhaps a link to the excellent article further down in the Sceptic Report - "The Current State of Parapsychology Research" by Scott Teresi

What hoot! HOOT!

[Dancing David]

Originally posted by davidsmith73
"


Pre-autoganzfeld experiments were reviewed by Hyman and Honorton. Hyman claimed he found a correlation between study quality and effect sizes while Honortan did not (1) This was probably down to their different criteria applied to the study quality (interestingly the independent reviewed agreed with Honorton). Because of this discrepency, Hyman and Honorton came up with an agreed set of improved protocols for the autoganzfeld studies. According to your claim this should have resulted in an immediate disappearance of the effect. This did not happen. The results were consistent with the pre-ganzfeld studies (2) More recently, a meta-analysis revealed that the overall effect disappeared with the addition of all most recent experiments at the time of writing (well almost all, Wiseman and Milton left out 10 experiments that, if they had been included, would have resulted in a significantly replicated effect size) (3). However this late disappearance was found to correlate with the degree of standardness of the ganzfeld protocol rather than the degree of study quality, since all the recent studies follow the same autoganzfeld procedures as recommended by Hyman and Honortan (4)

1. Honorton, C. (1985). Meta-analysis of psi ganzfeld re search: A response to Hyman. Journal of Para psychology, 49, 51–91.

2. Bem, D. J., & Honorton, C. (1994). Does psi exist? Replicable evidence for an anomalous process of information transfer. Psy chological Bulletin, 115, 4–18.

3. Milton, J. & Wiseman, R. (2001). Does Psi Exist? Reply to Storm and Ertel (2001). Psychological Bulletin, 127, 434-438.

4. Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218.


Blah, blah blah, what a waste of space. The ganzfeld effct does not exist and if it does exist it is the product of so many confounding factors that psi is the least likely candidate.

You are really up a tree if you want to suggest that the ganzfeld experiments have any meaning. Unless you want to act like Amhearst and just assert that they have meaning.

Ganfeld effect discussed endlessly (http://forums.randi.org/showthread.php?s=&threadid=40737&highlight=ganzfeld)

If you read through this thread, you will find that there is a reasonable discussion of what the confounding effects are. If you don't have controls then you don't have an effect.

[T'ai Chi]

Originally posted by TLN
Sure. Let me sum this up for everyone involved. You and others claim that there are serious parapsychological studies that show a reproducible effect.

I didn't realize that "Sagan, while personally thinking that they aren't real, realizes that they 'deserve serious study'." implied all of that.

Edited to add an example of RNG experiment work that could have been discussed, if the author chose to do so, in his "Current State" paper from 2000: http://www.boundaryinstitute.org/articles/rngma.pdf

[CFLarsen]

Originally posted by jzs
I didn't realize that "Sagan, while personally thinking that they aren't real, realizes that they 'deserve serious study'." implied all of that.

But you were aware of the remainder of the quote?

How come you can take a neutral stance on paranormal matters (or at least claim that you do), when you demanded that people expressed either a pro-psi or con-psi stance, before they were allowed to sign up for your (botched) experiment?

[davidsmith73]

Originally posted by CFLarsen
Look at the history of paranormal research: The experiments generally are full of design flaws. Radin, Schwartz, Rhine, Soal, Targ, Puthoff...need I go on?


Yes, you need go on!

Radin - which experiments and which flaws?

Schwartz - ok you got one there!

Rhine - not really up to date, but go on, which experiments and which flaws?

Soal - experiments and flaws?

Targ and Puthoff - probably got another one there

So is that it? Out of all parapsychological experiments you have a handfull of examples of poor methodology (which you haven't yet given specifics about). Hardly an "abundance". I'm not denying that incompetent research exists but if you are seriously interested in the truth then you cannot claim that there is an abundance of it.

Lets take the ganzfeld experiments again. From Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218.

...and a hit rate of 32% (p = .0008) for 10 computer controlled (“autoganzfeld�) studies conducted between 1983 and 1989 that had been specifically designed to eliminate methodological flaws identified in some of the earlier studies....
...More recently, Milton and Wiseman (1999) published a follow-up
meta-analysis of 30 additional ganzfeld studies that had been conducted from 1987 through 1997....
...In addition to the 30 studies analyzed by Milton and Wiseman (1999),an additional 10 studies were located by examining the six major publication outlets for parapsychological research."

So we have 50 ganzfeld studies that have stringent methodological design in accordance with the guidelines set out bu Hyman and Honorton. Is this already many more than you can find that are incompetent? And that's just the ganzfeld studies.


Davidsmith73: Prove that there is a correlation between improved study quality and disappearing effect sizes.

Claus:Excuse me? You claim to be knowledgable about paranormal research, and you are not aware of this?

Answering a question with a question is not proving your claim. You claimed that there is a correlation between improved study quality and disappearing effect sizes. Now put your money where your mouth is and prove it.


I see you misunderstand me, and I can understand why: When I said "disappearing", I should have said "diminishing".

Ok, prove your claim that when research is done with sound methodology this has resulted in rapidly diminishing effect sizes.


Who, among skeptics, reviewed this?

R. Rosenthal



Let me get this straight, OK? Are you saying that we can, today, perform a paranormal experiment that produces the same effect size, and that is not due to experimental flaws?

No, I'm saying that to my knowledge there is no evidence that there is a correlation between diminishing effect sizes and study quality. Show me the evidence if you think otherwise. It's your claim



Your complaint is therefore dismissed.

(*creak* as Claus reclines into his well used armchair)

[CFLarsen]

Originally posted by davidsmith73
Yes, you need go on!

Why? You asked for examples, I gave them.

Originally posted by davidsmith73
Radin - which experiments and which flaws?

An evening with Dean Radin (http://www.skepticreport.com/psychics/radin2002.htm)
Tell me that Radin doesn't select his data.

Book Review: Dean Radin, "The Conscious Universe" (http://www.skepticreport.com/psychics/radinbook.htm)
Tell me that Radin doesn't select his data.

Originally posted by davidsmith73
Schwartz - ok you got one there!

Oh, yes.

Originally posted by davidsmith73
Rhine - not really up to date, but go on, which experiments and which flaws?

Gimmeabreak! There were virtually no controls! People could see through the cards, cards were reflected in glasses, pre-trials were allowed to be recorded as real ones...

Originally posted by davidsmith73
Soal - experiments and flaws?

Proven cheater.

Originally posted by davidsmith73
Targ and Puthoff - probably got another one there

Oh, yes.

Originally posted by davidsmith73
So is that it? Out of all parapsychological experiments you have a handfull of examples of poor methodology (which you haven't yet given specifics about). Hardly an "abundance". I'm not denying that incompetent research exists but if you are seriously interested in the truth then you cannot claim that there is an abundance of it.

A "handful"? Are you crazy? These are the All-Stars of parapsychology research, and their work is heavily flawed.

Originally posted by davidsmith73
So we have 50 ganzfeld studies that have stringent methodological design in accordance with the guidelines set out bu Hyman and Honorton. Is this already many more than you can find that are incompetent? And that's just the ganzfeld studies.

Do I understand you correctly? Can I - or anyone else - do an experiment that will produce evidence of a paranormal phenomenon? Just yes or no.

No, I am not interested in your explanations. All I want is a "Yes" or a "No".

Originally posted by davidsmith73
Answering a question with a question is not proving your claim. You claimed that there is a correlation between improved study quality and disappearing effect sizes. Now put your money where your mouth is and prove it.

I did.

Originally posted by davidsmith73
Ok, prove your claim that when research is done with sound methodology this has resulted in rapidly diminishing effect sizes.

I don't see any headlines. I don't see any extra news programs. I don't see any Nobel prizes. Where is this evidence of a replicable paranormal event?

Originally posted by davidsmith73
R. Rosenthal

....and??

Originally posted by davidsmith73
No, I'm saying that to my knowledge there is no evidence that there is a correlation between diminishing effect sizes and study quality. Show me the evidence if you think otherwise. It's your claim

No, no, no. You claim that there is no such diminishing effect. Show it, then.

The onus is on you.

Originally posted by davidsmith73
(*creak* as Claus reclines into his well used armchair)

It would be funny, if true. Since it isn't....

[Ersby]

So we have 50 ganzfeld studies that have stringent methodological design in accordance with the guidelines set out bu Hyman and Honorton. Is this already many more than you can find that are incompetent? And that's just the ganzfeld studies.

Strictly speaking, one of those experiments isn’t a ganzfeld study at all (the one using drum beats instead of white noise). So that should be 49.

We do not need to find incompetent studies (although that would be nice, and some would say that the 10 PRL trials offered avenues for sensory cues (sound leakage, video degredation) and so could be included in that category), but instead we need to find those experiments whose results cancel out this result. If you want to draw a line in the sand and say that we begin at the PRL experiments and Milton and Wiseman’s meta-analysis, that still only gives an average hit rate of 28% (including, of course, ganzfled studies until 20004).

Next point to add is that Rhine (certainly later in life, maybe earlier, but we can’t say for sure) had a policy at the Rhine Institute or repressing negative results, and this is why he and Honorton fell out (btw, the more I learn about Honorton, the more I like him) and prompted Honorton to establish a policy within the Parapsychological Association of reporting all results, good and bad.

It may also be worth adding that Charles Akers’ review of Hyman and Honorton’s debate came down nearer Hyman’s point of view regarding flawed studies. The fact that people tend to mention Rosenthal, but not Akers bothers me somewhat.

And there is a diminishing effect in the ganzfeld studies. The ealiest ganzfeld experiments are where you’ll find the best results. Dalton’s 1997 result is something of a freak, being the highest since 1979.

I still haven’t had a reply from Westerlund about his claims regarding Dalton’s work. Perhaps my hotmail address meant I got sent straight to his junk mail folder. I’ll try again in a week or so.

[davidsmith73]

Originally posted by CFLarsen
Why? You asked for examples, I gave them.

I asked for specific peer reviewed experimental papers, not just names.


An evening with Dean Radin (http://www.skepticreport.com/psychics/radin2002.htm)
Tell me that Radin doesn't select his data.

Where can I see the data that he has selected and the data he left out? And to be fair, your article looks like it was written on the basis of your memory of events.


Book Review: Dean Radin, "The Conscious Universe" (http://www.skepticreport.com/psychics/radinbook.htm)
Tell me that Radin doesn't select his data.

How many experiments has Radin contributed to the body of parapsychology that use selected data?



Gimmeabreak! There were virtually no controls! People could see through the cards, cards were reflected in glasses, pre-trials were allowed to be recorded as real ones...

I'll have to take your word for that. Also, experiments done in the 1930's are not representative of the methodology of modern parapsychology. Criticisms against these experiments were cruical in designing the well controlled experiments post-Rhine.



Proven cheater.

Which peer reviewed experiments has Soal published?



A "handful"? Are you crazy? These are the All-Stars of parapsychology research, and their work is heavily flawed.


Science does not function on status. You may think that these experiments constitute a relative "abundance" of research in parapsychology, but the truth is that many more experimenters exists that perform well conducted, well controlled experiments. You can examine the experiments for yourself if you were to divert your attention from the soft targets



Do I understand you correctly? Can I - or anyone else - do an experiment that will produce evidence of a paranormal phenomenon? Just yes or no.

My point is about methodology. You are trying to change the focus of this debate. Lets stay on topic.


No, I am not interested in your explanations. All I want is a "Yes" or a "No".

Claus, you have a habit of trying to change the subject. Methodology is my concern, not results.


I did.

You have not proven your claim that there is a correlation between disapp.. sorry, diminishing effect sizes and study quality. Try again.



I don't see any headlines. I don't see any extra news programs. I don't see any Nobel prizes. Where is this evidence of a replicable paranormal event?

Prove your claim that there is a correlation between disapp.. sorry, diminishing effect sizes and study quality.



....and??

You're the one who wanted to know who the indepedent reviewer of the Honorton and Hyman meta-analysis was!



No, no, no. You claim that there is no such diminishing effect. Show it, then.

The onus is on you.

Why is the onus on me when it was you who first claimed that:
"it would be most incorrect to give readers not familiar with parapsychological research the impression that the "majority" (as you call it) of research is being done with sound methodology without also stating that this has resulted in rapidly disappearing effect sizes"

So come on Claus, prove your assertion.

[davidsmith73]

Originally posted by Ersby

We do not need to find incompetent studies (although that would be nice, and some would say that the 10 PRL trials offered avenues for sensory cues (sound leakage, video degredation) and so could be included in that category), but instead we need to find those experiments whose results cancel out this result. If you want to draw a line in the sand and say that we begin at the PRL experiments and Milton and Wiseman’s meta-analysis, that still only gives an average hit rate of 28% (including, of course, ganzfled studies until 20004).

Is there a published meta-analysis that gives a 28% hit rate or is that your own analysis?

Also, my point is that there is no "abundance" of incompetent experiments as Claus claims. The article that appears in the Sceptic Report does not mention anything of the standard of modern telepathy experiments and could mislead uninformed readers. This is the standard of the Edinburgh ganzfeld protocol today - http://moebius.psy.ed.ac.uk/Ganzfeld_H.html#tg


Next point to add is that Rhine (certainly later in life, maybe earlier, but we can’t say for sure) had a policy at the Rhine Institute or repressing negative results, and this is why he and Honorton fell out (btw, the more I learn about Honorton, the more I like him) and prompted Honorton to establish a policy within the Parapsychological Association of reporting all results, good and bad.

Ok, and I think we have established that the ganzfeld experiments are a different kettle of fish from the telepathy "test" described in the Sceptic Report article.


It may also be worth adding that Charles Akers’ review of Hyman and Honorton’s debate came down nearer Hyman’s point of view regarding flawed studies. The fact that people tend to mention Rosenthal, but not Akers bothers me somewhat.

Do you have a reference for the Akers' review?


And there is a diminishing effect in the ganzfeld studies. The ealiest ganzfeld experiments are where you’ll find the best results. Dalton’s 1997 result is something of a freak, being the highest since 1979.

Is the difference between the pre-PRL and PRL effect sizes significant? The original pre-PRL studies had an effect size of 0.62, with a 95% confidence interval from 0.55 to .069. The PRL studies effect size was 0.59, with a 95% confidence interval from 0.53 to 0.64. Since the confidence intervals overlap we can't conclude that the difference in effect size is significant.

The Milton and Wiseman paper gives an effect size of 0.013 (different type of effect size - the PRL equivalent is 0.162) When an additional 10 studies were added to this analysis the effect size was 0.051 - larger but still significantly smaller than the PRL studies . However, the Bem, Palmer and Broughton paper shows that the effect size is correlated with adherence to the degree of "standardness". The further the experiments deviate from standard protocol the less the effect size becomes.

So we only have one significant decrease in effect size which is comparing PRL studies with those following, and this is shown to be more likely due to deviation from standard protocol, which is expected given the amount of process oriented experiments included in the later studies.

If the assertion that decreasing effect size is due to tightening of controls is true, you would expect a significant decrease between the pre-PRL studies and the PRL studies as well as between the PRL and post-PRL studies. This was not the case.

[Ersby]

Originally posted by davidsmith73
Is there a published meta-analysis that gives a 28% hit rate or is that your own analysis?


My own, though I wouldn't call it an analysis, since it's just a collection of results.


Also, my point is that there is no "abundance" of incompetent experiments as Claus claims. The article that appears in the Sceptic Report does not mention anything of the standard of modern telepathy experiments and could mislead uninformed readers. This is the standard of the Edinburgh ganzfeld protocol today - http://moebius.psy.ed.ac.uk/Ganzfeld_H.html#tg

Ok, and I think we have established that the ganzfeld experiments are a different kettle of fish from the telepathy "test" described in the Sceptic Report article.


I've not read the report. I don't think there's a great deal of incompetent experiment in the ganzfeld database, simply because not a great deal is needed to bring things back to chance (imo).


Do you have a reference for the Akers' review?


He writes about in the the Skeptics Handbook to Parapsychology. Prometheus Books, edited by Paul Kurtz.


Is the difference between the pre-PRL and PRL effect sizes significant? The original pre-PRL studies had an effect size of 0.62, with a 95% confidence interval from 0.55 to .069. The PRL studies effect size was 0.59, with a 95% confidence interval from 0.53 to 0.64. Since the confidence intervals overlap we can't conclude that the difference in effect size is significant.

The Milton and Wiseman paper gives an effect size of 0.013 (different type of effect size - the PRL equivalent is 0.162) When an additional 10 studies were added to this analysis the effect size was 0.051 - larger but still significantly smaller than the PRL studies . However, the Bem, Palmer and Broughton paper shows that the effect size is correlated with adherence to the degree of "standardness". The further the experiments deviate from standard protocol the less the effect size becomes.

So we only have one significant decrease in effect size which is comparing PRL studies with those following, and this is shown to be more likely due to deviation from standard protocol, which is expected given the amount of process oriented experiments included in the later studies.

If the assertion that decreasing effect size is due to tightening of controls is true, you would expect a significant decrease between the pre-PRL studies and the PRL studies as well as between the PRL and post-PRL studies. This was not the case. [/B]

This all depends on what effect size you're using, doesn't it?

Sorry for the brief responses. Have to run!

[CFLarsen]

davidsmith73,

I can see that no matter what I present, you will ignore it. I can understand that.

You also refuse to show just one experiment that will produce evidence of a paranormal phenomenon. I can understand that, too.

It should be easy, though, since you claim that these experiments show quite a different picture than previous research. But you won't. Or can't. You even refuse to write an article about it.

Same old manure. Same old smokescreen. Feh...

[davidsmith73]

Originally posted by CFLarsen
davidsmith73,

I can see that no matter what I present, you will ignore it. I can understand that.

Like I said, I do not deny that there is incompetent research in parapsychology. I do deny that there is an "abundance" of it. You have not shown that there is an "abundance" as indicated by your examples. I have suggested that the ganzfeld experiments are an example of the fact that parapsychology is not littered with incompetant research. The greater numbers of experiments with sound methodology in the ganzfeld field alone compared with the examples you gave indicate you are wrong.


You also refuse to show just one experiment that will produce evidence of a paranormal phenomenon. I can understand that, too.

Like I said, this particular debate is not about the evidence. Its about the methodology.


It should be easy, though, since you claim that these experiments show quite a different picture than previous research. But you won't. Or can't. You even refuse to write an article about it.

Are the ganzfeld experiments comparable to the card experiments of Rhine in terms of methodological flaws? I think not.
I've given you references that show the overall effect size of the PRL ganzfeld studies is not significantly different from the pre-PRL studies, previously shown to have methodological flaws. Also, when the more recent studies are analysed for degree of standardness, from Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218:

"This same outcome can be observed by defining as standard the 29 replications whose ratings fell above the midpoint of the scale (4) and defining as nonstandard the 9 replications that fell below the mid point (2 replications fell at the mid point): The standard replications obtain an overall hit rate of 31.2%, ES = .096, Stouffer Z = 3.49, p = .0002, one-tailed. In contrast, the nonstandard replications obtain an over all hit rate of only 24.0%, ES = –.10, Stouffer Z = –1.30, ns. The difference between the
standard and non standard replications is itself significant, U = 190.5, p =.020, one-tailed. Most importantly, the mean effect size of the standard replications falls within the 95% confidence inter vals of both the 39 preautoganzfeld studies and the 10 autoganzfeld studies summarized by Bem and Honorton (1994). In other words, ganzfeld studies that adhere to the standard ganzfeld protocol continue to replicate with effect sizes comparable with those obtained in previous studies."


Same old manure. Same old smokescreen. Feh...

Same old armchair dismissal

[Interesting Ian]

Originally posted by CFLarsen
Originally posted by davidsmith73
Rhine - not really up to date, but go on, which experiments and which flaws?


Larsen
Gimmeabreak! There were virtually no controls! People could see through the cards, cards were reflected in glasses, pre-trials were allowed to be recorded as real ones...


WOW! That's interesting Claus. Could see through the cards? Used poor quality cards did they? ;) Can you substantiate any of this? Or are you simply repeating what skeptics said at the time? I suspect the latter; but it is my understanding that it was just wild unsubstantiated speculation by skeptics. But prove me wrong.

[Interesting Ian]

Originally posted by CFLarsen
Originally posted by davidsmith73
No, I'm saying that to my knowledge there is no evidence that there is a correlation between diminishing effect sizes and study quality. Show me the evidence if you think otherwise. It's your claim



No, no, no. You claim that there is no such diminishing effect. Show it, then.

The onus is on you.


Er . .excuse me . .he did not. He said he wasn't aware of any evidence showing such a diminishing effect. But I'm sure he would be delighted to become acquainted with this evidence, as would I. Now you have claimed there is indeed evidence showing a diminishing effect. Do you have this evidence, or don't you have this evidence?

[CFLarsen]

Originally posted by davidsmith73
Like I said, I do not deny that there is incompetent research in parapsychology. I do deny that there is an "abundance" of it. You have not shown that there is an "abundance" as indicated by your examples. I have suggested that the ganzfeld experiments are an example of the fact that parapsychology is not littered with incompetant research. The greater numbers of experiments with sound methodology in the ganzfeld field alone compared with the examples you gave indicate you are wrong.

Show me a replicable Ganzfeld experiment that shows evidence of a paranormal phenomenon.

Originally posted by davidsmith73
Like I said, this particular debate is not about the evidence. Its about the methodology.

It is very much about the evidence.

Originally posted by davidsmith73
Are the ganzfeld experiments comparable to the card experiments of Rhine in terms of methodological flaws? I think not.

They are a brilliant example of how the effect diminishes when better controls are put in place.

Originally posted by davidsmith73
I've given you references that show the overall effect size of the PRL ganzfeld studies is not significantly different from the pre-PRL studies, previously shown to have methodological flaws. Also, when the more recent studies are analysed for degree of standardness, from Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218:

Yes, a lot of words. Show me a replicable Ganzfeld experiment that shows evidence of a paranormal phenomenon.

Originally posted by davidsmith73
Same old armchair dismissal

Show me a replicable Ganzfeld experiment that shows evidence of a paranormal phenomenon.

Everything else is merely a smokescreen.

[T'ai Chi]

Originally posted by Ersby

This all depends on what effect size you're using, doesn't it?


In your collecting of the results, which effect sizes are you looking at?

[TLN]

Originally posted by davidsmith73
I asked for specific peer reviewed experimental papers, not just names.

So did I. I'm still waiting.

David, can I please see one serious, well-conducted scientific study that shows a psi effect? The experiment should be complete with raw data, testing protocol, and wherever possible, replicated by an independent lab and scientists.

You claim this exists. Show it.

[Ersby]

Originally posted by jzs
In your collecting of the results, which effect sizes are you looking at?

effect size r and cohen's d.

Assuming my sums are correct, of course.

(NB, I'm pretty confident my sums regarding the overall hit rate are okay. Not so sure about effect sizes and the like.)

[Dancing David]

Originally posted by TLN

David, can I please see one serious, well-conducted scientific study that shows a psi effect? The experiment should be complete with raw data, testing protocol, and wherever possible, replicated by an independent lab and scientists.

You claim this exists. Show it.

Yes please!

[davidsmith73]

Originally posted by TLN
So did I. I'm still waiting.

David, can I please see one serious, well-conducted scientific study that shows a psi effect? The experiment should be complete with raw data, testing protocol, and wherever possible, replicated by an independent lab and scientists.

You claim this exists. Show it.

http://www.randi.org/vbulletin/showthread.php?s=&threadid=49167

Very interesting results here. Raw EEG data is too dense to put in a paper though.

[CFLarsen]

Originally posted by davidsmith73
http://www.randi.org/vbulletin/showthread.php?s=&threadid=49167

Very interesting results here. Raw EEG data is too dense to put in a paper though.

Does it show a psi effect, yes or no?

[deBergerac]

Originally posted by davidsmith73
Claus,
I have been browsing your Sceptic Report site and although I admire all efforts in bringing to light the rising tides of nonsense that's out there, I think that the article "Analysis of a telepathy test" by Per Johan Rasmark paints an incomplete picture of research into telepathy. Perhaps this distortion is inadvertant but I feel I should express my concerns. The author rightly states that the test described in the article is not scientific, nor was it intended to be. However this makes the test entirely unrepresentative of parapsychological research. What concerns me is the fact that no mention is made in the article of the fact that experiments by serious parapsychologists do not employ such sloppy methodology. Indeed, the author goes as far as to suggest the design of a better experiment than that described. An uninformed reader would not know that better experiments of the type the author suggests have been carried out for many decades now. Do you not think that it is your duty as a sceptic to inform the reader of this?


http://www.skepticreport.com/psychics/telepathy.htm

Even if I understand your critique I do not think it is justified. As you point out it is clearly stated that the test examined was not a scientific test. In fact scientific parapsychological research is not discussed at all in the article. The aim of the article was to show how unscientific tests can give an impression of a successful experiment even if this is not the case.
That it is possible to conduct scientific testing in parapsychological research I believe to be self-evident to readers of Sceptic Report. Perhaps I am wrong but I still do not think the article is misleading by not discussing the scientific research that exists. But that is, of course, my opinion and you are free to have another.

[davidsmith73]

Originally posted by CFLarsen
Does it show a psi effect, yes or no?

In my opinion, yes.

[Beady]

Originally posted by davidsmith73
In my opinion, yes.

So, it's a matter of opinion, rather than a demonstrable and irrefutable fact?

[davidsmith73]

Originally posted by deBergerac
Even if I understand your critique I do not think it is justified. As you point out it is clearly stated that the test examined was not a scientific test. In fact scientific parapsychological research is not discussed at all in the article. The aim of the article was to show how unscientific tests can give an impression of a successful experiment even if this is not the case.
That it is possible to conduct scientific testing in parapsychological research I believe to be self-evident to readers of Sceptic Report. Perhaps I am wrong but I still do not think the article is misleading by not discussing the scientific research that exists. But that is, of course, my opinion and you are free to have another.

I think that nowadays there is a grave problem with peoples expectation towards parapsychology. In my experience they tend to fall into two categories. One is the uncritical believer and the other is the uncritical dismisser. The believer will indiscriminately accept that all parapsychology experiments are proof of psi merely by virtue of their subject matter, without examining what the experiments are really showing. Likewise the uncritical dismisser will accept that all parapsychology experiments do not show "any shred of evidence" for psi, again merely by virtue of subject matter. This forum is rare in that, generally, the evidence is looked at and discussed. When articles, such as the one in Sceptic Report, examine such a poor test of telepathy and rightly dismiss it as crap but do not mention that proper tests exist, this will tend to reinforce the unciritcal dismisser's expectations about the standard of psi experiments. I see your point in that the article was not about scientific experiments, but I think that as sceptics we have a duty not to widen the divide between the uncritical believer and dismisser because the truth lies somewhere in between.

[davidsmith73]

Originally posted by Beady
So, it's a matter of opinion, rather than a demonstrable and irrefutable fact?

In my opinion its somewhat a matter of opinion ;)

[davidsmith73]

Originally posted by CFLarsen
Show me a replicable Ganzfeld experiment that shows evidence of a paranormal phenomenon.

Yet another shift away from the actual subject of this thread.

Do you agree that parapsychology is not "littered" with an "abundance" of incompetent research?


It is very much about the evidence.

No, I'm addressing the general opinion on the methodology of parapsychology. The evidence rightly deserves a separate thread.


They are a brilliant example of how the effect diminishes when better controls are put in place.

references? Also I don't think that applying your argument to research from the 1930's is very immpressive.


Yes, a lot of words.

Care to comment on those words?

Where is your proof that there is a correlation between diminishing effect sizes and study quality?

[CFLarsen]

Originally posted by davidsmith73
In my opinion its somewhat a matter of opinion ;)

So, there is no evidence? Only your opinion?

[CFLarsen]

Originally posted by davidsmith73
Do you agree that parapsychology is not "littered" with an "abundance" of incompetent research?

Parapsychology is littered with an abundance of incompetent research. That is a fact.

Originally posted by davidsmith73
No, I'm addressing the general opinion on the methodology of parapsychology. The evidence rightly deserves a separate thread.

We have had a plethora of threads where claims of evidence have been made. Still no evidence. You are, of course, free to open as many threads about evidence as you want, but if all you have is your opinion....that doesn't cut it here.

Originally posted by davidsmith73
references? Also I don't think that applying your argument to research from the 1930's is very immpressive.

How disingenious. I also have showed research from the past few years.

Originally posted by davidsmith73
Care to comment on those words?

No, I prefer to focus on evidence. After that is found, then we can talk.

Originally posted by davidsmith73
Where is your proof that there is a correlation between diminishing effect sizes and study quality?

Look at Ganzfeld, e.g. It started out with these fantastic results, but the effect is all but gone.

[davidsmith73]

Originally posted by CFLarsen
Parapsychology is littered with an abundance of incompetent research. That is a fact.

No, that's a statement you have not demonstrated to be true. The examples you gave are a very small proportion of the sound experiments that have been published. Try again.


We have had a plethora of threads where claims of evidence have been made. Still no evidence. You are, of course, free to open as many threads about evidence as you want, but if all you have is your opinion....that doesn't cut it here.

My opinion is that there is evidence. Fact's aren't written in stone. You don't seem to doubt your own opinion.



How disingenious. I also have showed research from the past few years.

Research that does not show evidence for a correlation between effect sizes and study quality.



No, I prefer to focus on evidence. After that is found, then we can talk.

You can prefer all you want. Since we are discussing the methodology of parapsychology, the evidence is irrelevant.


Look at Ganzfeld, e.g. It started out with these fantastic results, but the effect is all but gone.

I'll just repeat what I posted earlier since you have yet to respond to any of it.

Are the ganzfeld experiments comparable to the card experiments of Rhine in terms of methodological flaws? I think not.
I've given you references that show the overall effect size of the PRL ganzfeld studies is not significantly different from the pre-PRL studies, previously shown to have methodological flaws. Also, when the more recent studies are analysed for degree of standardness, from Bem, D.J, Palmer, J. and Broughton, R.S. (2001). Updating the Ganzfeld database: a victim of its own success? Journal of Parapsychology, 65, 207-218:

"This same outcome can be observed by defining as standard the 29 replications whose ratings fell above the midpoint of the scale (4) and defining as nonstandard the 9 replications that fell below the mid point (2 replications fell at the mid point): The standard replications obtain an overall hit rate of 31.2%, ES = .096, Stouffer Z = 3.49, p = .0002, one-tailed. In contrast, the nonstandard replications obtain an over all hit rate of only 24.0%, ES = –.10, Stouffer Z = –1.30, ns. The difference between the
standard and non standard replications is itself significant, U = 190.5, p =.020, one-tailed. Most importantly, the mean effect size of the standard replications falls within the 95% confidence inter vals of both the 39 preautoganzfeld studies and the 10 autoganzfeld studies summarized by Bem and Honorton (1994). In other words, ganzfeld studies that adhere to the standard ganzfeld protocol continue to replicate with effect sizes comparable with those obtained in previous studies."

[davidsmith73]

Originally posted by CFLarsen
So, there is no evidence? Only your opinion?

No, there is evidence, and that's my opinion

[Zep]

Originally posted by davidsmith73
...there is evidence... Would you care to share it with us all here?

He asked, hopefully.

[davidsmith73]

Originally posted by Zep
Would you care to share it with us all here?

He asked, hopefully.

I'm sorry I thought I already had, above with the ganzfeld stuff and here (again):

http://www.randi.org/vbulletin/showthread.php?s=&threadid=49167

and here:

http://a1162.fmg.uva.nl/research/PSI/PA2004org.html

These are the relavent papers :

"Anticipatory skin conductance responses: a possible example of decision augmentation theory"

"The precognitive habituation effect: an adaptation using spider stimuli" (not significant but suggestive)

"Physiological correlates of ESP: heart rate differences between targets and non-targets in clairvoyance and precognition forced choice tasks"

"Precognitive avoidance and precognitive déjß vu"

[CFLarsen]

Originally posted by davidsmith73
I'm sorry I thought I already had, above with the ganzfeld stuff and here (again):

No, you hadn't.

Originally posted by davidsmith73
"Anticipatory skin conductance responses: a possible example of decision augmentation theory"

"possible example". Not "evidence".

Originally posted by davidsmith73
"The precognitive habituation effect: an adaptation using spider stimuli" (not significant but suggestive)

"Not significant, but suggestive". Not "evidence".

Originally posted by davidsmith73
"Physiological correlates of ESP: heart rate differences between targets and non-targets in clairvoyance and precognition forced choice tasks"

Pictures represented coloured calm images, i.e. landscapes, plants, flowers, portraits. Their degree of emotionality was measured asking to ten independent judges to rate each picture on a ten points scale from 0 (no emotion) to 10 (high level of emotion).

Give me a friggin' break. They have no idea how the participants "rated" the images. How can they know if one participant rates a picture 0 and another participant gives it a straight 10? The data from the heart rates are completely separated from what pictures were shown!

If some artefacts were noticed (for instance anomalous heart rate registrations or apparatus malfunctioning), the task was interrupted and restarted again.

You shittin' me? Do you know what they are doing? They take the data that doesn't fit (how the heck do they know that an "anomalous" heart rate isn't evidence of psi?) and simply throw it out!

Even if the raw difference is very low, less than one heart rate bit per second on average, it seems quite reliable as demonstrated by the analysis of our data.

This means it was extremely difficult to determine if the heart rate was high or not.

The difference between the clairvoyance and the precognition experiment seems not very reliable, even is in the second condition the heart rate difference between targets and non targets appears less evident.

Oops.

However, in both experiments, at the overt cognitive level, the means of hits is close to chance.

Oops.

The main hypothesis was that heart rate could change according to the categories of pictures, targets vs. non targets. The direction of this difference was not predictable in advance because, at our knowledge, there are not similar evidences in literature.

So, they are doing a brand new experiment, instead of trying to replicate an old one. Why? (We know why, to secure funding and having to avoid that they haven't replicated anything)

Originally posted by davidsmith73
"Precognitive avoidance and precognitive déjß vu"

The PH hypothesis is that the repeated exposures of the target can reach back in time to diminish the arousing targets less positive.

No: The more we see something that offends us, the less agitated we get. The HP hypothesis is simply not valid.

Erotic and positive (nonerotic) pictures are not yet showing any systematic patterns.

Whoa...this is contradicted by the previous experiment!

A pilot version of this procedure was administered to 20 participants in a weekend conference at the Institute of Noetic Sciences. They showed a significant PDV effect, especially those who were high on the personality trait of Openness to Experience.

Hardly an unbiased group of people, in an unbiased environment!

Who is the "skeptical investigator" who "independently" replicated the result? He/she is never named, nor his result. Why not? (I think I know why...)

Sorry, but I am still not seeing any evidence of a paranormal phenomenon anywhere.

[davidsmith73]

Originally posted by CFLarsen


"possible example". Not "evidence".

a possible example of their theory. The anomalous psi effect is still there regardless.



Give me a friggin' break. They have no idea how the participants "rated" the images. How can they know if one participant rates a picture 0 and another participant gives it a straight 10?

If someone rates a picture as higly emotional then it will be reflected in the normal change in heart rate after the picture is shown. If this is the case, how is this a flaw? Under the null hypothesis you still would not expect a pre-stimulus response if the target was emotional or calm.


You shittin' me? Do you know what they are doing? They take the data that doesn't fit (how the heck do they know that an "anomalous" heart rate isn't evidence of psi?) and simply throw it out!

No, they were looking to see if obvious artifacts were being produced due to faulty recording. You don't just keep every bit of data you record without removing instances of equipment failure. I'm just imagining what you would be saying if they kept the obvious artifacts!



This means it was extremely difficult to determine if the heart rate was high or not.

Don't know what you mean



Oops.

the precognition condition was still significant.



Oops.

Your point being? This was an experiment with evidence for physiological correlates of psi.



So, they are doing a brand new experiment, instead of trying to replicate an old one. Why? (We know why, to secure funding and having to avoid that they haven't replicated anything)


Not relavent to whether or not the experiment shows evidence.



No: The more we see something that offends us, the less agitated we get. The HP hypothesis is simply not valid.


Erm Claus, that's what the HP hypothesis is essentially based on. Wipe the froth away and think about what you are posting next time.


Whoa...this is contradicted by the previous experiment!

Lets have the full quote shall we:
At this point, it appears that there are systematic individual differences: Those high in anxiety show the predicted effects on the negative trials, but those high in sensation seeking show the reverse effect, significantly exposing themselves to the negatively arousing images. Erotic and positive (nonerotic) pictures are not yet showing any systematic patterns. At the moment, there are too few sessions to be confident of these patterns, but there does appear to be precognitive responding with this protocol.



Hardly an unbiased group of people, in an unbiased environment!

What are you saying here? Is the psychological characteristics of the participants a flaw in the methods? If so why?


Who is the "skeptical investigator" who "independently" replicated the result? He/she is never named, nor his result. Why not? (I think I know why...)

Replications are underway. You'll have to wait.

[davidsmith73]

Claus, do you have evidence for your claim that there is a correlation between diminishing effect size and study quality?

[Open Mind]

Originally posted by CFLarsen

Hardly an unbiased group of people, in an unbiased environment!

Who is the "skeptical investigator" who "independently" replicated the result? He/she is never named, nor his result. Why not? (I think I know why...)

Sorry, but I am still not seeing any evidence of a paranormal phenomenon anywhere.

And ……

Originally posted by TLN
replicated by an independent lab and scientists.


So those getting positive results are biased? What are you suggesting a conspiracy to produce fraudulent PSI? :) Or are you just accusing them of unconcious zombie fraud? Or is it just lucky incompetence?

Who decides who is 'independent' or unbiased? For example the critique of Ganzfeld experiments were largely by professional skeptic/psychologist Ray Hyman of CSICOP ...... and the later selective meta-analysis of Ganzfeld trials was by professional media skeptic/psychologist Richard Wiseman of CSICOP/skeptical inquirer ......... CSICOP is a political organization to debunk claims of the paranormal and defend conventional science, it’s hardly a neutral opinion.

I doubt there is great point in presenting evidence here (Amherst did it, other have too) , all that happens is the typical CSICOP counterclaims get quoted. Remove these counterclaims and is the skeptic argument as strong?

[CFLarsen]

Originally posted by davidsmith73
a possible example of their theory. The anomalous psi effect is still there regardless.

Oh, no, no, no...don't start with this "anomalous" crap. You cannot call possible experimental flaws for "psi". Flaws have nothing to do with the paranormal.

Either there is an effect, or there is not.

Originally posted by davidsmith73
If someone rates a picture as higly emotional then it will be reflected in the normal change in heart rate after the picture is shown. If this is the case, how is this a flaw? Under the null hypothesis you still would not expect a pre-stimulus response if the target was emotional or calm.

You don't understand: The people who were rating the pictures during the experiment did not know what the scale was. It's equivalent to asking you to grade someone's exam, but not telling you how to grade it.

Originally posted by davidsmith73
No, they were looking to see if obvious artifacts were being produced due to faulty recording. You don't just keep every bit of data you record without removing instances of equipment failure. I'm just imagining what you would be saying if they kept the obvious artifacts!

Please explain how these faulty recordings were not due to psi. If you cannot, then you have no case.

Originally posted by davidsmith73
Don't know what you mean

The difference between a "hit" and a "miss" was simply too small to be determined objectively.

Originally posted by davidsmith73
the precognition condition was still significant.

Don't play wordgames: Was there evidence of precognition? If so, where?

Originally posted by davidsmith73
Your point being? This was an experiment with evidence for physiological correlates of psi.

Cart before the horse. We have yet to see the existence of psi.

Originally posted by davidsmith73
Not relavent to whether or not the experiment shows evidence.

But the experiment does not show evidence.

Originally posted by davidsmith73
Erm Claus, that's what the HP hypothesis is essentially based on. Wipe the froth away and think about what you are posting next time.

So, how can we determine the difference between the HP hypothesis and the fact that we get used to seeing bad pictures, so they upset us less?

Originally posted by davidsmith73
Lets have the full quote shall we:
At this point, it appears that there are systematic individual differences: Those high in anxiety show the predicted effects on the negative trials, but those high in sensation seeking show the reverse effect, significantly exposing themselves to the negatively arousing images. Erotic and positive (nonerotic) pictures are not yet showing any systematic patterns. At the moment, there are too few sessions to be confident of these patterns, but there does appear to be precognitive responding with this protocol.

Doesn't help. It still contradicts the previous experiment.

Originally posted by davidsmith73
What are you saying here? Is the psychological characteristics of the participants a flaw in the methods? If so why?

I am saying that conducting a psi experiment at INS with believers is hardly the most reliable way to do it.

Originally posted by davidsmith73
Replications are underway. You'll have to wait.

Jeebus creebus, we have heard this for so long! How long can it take to replicate just one of those experiments that are claimed to show evidence of psi?

This doesn't happen - instead, we see yet another experiment, with a slightly changed protocol, invariably claiming "new" evidence of psi. And then, another. And then, another.

Just replicate it. Why is that so hard?

Oh, and...who was this skeptic, and what result did he get?

[CFLarsen]

Originally posted by Open Mind
So those getting positive results are biased? What are you suggesting a conspiracy to produce fraudulent PSI? :) Or are you just accusing them of unconcious zombie fraud? Or is it just lucky incompetence?

We know that believers have a wish to see their beliefs come true. An example is how sitters try desperately to fit the psychic's coldreading to their own situation. We also know that the field of parapsychology is infested with cheating and fraudulent behavior.

Originally posted by Open Mind
Who decides who is 'independent' or unbiased? For example the critique of Ganzfeld experiments were largely by professional skeptic/psychologist Ray Hyman of CSICOP ...... and the later selective meta-analysis of Ganzfeld trials was by professional media skeptic/psychologist Richard Wiseman of CSICOP/skeptical inquirer ......... CSICOP is a political organization to debunk claims of the paranormal and defend conventional science, it’s hardly a neutral opinion.

The data is there for all to see. It isn't a question of opinion, but of whether or not there is evidence. So far, there is none.

Originally posted by Open Mind
I doubt there is great point in presenting evidence here (Amherst did it, other have too) , all that happens is the typical CSICOP counterclaims get quoted. Remove these counterclaims and is the skeptic argument as strong?

There is a very great point in presenting evidence here: You have a host of people, genuinely interested, who are more than willing to see if the results hold up to scrutiny.

If not here, where, then? A believer's forum?

[dharlow]

Originally posted by CFLarsen
We know that believers have a wish to see their beliefs come true.

This can cut both ways, though. Goodstein and Brazis mailed abstracts of an astrological study to a random sample of psychologists. They varied the abstracts in only one respect...half reported positive findings while half reported negative findings. In other words, it was the same design with only the result being changed. The psychologists were asked to rate the studies (study really), and it was found that those who received the "negative" results rated the studies as far better designed, more valid, and having more adequate conclusions than those that rated the "positive" studies.

Goodstein , L.D. and Brazis, K.L. Psychology of the Scientist:XXX. Credibility of psychologists: An Empirical Study. Psychological Reports, 1970, 27, 835-838.

[Dr Adequate]

It doesn't just work like that with parapsychology, either. I read recently of a paper submitted for review to a large number of reviewers, with the results and conclusions tailor-made to be in line with or opposed to the known beliefs of the referees. They differed, as you'd expect, about the worth of the paper, but also the experimenters realised after they sent it out that they'd left an absolute howler in the paper. Guess which referees were most likely to spot it?

Everyone's biased and no-one's objective. That's why we have the scientific method.

[Dancing David]

Davidsmith73,

I am very open to the possibility that psi might exist, I think it would be cool if it did.

But the ganzfeld effect studies are seriously flawd in ways that the ,eta analysis can not compensate for, there is the possibility of confounding effects that are totaly unmeasured and un-controlled for. I don't know of any other area of social science where metaanalysis can compensate for flawed methdology.

I read the paper on the photoic stimulation that you presented in the More Evidence thread and it suffers from a serious problem,

The standard deviation for the sample is higher than the effect size. Therefore all the supposed effect could be attributed to the random variation of the sample during the test period. This is especialy relevant because of the small sample size.

I found the 'linked biological' theory very interesting, but again if the standard deviation is higher than the effect size, then, well it could all be sampling error.

If you could show some more reasonable effect size in relation to standard deviation then I might accept the effect. There is also the issue of removing overlapping cycles from the sampole, which means that the sampling of the 'photoic stimulation epochs' is not random and excludes all the samples that might occur during overlapping times periods. This is a serious sampling flaw. There is alos the elimination of anomolous data from teh data sets. i think that those should have been examined for random variation.

I see no reason to believe that there is an effect in that paper. There are some flaws in the method and the analysis that do not even make it suggestive until there is replication.

(This is actualy a real problem in the social sciences in general, most studies show only suggestive effects that are unreplicated. And researcehers are loathe to go and actualy do the replication.)

[Open Mind]

Originally posted by CFLarsen
We know that believers have a wish to see their beliefs come true. An example is how sitters try desperately to fit the psychic's coldreading to their own situation.


I agree keen believers do remember hits better than misses ….. what also must be recognized is that a keen skeptic will remember a misses better than hits.

If it possible for such positive bias to leak into a tightly controlled positive PSI trial, we must consider negative expectation of a professional skeptic could leak into a controlled no PSI result. ……

….Yes, I am concerned about closed minded skeptic researchers (who feel they already know the answer before the trial) assuming their attitude and careless choice of experimenters cannot possibly have an effect upon results.

IMO ...... all PSI researchers and experimenters in any PSI trial should by requirement sign their personal opinion/feeling/belief on the existence of PSI before any trial conducted……where this might seem an absurd or unnecessary suggestion to many skeptics, it is essential in my opinion. These trials are on the transfer/reception of thought and to assume only the thoughts of the those under test have any influence shows blinkered lack of theoretical reasoning or a prior disrespect for the concept thought transfer effects could be affected by all observers or expectations

I’d like to see tests involve natural psychics who have claimed evidential experiences since early childhood and are still convinced of the genuineness of their experiences, how many trials have used such obvious a choice? None I know of …… instead often researchers have in past used university students who are probably studying psychology, science or something materialistic. The cynical researcher might give the impression PSI doesn’t exist ‘let’s repeat this experiment to see what they did wrong’ and the expectation of those under tests follow on in monotonous drudge towards failure. If PSI exists it is something at the edge of human ability, it requires enthusiasm .


The data is there for all to see. It isn't a question of opinion, but of whether or not there is evidence.

I don’t think so. Hyman and Utts didn’t agree on the interpretation of CIA remote viewing data. In another trial on animal telepathy Sheldrake claimed positive trial, Wiseman repeated and claimed negative trial …..but their data was basically the same.

People choose their preferred interpretation of the data ..
’ There are some myths about science and scientists that need to be dispelled. Science gets mistaken as a body of knowledge for its method. Scientists are regarded as having superhuman abilities of rationality inside objectivity. Many studies in the psychology of science, however, indicate that scientists are at least as dogmatic and authoritarian, at least as foolish and illogical as everybody else, including when they do science. In one study on falsifiability, an experiment was described, an hypothesis was given to the participants, the results were stated, and the test was to see whether the participants would say, "This falsifies the hypothesis". The results indicated denial, since most of the scientists refused to falsify their hypotheses, sticking with them despite a lack of evidence! Strangely, clergymen were much more frequent in recognizing that the hypotheses were false.
http://www.fiu.edu/~mizrachs/truzzi.html

I think Truzzi is referring to Mahoney’s research of some years ago. ….it shows what can happen.



There is a very great point in presenting evidence here: You have a host of people, genuinely interested, who are more than willing to see if the results hold up to scrutiny.



OK, I will look forward to debating newer research in future, as it arrives.

[T'ai Chi]

Originally posted by Open Mind
I don’t think so. Hyman and Utts didn’t agree on the interpretation of CIA remote viewing data. In another trial on animal telepathy Sheldrake claimed positive trial, Wiseman repeated and claimed negative trial …..but their data was basically the same.


Very true. Even more, is that things are very rarely settled once and for all by one experiment (no matter how well designed). It is the cumulative evidence (or lack of) that eventually decides the matter.

[Lothian]

Originally posted by davidsmith73
This is the standard of the Edinburgh ganzfeld protocol today - http://moebius.psy.ed.ac.uk/Ganzfeld_H.html#tg
While I fail to see why the receiver can not pick the sent image rather than the experimenter I think the procedures here look ok.

I had a look on the Edinburgh site for papers on the results obtained using this procedure. The only paper mentioning Ganzfeld (http://moebius.psy.ed.ac.uk/Research/Telepathy_Results.html) concluded

4. Summary of findings
· No evidence for telepathy….
· Opposite to prediction, there was an slightly inverse relationship between telepathy success and reactivity to magnetic field

Has an experiment ever been done using this procedure at Edinburgh and what were the results ?

[CFLarsen]

Originally posted by Open Mind
I agree keen believers do remember hits better than misses ….. what also must be recognized is that a keen skeptic will remember a misses better than hits.

This is not correct. Skeptics are more prone to remember the correct ratio.

Originally posted by Open Mind
If it possible for such positive bias to leak into a tightly controlled positive PSI trial, we must consider negative expectation of a professional skeptic could leak into a controlled no PSI result. ……

But skeptics don't - by definition - have a negative expectation. Skeptics expect nothing - skeptics judge from the data alone.

Originally posted by Open Mind
….Yes, I am concerned about closed minded skeptic researchers (who feel they already know the answer before the trial) assuming their attitude and careless choice of experimenters cannot possibly have an effect upon results.

Whaa?? You have to exemplify this. Let's see some evidence.

Originally posted by Open Mind
IMO ......

I would prefer evidence.

Originally posted by Open Mind
all PSI researchers and experimenters in any PSI trial should by requirement sign their personal opinion/feeling/belief on the existence of PSI before any trial conducted……where this might seem an absurd or unnecessary suggestion to many skeptics, it is essential in my opinion. These trials are on the transfer/reception of thought and to assume only the thoughts of the those under test have any influence shows blinkered lack of theoretical reasoning or a prior disrespect for the concept thought transfer effects could be affected by all observers or expectations

We have seen here, on this board, that people will lie about their beliefs in the paranormal. Such signed confessions are worthless. What we should do is focus on the data. The experimental design. The protocols.

Originally posted by Open Mind
I’d like to see tests involve natural psychics who have claimed evidential experiences since early childhood and are still convinced of the genuineness of their experiences, how many trials have used such obvious a choice? None I know of

Schwartz used only well-known psychics. Targ & Puthoff used Geller.

But you have a point: These self-proclaimed psychics are notoriously hard to drag into the lab. One can wonder why...

Originally posted by Open Mind
…… instead often researchers have in past used university students who are probably studying psychology, science or something materialistic. The cynical researcher might give the impression PSI doesn’t exist ‘let’s repeat this experiment to see what they did wrong’ and the expectation of those under tests follow on in monotonous drudge towards failure. If PSI exists it is something at the edge of human ability, it requires enthusiasm.

So, you want to prevent skeptics - whom you say have a negative attitude towards the paranormal, and instead focus on those who has "enthusiasm"? You are advocating that people should have a positive attitude?

If psi exists, then it is not at the edge of human ability: It is quite contrary a huge part of reality, judging from the sheer number of psychics, healers, tarot-readers, astrologers, etc.

Originally posted by Open Mind
I don’t think so. Hyman and Utts didn’t agree on the interpretation of CIA remote viewing data. In another trial on animal telepathy Sheldrake claimed positive trial, Wiseman repeated and claimed negative trial …..but their data was basically the same.

What about their results? Do you claim that Wiseman's results showed evidence of psi?

It isn't all up to opinion: If that was the case, then why bother go into a lab at all?

Originally posted by Open Mind
People choose their preferred interpretation of the data ..

Some people might do that. The rest of us don't.

Originally posted by Open Mind
I think Truzzi is referring to Mahoney’s research of some years ago. ….it shows what can happen.

Sure, nobody denies it. So, what do we do? Throw our hands into the air and say "There's no way we can know, so let's all have our own, personal idea of what is real"?

Originally posted by Open Mind
OK, I will look forward to debating newer research in future, as it arrives.

Will you also be prepare to draw the consequences, if the results turn out to be negative?

[davidsmith73]

Originally posted by CFLarsen
Oh, no, no, no...don't start with this "anomalous" crap. You cannot call possible experimental flaws for "psi". Flaws have nothing to do with the paranormal.

Psi is defined to be anomalous results and as such there is evidence for such anomaly. If there are flaws that are producing the effect then I would sure like to hear about them, and prove that they are responsible. Otherwise we still have "psi" based on an an anomalous infomation aquisition.


Either there is an effect, or there is not.

There is. And it's an anomalous effect.


You don't understand: The people who were rating the pictures during the experiment did not know what the scale was. It's equivalent to asking you to grade someone's exam, but not telling you how to grade it.

I understand what you are saying. I don't understand how this is a flaw that will affect the hypothesis?


Please explain how these faulty recordings were not due to psi.

because they knew what the source of the artifacts were.


The difference between a "hit" and a "miss" was simply too small to be determined objectively.

How do you figure that? If the difference between the heart rates is very small then you use many trials.


Don't play wordgames: Was there evidence of precognition? If so, where?

In the statistically signifianct results. Anomalous precognition refers to the conditions within which the anomalous effects are observed, i.e, a response before the stimulus has been selected.



Cart before the horse. We have yet to see the existence of psi.

Here, there is evidence of the anomalous aquisition of a physiological response to remote stimuli.


But the experiment does not show evidence.

You have not explained why


So, how can we determine the difference between the HP hypothesis and the fact that we get used to seeing bad pictures, so they upset us less?

For the precognitive habituation experiments, the participants are simultaneously shown two two negatively arousing pictures (violence) and they are asked to chose which of the pair they prefer. After they chose, one of the pair of pictures is randomly chosen by the computer and subliminally presented to them. According to the PH hypothesis the subject should prefer the subliminally presented picture more often. In trials with positively arousing pictures, they should choose the picture which is not subliminally presented.



Doesn't help. It still contradicts the previous experiment.

Really? how so?



I am saying that conducting a psi experiment at INS with believers is hardly the most reliable way to do it.

Why on earth not? As long as the experimenter remains objective and sceptical, using "believers" should pose no problems at all. In fact it should be encouraged in order to found out if this is a factor in performance.



Jeebus creebus, we have heard this for so long! How long can it take to replicate just one of those experiments that are claimed to show evidence of psi
This doesn't happen - instead, we see yet another experiment, with a slightly changed protocol, invariably claiming "new" evidence of psi. And then, another. And then, another.

Not really. We see experiments that replicate previous findings as well as exploring new parameters in the experiment.


Oh, and...who was this skeptic, and what result did he get?

Which sceptic? where?

[davidsmith73]

Claus, do you have evidence for your claim that there is a correlation between diminishing effect size and study quality?

[Ersby]

Originally posted by Open Mind
I doubt there is great point in presenting evidence here (Amherst did it, other have too) , all that happens is the typical CSICOP counterclaims get quoted. Remove these counterclaims and is the skeptic argument as strong?

I think it may be stronger. Perhaps I'm going mad, but most of CSICOP's arguments are based on discrediting the meta-analyses without botheringto check if the data itself is correct. Honorton's oft quoted gnzfeld m-a relies on 28 pre-1981 ganzfeld experiments. I know of over 50. Do you think the results are the same with 28 or 50 experiments? If so, why?

There's no conspiracy.

Just a general reliance on what other people have said about what other people did.

As for the debate about effect size and study quality, that depends on what you consider to be "quality". If you think that ganzfeld is worse than autoganzfeld which is worse than digital ganzfeld, then yes, there is a demonstrable decline in effect size.

[davidsmith73]

Originally posted by Dancing David
Davidsmith73,

I am very open to the possibility that psi might exist, I think it would be cool if it did.

But the ganzfeld effect studies are seriously flawd in ways that the ,eta analysis can not compensate for,

What flaws?


there is the possibility of confounding effects that are totaly unmeasured and un-controlled for.

True. However one can never rule out the possibility that you haven't controlled for effects you don't know about, in any experiment. You can only reduce the likely hood of this as much as possible.


I don't know of any other area of social science where metaanalysis can compensate for flawed methdology.

If indeed the meta-analysis is compensating for flawed methodology. I do think that meta-analysis is far from the ideal method of demonstrating a replicable effect. Unfortunately, this is the situation at the moment for ganzfeld. The problem can be summed up nicely by Bem and Honortan:

"Given its larger effect size, the prospects for successfully replicating the psi ganzfeld effect are not quite so daunting, but they are probably still grimmer than intuition would suggest. If the true hit rate is in fact about 34% when 25% is expected by chance, then an experiment with 30 trials (the mean for the 28 studies in the original meta-analysis) has only about 1 chance in 6 of finding an effect significant at the .05 level with a one- tailed test. A 50-trial experiment boosts that chance to about 1 in 3. One must escalate to 100 trials in order to come close to the break even point, at which one has a 50-50 chance of finding a statistically significant effect (Utts, 1986). (Recall that only 2 of the 11 autoganzfeld studies yielded results that were individually significant at the conventional .05 level.) Those who require that a psi effect be statistically significant every time before they will seriously entertain the possibility that an effect really exists know not what they ask." (from Bem, Daryl J. and Honorton, Charles (1994). Does Psi Exist? Replicable Evidence for an Anomalous Process of Information Transfer, Psychological Bulletin, 115(1): 4-18 )



I read the paper on the photoic stimulation that you presented in the More Evidence thread and it suffers from a serious problem,

The standard deviation for the sample is higher than the effect size. Therefore all the supposed effect could be attributed to the random variation of the sample during the test period. This is especialy relevant because of the small sample size.

I think you mean to say that the SD is higher than the mean. They use a Wilcoxon signed rank test which is a distribution free method of looking at the difference between two means within the same population. This test is appropriate for small sample sizes.


I found the 'linked biological' theory very interesting, but again if the standard deviation is higher than the effect size, then, well it could all be sampling error.

I don't think they conducted any statistics on the spatial distribution, so that's a possibility. Hopefully the follow up experiment will include a large enough sample.


There is also the issue of removing overlapping cycles from the sampole, which means that the sampling of the 'photoic stimulation epochs' is not random and excludes all the samples that might occur during overlapping times periods. This is a serious sampling flaw.

I don't see why this means the photic stimulation sample is not random. Since both the original stimulation marks and the control marks were both randomly generated, then the samples taken out of the photic stimulation epochs will be random. If you take a random sample from a random sample, you are still left with a random sample are you not?


There is alos the elimination of anomolous data from teh data sets. i think that those should have been examined for random variation.

Do you mean the artifact removal? That's not anomalous data since the source of artifact is known - equipment failure. I believe this is standard practice in EEG analysis.


I see no reason to believe that there is an effect in that paper. There are some flaws in the method and the analysis that do not even make it suggestive until there is replication.



The paper supposedly replicates previous results:

Wackermann, J., Seiter, C., Keibel, H., & Walach, H. (2003). Correlations between brain electrical activities of two
spatially separated human subjects. Neuroscience Letters, 336, 60-64.

I haven't read this paper yet

[davidsmith73]

Originally posted by Lothian
While I fail to see why the receiver can not pick the sent image rather than the experimenter I think the procedures here look ok.


The receiver picks the sent image?! Was that a mistype?

(edited)
I think I see that you mean the receiver does the judging yes? In some of the cases they will (it's mentioned on the webpage)


I had a look on the Edinburgh site for papers on the results obtained using this procedure. The only paper mentioning Ganzfeld (http://moebius.psy.ed.ac.uk/Research/Telepathy_Results.html) concluded

Has an experiment ever been done using this procedure at Edinburgh and what were the results ?

I don't know if there are any successful experiments done there with the fully updated methods. I'll have a trawl through the available papers

[Lothian]

Originally posted by davidsmith73
The receiver picks the sent image?! Was that a mistype?

(edited)
I think I see that you mean the receiver does the judging yes? In some of the cases they will (it's mentioned on the webpage) Yes that is what I meant, Sorry I missed it.


I don't know if there are any successful experiments done there with the fully updated methods. I'll have a trawl through the available papers Thanks, but it would be interesting to know about all the experiments (rather than just the successes:D ). I can't believe they went to the trouble of designing experiments along with maps of the actual rooms without doing an actual experiment.

[Dancing David]

Originally posted by davidsmith73
What flaws?

The most prominent ones were discussed ad nauseum in the thread I linked to earlier but here is a hit of the high lights.
1. The way that the sample picture sets are generated in the first place.
2. The fact that there is no testing for word lists generated to match a particular picture, this is crucial for two reasons, one it would allow the experiementer to make sure that the sets of pictures do not have cross over on the 'hit' words for the target in a set. Two it would allow for the objective rating of wether a response was a hit.
3. The lack of any testing to see what the response to random sets of sender word strings to given pictures in a set and the total of all sets. this is a serious flaw because it could produce hit rates higher than expected, given the lack of cross match elimination.
4. The method for selecting 'hits' could actualy create an artifact in itself.

Those were the most serious flaws.

True. However one can never rule out the possibility that you haven't controlled for effects you don't know about, in any experiment. You can only reduce the likely hood of this as much as possible.

This shows a gross misunderstanding of the methods of science, although it is more common in social sciences.
In designing an experimental study you are under the burden of trying to elimanate and possible confounding factors. This is not limited to parapsychology, this is true of all research.

Research requires that the experimenter try to control for as many possible confounding factors as possible, otherwise the results are meaningless.

It is absurd to say that you can't control for an effect you didn't know about, the elimination and calibration of confounding and linked effects is at the core of science. You may not have know about a possible effect but that does not mean that you should not then try to test to see if it is there. That is at the core of science!

If indeed the meta-analysis is compensating for flawed methodology. I do think that meta-analysis is far from the ideal method of demonstrating a replicable effect. Unfortunately, this is the situation at the moment for ganzfeld. The problem can be summed up nicely by Bem and Honortan:

"Given its larger effect size, the prospects for successfully replicating the psi ganzfeld effect are not quite so daunting, but they are probably still grimmer than intuition would suggest. If the true hit rate is in fact about 34% when 25% is expected by chance, then an experiment with 30 trials (the mean for the 28 studies in the original meta-analysis) has only about 1 chance in 6 of finding an effect significant at the .05 level with a one- tailed test. A 50-trial experiment boosts that chance to about 1 in 3. One must escalate to 100 trials in order to come close to the break even point, at which one has a 50-50 chance of finding a statistically significant effect (Utts, 1986). (Recall that only 2 of the 11 autoganzfeld studies yielded results that were individually significant at the conventional .05 level.) Those who require that a psi effect be statistically significant every time before they will seriously entertain the possibility that an effect really exists know not what they ask." (from Bem, Daryl J. and Honorton, Charles (1994). Does Psi Exist? Replicable Evidence for an Anomalous Process of Information Transfer, Psychological Bulletin, 115(1): 4-18 )
The statistics are bogus, one can not draw any conclusions about the chances of flawed data, flawed data is flawed data, the chances are not computable when you haven't used a strict method and protocol.

The statistical significance of uncontrolled data is meaningless.

I think you mean to say that the SD is higher than the mean. They use a Wilcoxon signed rank test which is a distribution free method of looking at the difference between two means within the same population. This test is appropriate for small sample sizes.

You misunderstand me completely, the 'effect' that they claim to exist between their different groups falls below the level of the standard deviation. The difference bewteen the sample means, or whatever it was that they claimed was the significant effect is less than the standard deviation in the samples. Therefore the effect could have been produced through the variation of the levels in the samples and not be an effect of 'pair matching'.

I don't think they conducted any statistics on the spatial distribution, so that's a possibility. Hopefully the follow up experiment will include a large enough sample.

Excuse me , but this is a crucial flaw in the study, in that they depend on results prior to the photic stimulation to raise the effectcience in a time ordered paradigm.
Quantum effects do not travel backwards in time, tachyons do but not particles on this side of the light speed.

I don't see why this means the photic stimulation sample is not random. Since both the original stimulation marks and the control marks were both randomly generated, then the samples taken out of the photic stimulation epochs will be random. If you take a random sample from a random sample, you are still left with a random sample are you not?

Sorry David but that is more wishful thinking, you can't just eliminate data from the sets and then claim that they are random. They could have adjusted the interval spacing to eliminate this completely and it is a methodological flaw. It is statisticaly possible that the intervals they chose to eliminate were containing data that would have leveled out the effect completely.
Methododlogy is methodology it is not kind to any one who choses to violate it.

Do you mean the artifact removal? That's not anomalous data since the source of artifact is known - equipment failure. I believe this is standard practice in EEG analysis.

They also state that they remove data for other things that can effect an EEG measurement such as eye movement, it would be standard protocol to then discuss what objective criteria were used in the elimination of those data sets. In that there should be clear guidelenes that any human could enforce. Then you have a sub-test for inter rater reliability to make sure that all raters remove the same data sets consistantly.
There should be a record kept of what was removed for the equipment failures and what data was removed due to human judgement.

The paper supposedly replicates previous results:

Wackermann, J., Seiter, C., Keibel, H., & Walach, H. (2003). Correlations between brain electrical activities of two
spatially separated human subjects. Neuroscience Letters, 336, 60-64.

I haven't read this paper yet

Replication generally requires twenty or more if not hundreds!

I want to make clear that I am just as critical of non parapsychological research. The social sciences are rife with bad methodology, messy protocols and lack of replication. I taer into psychology the same way I tear into parapsychology.
(Like the claims made in Talking to Prozac, they are seriously flawed in different ways.)

[Open Mind]

Originally posted by CFLarsen
This is not correct. Skeptics are more prone to remember the correct ratio.



To use a CF Larsen technique ‘Please prove that by a repeated controlled trial’ ;) I’m kidding, save your time, if you don’t have one close at hand..


But skeptics don't - by definition - have a negative expectation. Skeptics expect nothing - skeptics judge from the data alone.

Yes true skeptics might not, skeptics with minds made up, the ones who view it as political cause most probably do. That means probably most in CSICOP (with some exceptions of course) are probably too biased for a neutral intrepretation


We have seen here, on this board, that people will lie about their beliefs in the paranormal.

Unfortunately people lie, how do you know a professional skeptic wouldn’t be economical with the truth? To them perhaps it’s not insignificant lying, they are merely assisting their perceived correct conclusion for the benefit of science and the skeptic movement. ;)


Such signed confessions are worthless. What we should do is focus on the data. The experimental design. The protocols.

Good ….. but with researchers and experimenters enthusiastic to find PSI … to assume bored skeptic going through the motions is going to find PSI is extraordinarily bad science when PSI is supposed to be connected with thought/belief by proponents ….. the researchers must state there opinions on PSI beforehand, if some lie, so be it, it’s still better than a professional skeptic showing interest in running a trial and yawning all the way through it. Some psychologists are doing PSI trials as research into ‘self deception’ not through any sincere effort to find PSI.


Schwartz used only well-known psychics. Targ & Puthoff used Geller.

But you have a point: These self-proclaimed psychics are notoriously hard to drag into the lab. One can wonder why...

I’m saying not to use professional psychics, I’m saying to use natural psychics (they have less motive to cheat or to defend reputation) ….. the ones who have claimed to have seen dead people, etc. since childhood ….put them in friendly environment with strict controls … AFAIK it has not been done .. most ESP trials have measured the general public …. Student musicians and artists did slightly better in some trials.


So, you want to prevent skeptics - whom you say have a negative attitude towards the paranormal, and instead focus on those who has "enthusiasm"? You are advocating that people should have a positive attitude?

Yes I think to not have a positive attitude during a PSI trial is a contradiction of what is under trial. Some trials claimed to have measure ‘belief’ to see if it increases PSI by means of deception … one trial involved lying to the ones being tested telling them they were doing well. (and another group telling correct results as wrong) … This is also anti-PSI because lying in essence is the opposite of telepathy. If 100% perfect telepathy existed (thank goodness it doesn’t) it would probably mean no individuality of human mind, so lying/deception is the concealment of one mind to another – the opposite of telepathy ……Tests involving deception aren’t fair trials – they must be honest but strictly controlled attempts to find PSI with a positive attitude ….. then I would welcome negative researchers repeating and probably failing more often :) In other words the attitude of the researchers/experimenters must be known before and during trial…. That should be the next step in PSI research, an agreement of publishing accompanying attitude (yes, no skeptic is going to sign ‘this is absolute crap!’ :) they will probably claim a modest ‘very doubtful’ …similarly a proponent will probably down play their belief …… but I think science can take this into account with common sense analysis)


If psi exists, then it is not at the edge of human ability: It is quite contrary a huge part of reality, judging from the sheer number of psychics, healers, tarot-readers, astrologers, etc.

Psychics being human tend to exaggerate their abilities. This is not necessarily conscious but the underlying ego in every human that makes humans think they do things better than others see them do it. Yes and many believers are too keen to trust, make things fit and that is where the problem lies IMHO in making PSI appear
absurd …... the observation of this in no way proves real weak PSI doesn’t exist.

With regard to … tarot, astrology, palmistry, etc. …. None of these actually
require PSI ability …. I have little interest in the sort of psychics who use such. To me these are psychics who aren’t really psychic and needs toys. ;)

Spiritual healing? Let’s leave that one for another topic – too complex an issue here.


What about their results? Do you claim that Wiseman's results showed evidence of psi?

According to Sheldrake ……… Wiseman is one of Britain’s leading media sceptics. He is an informed sceptic, in the sense he reads the literature and knows what’s going on and he actually does experiments. However, he is a very committed sceptic who believes these things are basically impossible, and Wiseman went along to do these experiments with Pam Smart. He invented a criterion for the success or failure of the dog, which was, that it should go to the window, for no reason apparent to Wiseman … the first experiment it was 60 seconds. Then he changed the criterion to being two minutes for no apparent reason. If the dog went to the window for no apparent reason when she wasn’t coming home, it failed the test. He published a paper in the British Journal of Psychology, saying it had failed the test. There’s the paper. He put a Press Release. It was in all the science correspondence …………… However, if you plant Richard Wiseman’s data on a graph, which he didn’t do in his papers, even though I sent him graphs before he submitted it, showing it’s a self-reinforcing system, reinforced by a system of taboos and prejudices, which I think, are a shame to science. I think that this is an outrage, really, that in a scientific world we have this kind of behaviour going on, which I think, brings discredit on the whole of science, and I think one of the things that really disillusions people with science is the feeling that science is not actually about evidence … it’s about dogma, and my view is that science needs to be about evidence, not dogma, and personally, I see telepathy as a test case for this very principle.’


Will you also be prepare to draw the consequences, if the results turn out to be negative?
Yes ….. but I’m not sure we would completely agree on the consequences.

[dharlow]

Originally posted by Open Mind
Some trials claimed to have measure ‘belief’ to see if it increases PSI by means of deception … one trial involved lying to the ones being tested telling them they were doing well. (and another group telling correct results as wrong)

Which study was this?

[davidsmith73]

Originally posted by Dancing David


The most prominent ones were discussed ad nauseum in the thread I linked to earlier but here is a hit of the high lights.
1. The way that the sample picture sets are generated in the first place.
2. The fact that there is no testing for word lists generated to match a particular picture, this is crucial for two reasons, one it would allow the experiementer to make sure that the sets of pictures do not have cross over on the 'hit' words for the target in a set. Two it would allow for the objective rating of wether a response was a hit.
3. The lack of any testing to see what the response to random sets of sender word strings to given pictures in a set and the total of all sets. this is a serious flaw because it could produce hit rates higher than expected, given the lack of cross match elimination.
4. The method for selecting 'hits' could actualy create an artifact in itself.

Dancing David,
I have read your criticisms of the ganzfled protocol on the other thread and I believe you to be in error. With regard to your claim that a 25% hit rate is not expected by chance because of biased matchings of receivers words to targets I shall say this:

The exact same scenario occurs during games of roulette. People have their favourite numbers - 0, 13 (unlucky for some), birthdays etc. This creates exactly the same bias as you are talking about. In other words, the "target" numbers will not be chosen by the gamblers at a consistent frequency. However we all know that the gambler cannot beat the odds. Just because there exists a bias in the selection of the winning number by the gambler, the chances that each number will come up remains consistent.


This shows a gross misunderstanding of the methods of science, although it is more common in social sciences.
In designing an experimental study you are under the burden of trying to elimanate and possible confounding factors. This is not limited to parapsychology, this is true of all research.

Research requires that the experimenter try to control for as many possible confounding factors as possible, otherwise the results are meaningless.

Yes, this is known factors. I'd like to see you try to control for an effect that is not known!


It is absurd to say that you can't control for an effect you didn't know about,

That's funny, because this is the tired old criticism of parapsychology!


the elimination and calibration of confounding and linked effects is at the core of science. You may not have know about a possible effect but that does not mean that you should not then try to test to see if it is there. That is at the core of science!

How do you control for something when you have no idea what it is you are supposed to be controlling for! This is my objection to this criticism of psi research. The criticism asserts that psi experiments could be down to an unknown factor that you haven't accounted for.


The statistics are bogus, one can not draw any conclusions about the chances of flawed data, flawed data is flawed data, the chances are not computable when you haven't used a strict method and protocol.

The flaws you point out do not exist. See above.


You misunderstand me completely, the 'effect' that they claim to exist between their different groups falls below the level of the standard deviation. The difference bewteen the sample means, or whatever it was that they claimed was the significant effect is less than the standard deviation in the samples. Therefore the effect could have been produced through the variation of the levels in the samples and not be an effect of 'pair matching'.


Which is why they have used a distribution free method of analysis. If they used a parametric t-test then you would have a point.


Excuse me , but this is a crucial flaw in the study, in that they depend on results prior to the photic stimulation to raise the effectcience in a time ordered paradigm.
Quantum effects do not travel backwards in time, tachyons do but not particles on this side of the light speed.

I have no idea what you are saying here


Sorry David but that is more wishful thinking, you can't just eliminate data from the sets and then claim that they are random.

Yes I can sat that because the elimination of the data was performed in a random way since all control epochs were generated randomly. The elimination was done before any analysis is made.


They could have adjusted the interval spacing to eliminate this completely and it is a methodological flaw.

Since it's not a flaw this would have been a waste of time.


It is statisticaly possible that the intervals they chose to eliminate were containing data that would have leveled out the effect completely.

This is exactly the same as saying that if they conducted an extra 50 light flashes and added this data to what they had it may have eliminated the effect. I think it is you who is guilty of wishful thinking. Remember that the elimination of the data was random (since all control epochs were randomly generated) and occured before any analysis was made.


They also state that they remove data for other things that can effect an EEG measurement such as eye movement, it would be standard protocol to then discuss what objective criteria were used in the elimination of those data sets. In that there should be clear guidelenes that any human could enforce. Then you have a sub-test for inter rater reliability to make sure that all raters remove the same data sets consistantly.
There should be a record kept of what was removed for the equipment failures and what data was removed due to human judgement.


Wow, the first thing we agree on! Perhaps they did keep records of this.

[BillHoyt]

Originally posted by davidsmith73
ganzfled protocol

= the hasty retreat and damage control psi researchers will engage in as the fundamental experimental problems are exposed.

[CFLarsen]

Originally posted by Open Mind
Yes true skeptics might not, skeptics with minds made up, the ones who view it as political cause most probably do. That means probably most in CSICOP (with some exceptions of course) are probably too biased for a neutral intrepretation

"Most". "Probably". Not very convincing....

Originally posted by Open Mind
Unfortunately people lie, how do you know a professional skeptic wouldn’t be economical with the truth? To them perhaps it’s not insignificant lying, they are merely assisting their perceived correct conclusion for the benefit of science and the skeptic movement. ;)

Which is why we rely on evidence.

Originally posted by Open Mind
Good ….. but with researchers and experimenters enthusiastic to find PSI … to assume bored skeptic going through the motions is going to find PSI is extraordinarily bad science when PSI is supposed to be connected with thought/belief by proponents …..

Excuse me? Some trials claim evidence, even though "normal" people (without a vested interest in the paranormal) were participating.

Originally posted by Open Mind
the researchers must state there opinions on PSI beforehand, if some lie, so be it, it’s still better than a professional skeptic showing interest in running a trial and yawning all the way through it. Some psychologists are doing PSI trials as research into ‘self deception’ not through any sincere effort to find PSI.

Bull. Opinions are irrelevant. Only controlled, replicated experiments count.

Originally posted by Open Mind
I’m saying not to use professional psychics, I’m saying to use natural psychics (they have less motive to cheat or to defend reputation) ….. the ones who have claimed to have seen dead people, etc. since childhood ….put them in friendly environment with strict controls …

Who are these "natural" psychics??

Originally posted by Open Mind
AFAIK it has not been done .. most ESP trials have measured the general public …. Student musicians and artists did slightly better in some trials.

Were they "enthusiastic" about finding PSI?

Originally posted by Open Mind
Yes I think to not have a positive attitude during a PSI trial is a contradiction of what is under trial. Some trials claimed to have measure ‘belief’ to see if it increases PSI by means of deception … one trial involved lying to the ones being tested telling them they were doing well. (and another group telling correct results as wrong) … This is also anti-PSI because lying in essence is the opposite of telepathy. If 100% perfect telepathy existed (thank goodness it doesn’t) it would probably mean no individuality of human mind, so lying/deception is the concealment of one mind to another – the opposite of telepathy ……Tests involving deception aren’t fair trials – they must be honest but strictly controlled attempts to find PSI with a positive attitude ….. then I would welcome negative researchers repeating and probably failing more often :) In other words the attitude of the researchers/experimenters must be known before and during trial…. That should be the next step in PSI research, an agreement of publishing accompanying attitude (yes, no skeptic is going to sign ‘this is absolute crap!’ :) they will probably claim a modest ‘very doubtful’ …similarly a proponent will probably down play their belief …… but I think science can take this into account with common sense analysis)

Words, words, words...what you want is to lower the bar, so sloppy experiments where the result is down to opinions are accepted as evidence. Sorry, no cigar.

Originally posted by Open Mind
Psychics being human tend to exaggerate their abilities. This is not necessarily conscious but the underlying ego in every human that makes humans think they do things better than others see them do it. Yes and many believers are too keen to trust, make things fit and that is where the problem lies IMHO in making PSI appear
absurd …... the observation of this in no way proves real weak PSI doesn’t exist.

For now, I don't care about the reasons: All I want to see any kind of PSI effect. So far - nada.

Originally posted by Open Mind
With regard to … tarot, astrology, palmistry, etc. …. None of these actually
require PSI ability …. I have little interest in the sort of psychics who use such. To me these are psychics who aren’t really psychic and needs toys. ;)

So, you are willing to discard all these paranormal phenomena - without even looking at them - simply because they don't require "psi"? Isn't that a wee bit narrowminded?

Originally posted by Open Mind
Spiritual healing? Let’s leave that one for another topic – too complex an issue here.

No, let's discuss all purported paranormal phenomena, regardless of they require psi or not. How do you know that psi doesn't control astrology?

Originally posted by Open Mind
According to Sheldrake ……… Wiseman is one of Britain’s leading media sceptics. He is an informed sceptic, in the sense he reads the literature and knows what’s going on and he actually does experiments. However, he is a very committed sceptic who believes these things are basically impossible, and Wiseman went along to do these experiments with Pam Smart. He invented a criterion for the success or failure of the dog, which was, that it should go to the window, for no reason apparent to Wiseman … the first experiment it was 60 seconds. Then he changed the criterion to being two minutes for no apparent reason. If the dog went to the window for no apparent reason when she wasn’t coming home, it failed the test. He published a paper in the British Journal of Psychology, saying it had failed the test. There’s the paper. He put a Press Release. It was in all the science correspondence …………… However, if you plant Richard Wiseman’s data on a graph, which he didn’t do in his papers, even though I sent him graphs before he submitted it, showing it’s a self-reinforcing system, reinforced by a system of taboos and prejudices, which I think, are a shame to science. I think that this is an outrage, really, that in a scientific world we have this kind of behaviour going on, which I think, brings discredit on the whole of science, and I think one of the things that really disillusions people with science is the feeling that science is not actually about evidence … it’s about dogma, and my view is that science needs to be about evidence, not dogma, and personally, I see telepathy as a test case for this very principle.’

Blah, blah, blah. Do you claim that Wiseman's results showed evidence of psi?

Originally posted by Open Mind
Yes ….. but I’m not sure we would completely agree on the consequences.

No? The consequences would be that psi was not shown to exist. Would you agree?

[davidsmith73]

Claus, do you have any evidence that there is a correlation between effect size and study outcome in published psi research?

[Dancing David]

Originally posted by davidsmith73
Dancing David,
I have read your criticisms of the ganzfled protocol on the other thread and I believe you to be in error. With regard to your claim that a 25% hit rate is not expected by chance because of biased matchings of receivers words to targets I shall say this:

The exact same scenario occurs during games of roulette. People have their favourite numbers - 0, 13 (unlucky for some), birthdays etc. This creates exactly the same bias as you are talking about. In other words, the "target" numbers will not be chosen by the gamblers at a consistent frequency. However we all know that the gambler cannot beat the odds. Just because there exists a bias in the selection of the winning number by the gambler, the chances that each number will come up remains consistent.

Then you did not understand the critique at all :

1. Matching words that match more than one picture, in the case where pictures are not tested against random reciever strings it is very possible that there could be more than one picture in a set that matches a given word spoken by the reciever. In this case the odds of a word spoken by the reciever matching one of the pictures is higher than 25%.
2. If you have many pictures that are unrated for random reciever word string match then you can have an effect where say you have four pictures that have a 50% chance of matching a 'random reciever string', that means that there is then a fifty percent chance of a hot being scored for any random string uttered by the reciever. This is a serious methodoligical falw that could be controlled for and in fact can be used to the benefit of psi research. So again this destroys the false assumption of a twenty five percent rate and makes it much higher.

The point is that before you assume that the pictures will match a random reciever word string at a twenty five per cent rate you need to determine the actual match rate.
That is why it is very important that the 'match criteria for a hit' be determined in a fashion described as 'objective', in that there should be a list of words for each picture and only those word will generate a hit, then sets should be devised such that each picture has unique 'match criteria for a hit'. This would eleiminate most of the flaw that i see

But you can't just assert that there is a twenty five percent match rate with out these criteria.

Yes, this is known factors. I'd like to see you try to control for an effect that is not known!

The factors that I discuss are known factors, this happens in research all the time, there are times where even the way that the experimenter presents the routine can effect the outcome, which is why you have video tape , trainings and scoring.
Assumiong that the match rate will be twenty five percent for a trial is a mistake and it can be fixed very easily.

That's funny, because this is the tired old criticism of parapsychology!

In science it is imporatnt to think about confounding factors before you design your experiment, and once they are suggested then that is just "Oh too bad." This is true on about eighty percent of psychology research, experimenter does a study, people point out potentail flaws, study is redone or abandoned, that is science.

How do you control for something when you have no idea what it is you are supposed to be controlling for! This is my objection to this criticism of psi research. The criticism asserts that psi experiments could be down to an unknown factor that you haven't accounted for.

No I don't say that you have to control for tidal forces, but when you design your research you have to be open to thinking about the flaws in your methodology. I am not saying that psi research is flawed I am saying that the methods of the studies I have seen are flawed.

Believe me, this is true of much research in the social science.

The flaws you point out do not exist. See above.

The flaw is in the bold assumption that there will be a twenty five percent match rate in a given set. there are reasons that this might not be the case. The methodology could be very easy to implement and there are a number of ways to do it.
1. rate the random match for each picture.
2. set up unique criteria for each picture in a set.
Either one will do, but you can't assume a twenty five percent match.

Which is why they have used a distribution free method of analysis. If they used a parametric t-test then you would have a point.

Doesn't matter if the standard deviation is higher than the effect than there is no effect.

I have no idea what you are saying here

Then you didn't read your own study or you didn't ubderstand it , part of the way that they generate the effect is by looking for responses prior to the stimulation.
At this time there is no mechanism to account for it. I find it interesting but distatseful as it strechs the times frame to make the effect appear.

Yes I can sat that because the elimination of the data was performed in a random way since all control epochs were generated randomly. The elimination was done before any analysis is made.

You misunderstand the nature of good sampling technique this is called an artifact of sampling, it can not be assumed to be random and it could have easily been eliminated. It is not a deadly flaw but it would not be used in any other field than psi research.

Since it's not a flaw this would have been a waste of time.

Your above comment shows that you have little understanding of sampling technique. It would not have wasted anytime at all, it would have allowed for clearer data sets and would not have had the artifact in the elimination of data that you are just asserting is randomly eliminated.

This is exactly the same as saying that if they conducted an extra 50 light flashes and added this data to what they had it may have eliminated the effect. I think it is you who is guilty of wishful thinking. Remember that the elimination of the data was random (since all control epochs were randomly generated) and occured before any analysis was made.

You so silly. You obviously don't know about the perils of sampling error, many a fine theory has gone down the tubes because of sampling error, it is a real possibility. Sorry, that is the way it is. I hold parapsychology to the same standard I hold any social science, you wouldn't understand that would you?
Sampling error is real thing that produces many a strange artifact of science. Oh Too Bad!
You also don't understand that the sample they produced is not a random sample any more.





Wow, the first thing we agree on! Perhaps they did keep records of this.

There is alot that we agree on, you just want to shelter your baby from the storm, which is understandable, but it isn't science.

Psychology has the same pitfalls all the time, and I have the same critique for them.

[CFLarsen]

Originally posted by davidsmith73
Claus, do you have any evidence that there is a correlation between effect size and study outcome in published psi research?

Already shown. You are of the opinion that no such evidence exists. That's fine.

[T'ai Chi]

Originally posted by CFLarsen
Already shown. You are of the opinion that no such evidence exists. That's fine.

What is the specific paper(s) that show this?
(evidence that there is a correlation between effect size and study outcome in published psi research)

[CFLarsen]

Originally posted by jzs
What is the specific paper(s) that show this?
(evidence that there is a correlation between effect size and study outcome in published psi research)

Yawn...

[davidsmith73]

Originally posted by Dancing David


1. Matching words that match more than one picture, in the case where pictures are not tested against random reciever strings it is very possible that there could be more than one picture in a set that matches a given word spoken by the reciever. In this case the odds of a word spoken by the reciever matching one of the pictures is higher than 25%.

Of course there will be a different probability than 25% of a receiver word matching a specific picture. However since each picture has a 25% chance of being the correct target this is irrelavent to your argument. It's simply not a flaw. See here:

http://www.randi.org/vbulletin/showthread.php?s=&threadid=49622


The point is that before you assume that the pictures will match a random reciever word string at a twenty five per cent rate you need to determine the actual match rate.
That is why it is very important that the 'match criteria for a hit' be determined in a fashion described as 'objective', in that there should be a list of words for each picture and only those word will generate a hit, then sets should be devised such that each picture has unique 'match criteria for a hit'. This would eleiminate most of the flaw that i see

But you can't just assert that there is a twenty five percent match rate with out these criteria.

Again, this is all true but irrelavent. Yes, pictures will not have a 25% chance of being chosen by the judges but each picture does have a 25% chance of being the randomly selected target.


Assumiong that the match rate will be twenty five percent for a trial is a mistake and it can be fixed very easily.


You clearly do not understand the ganzfeld protocol. No experimenter assumes that the match rate as you call it will be 25%. The judges will certainly be biased as to their choice of target. However, the selection of the actual target amongst all pictures is done randomly by a computer. Hence each picture has a 25% chance of being the target in each trial.


In science it is imporatnt to think about confounding factors before you design your experiment, and once they are suggested then that is just "Oh too bad." This is true on about eighty percent of psychology research, experimenter does a study, people point out potentail flaws, study is redone or abandoned, that is science.


I was talking about the criticism that "psi results are due to unkown factors"! For sure, if specific flaws are pointed out and they are valid then its the duty of the experimenter to correct them. However you can't just say - "oh there is a flaw in there somewhere but we don't know what it is"


No I don't say that you have to control for tidal forces, but when you design your research you have to be open to thinking about the flaws in your methodology.

Absolutely, and this is done to exhaustion in some psi studies especially ganzfeld.


Doesn't matter if the standard deviation is higher than the effect than there is no effect.

Why is the standard deviation relavent in a distribution-free method of analysis?


Then you didn't read your own study or you didn't ubderstand it

no, i just didn't understand what you said.


part of the way that they generate the effect is by looking for responses prior to the stimulation.
At this time there is no mechanism to account for it. I find it interesting but distatseful as it strechs the times frame to make the effect appear.


No. They compared the EEG response 500ms before to 500ms after onset of stimulation of the sender and found a significant difference. The analysis does not take into account the apparent "precognitive" effect. That was just mentioned as an interesting observation.


You misunderstand the nature of good sampling technique this is called an artifact of sampling, it can not be assumed to be random and it could have easily been eliminated. It is not a deadly flaw but it would not be used in any other field than psi research.


lets say we select people at random from a population in order to measure their height. Before me measure this sample of people we randomly eliminate a few of them from the sample. You are saying that this will cause an artifact in the remaining sample. How does this happen?


You also don't understand that the sample they produced is not a random sample any more.

Its all very well stating this, but please explain how. If I randomly eliminate some people from my height study, how does this make the remaining sample non-random?

[CFLarsen]

davidsmith73,

Just curious:

Do you see a problem at all with the judges determining how the people tested should react physically?

[davidsmith73]

Originally posted by CFLarsen
davidsmith73,

Just curious:

Do you see a problem at all with the judges determining how the people tested should react physically?

Sorry Claus, I don't know what you are refering to here. Which judges in which experiment and what physical reaction?

[CFLarsen]

Originally posted by davidsmith73
Sorry Claus, I don't know what you are refering to here. Which judges in which experiment and what physical reaction?

This experiment. Judges deciding which pictures will give a certain reaction in the people being tested, that is measured by the equipment.

[davidsmith73]

Originally posted by CFLarsen
This experiment. Judges deciding which pictures will give a certain reaction in the people being tested, that is measured by the equipment.

You may be confusing experiments. The one I was discussing above was the EEG reaction to uniform light flashes. There are no pictures involved or judging procedure. Do you mean the precognitive habituation experiments? In which case there is no equipment measuring a physiological response.

[T'ai Chi]

Originally posted by CFLarsen
Yawn...

I'll ask again since you were just going to sleep.

What is the specific paper(s) that show evidence that there is a correlation between effect size and study outcome in published psi research?

I think your quote may be appropriate here


So, there is no evidence? Only your opinion?

[davidsmith73]

Originally posted by jzs
I'll ask again since you were just going to sleep.

What is the specific paper(s) that show evidence that there is a correlation between effect size and study outcome in published psi research?

I think your quote may be appropriate here

[/B]

I wouldn't be too hopefull jzs. I've also asked 3 times already

[T'ai Chi]

Originally posted by davidsmith73
I wouldn't be too hopefull jzs. I've also asked 3 times already

I admit I may have overlooked the possibility that "Yawn..." is a scientific peer reviewed journal somewhere that I've never heard of. ;)

[Dancing David]

DavidSmith,

i am sorry that we appear to have reached an impasse in communication. I have tried to explain that the odds of a match can be influenced by the stated methodological flaws. And flaws they are. If you don'y wish to undertsand that is your choice.

As to the removing of the data on allegedly random basis, it is still an artifact that could create a non existant effect.

As for standard deviation, lets us ee, the experimenters say that there is this change. And that change is withing the parameters of the statistical variation of the sample. Therefore it is likely that the 'effect' solely the product of variation within the sample and not any correlation between the test groups.

I still maintain that it is sloppy methodology and that they could have had a more rigorous study in the ganfeld effect.

The EEG study is of interest but at this point insignificant.

I would like to thank you for not getting wildly off trach in the debate, You have been most reasonable.

[Interesting Ian]

Originally posted by jzs
I'll ask again since you were just going to sleep.

What is the specific paper(s) that show evidence that there is a correlation between effect size and study outcome in published psi research?

I think your quote may be appropriate here

[/B]

OK, let me help out Claus here. Just came across this article - The capricious, actively evasive, unsustainable nature of psi: a summary and hypotheses (http://www.findarticles.com/p/articles/mi_m2320/is_1_67/ai_104657309) - although I haven't read it yet. But I notice on page 2 it says:


LESS PSI WITH IMPROVED METHODOLOGY

Braud (1985) and Batcheldor (1994) have suggested that psi appears to occur more readily in situations with lower methodological quality. For example, Dunne and Jahn (2002) noted that their 25-year experience with remote viewing found a steady decline that occurred as they attempted to develop increasingly objective, quantitative evaluation methods. Braud (1985) also noted that more sophisticated, process-oriented studies seem less likely to obtain psi effects than simple studies intended to provide evidence for the existence of psi. Of course, skeptics interpret these trends as evidence that ostensible psi phenomena actually result from methodological problems.

[davidsmith73]

Originally posted by Dancing David
DavidSmith,

i am sorry that we appear to have reached an impasse in communication. I have tried to explain that the odds of a match can be influenced by the stated methodological flaws. And flaws they are. If you don'y wish to undertsand that is your choice.

it's not that I do not wish to understand, I just sincerely think that the flaw does not exist. Since a purely random set of guesses in a ganzfeld trial has an equal probability of hitting the correct target as a set of subjective guesses, the issue is one of pure probability. Thus relatively simple statistics can be performed that estimate the likely hood that the observed hit rate was due to chance. Subjective bias has no way of causing an increased hit rate, only a post hoc correlation in just the same way that a random set of guess would.


As to the removing of the data on allegedly random basis, it is still an artifact that could create a non existant effect.

I would really like to know how you conclude this. I just don't see how randomly removing data from a random sample can produce an artifact in the remaining data set, especially since it was done on the raw data rather than the analysed data.

[T'ai Chi]

Originally posted by Interesting Ian
OK, let me help out Claus here. ...

"LESS PSI WITH IMPROVED METHODOLOGY

Braud (1985) and Batcheldor (1994) have suggested that psi appears to occur more readily in situations with lower methodological quality. For example, Dunne and Jahn (2002) noted that their 25-year experience with remote viewing found a steady decline that occurred as they attempted to develop increasingly objective, quantitative evaluation methods. Braud (1985) also noted that more sophisticated, process-oriented studies seem less likely to obtain psi effects than simple studies intended to provide evidence for the existence of psi. Of course, skeptics interpret these trends as evidence that ostensible psi phenomena actually result from methodological problems."


Thanks Ian. I'll wait to see what studies Claus will offer (if anything). I doubt he'll offer the above one as evidence, since he generally rejects work by Dunne, Jahn, et al, so it would be a little odd if he embraced the above study.

[CFLarsen]

Originally posted by jzs
Thanks Ian. I'll wait to see what studies Claus will offer (if anything). I doubt he'll offer the above one as evidence, since he generally rejects work by Dunne, Jahn, et al, so it would be a little odd if he embraced the above study.

Why would that be odd? I don't reject studies based on who did them, but on the results, the methodology, etc.

It would be most productive if you could stop misrepresenting me.

[T'ai Chi]

Originally posted by CFLarsen
Why would that be odd? I don't reject studies based on who did them, but on the results, the methodology, etc.

It would be most productive if you could stop misrepresenting me.

I said "generally". I did not say you reject their papers just because of them, so stop misrepresenting me.

I'm still waiting to see if you actually admit to rejecting or accepting to above study as evidence for a decline in 'psi' being due to a tightning of the controls.