Winning the prelim. by chance?

[T'ai Chi]

from the faq:


Between 1964 and 1982, Randi declared that over 650 people had applied. Between 1997 and February 15, 2005, there had been a total of 352 official, notarized applications.
...
1.4. How many people have passed the preliminary test?
None. Most applicants never agree to a proper test protocol, so most are never tested.


I wonder what alpha is typically set to for these preliminary tests? Does anybody know?

[geni]

Originally posted by jzs
from the faq:



I wonder what alpha is typically set to for these preliminary tests? Does anybody know? [/B]

1000 to one. If the challange contines at it's current rate I expect someone (probably a dowser) to pass the preliminary test by chance in my lifetime.

[Carn]

From faq:
Between 1964 and 1982, Randi declared that over 650 people had applied. Between 1997 and February
15, 2005, there had been a total of 352 official, notarized applications.
...
1.4. How many people have passed the preliminary test?
None. Most applicants never agree to a proper test protocol, so most are never tested.

Originally posted by geni
1000 to one. If the challange contines at it's current rate I expect someone (probably a dowser) to pass the preliminary test by chance in my lifetime.

I guess JZS, that you hope to show with these numbers, that Randi is a lyer or trickster, that manipulates tests so, that no one can pass. And i agree, that if 3000 people would fail a 1 in 1000 chance it would start getting suspicious, chance for that is 4.9%. Then the explanation "Randi is cheating" cannot longer be disregarded without any further thought.
If 10000 fail to pass i would be convinced, that something is going wrong(0.0045%), though "Randi is cheating" could be one of several possible explanations.

The numbers above are 650+352=1002.
No one of 1002 passing a 1 in 1000 chance has a chance of 36.7%, so i guess you agree, that this does not prove anything.

But this number still has 4 mistakes, you have to keep in mind, if trying to argue, that way against Randi's credibility:

1.the "over 650" people "applied" according to the randi quote. It is uncertain, whether this means, that an actual test was done or whether there were just 650 negotiations about a possible test. Shortly said, the counting of 650 was done by Randi in a non reliable way with a uncertain defintion of "applied".

2. Before JREF Randi only had a non-formal challenge with his personal money at stake, which did only consist of a single test. Unless Randi provides some detailed data upon the tests done, it is uncertain, whether or not he applied the same 1 in 1000 limit or was more or less strict.

3.The 352 application are just that, 352 applications. From the challenge application section, we can see, that at least in the past year most applications do not result in any tests done. Therfore 352 applications likely mean less than 150 tests done.

4. This is the most important problem:
The 1 in 1000 chance is the minimum safeguard of JREFt to keep people by passing due to
chance, if in an actual protocol the chance is
lower due to claim or circumstances, JREF will
not change the protocol, so that the chance
goes up to 1 in 1000 again. Therefore it could
be that the average chance to pass for a random
applicant is lower. Then of
course far more people would have to fail test
before we could conclude "Randi cheating".
E.g. any applicant claiming to be able to fly,
has a ridiculously low chance to pass JREF
prelim by chance, it would be around 1 in
10**23 or lower.

To give a reliable upper limit for chance(that you need, you would have to argue "chance is at most 1 in 100 or so, that no one passed by chance up to now, therefore people should consider Randi cheating"), that no one has passed JREF challenge by chance, you have to sort all tests actually done and only those, according to the chance the applicants had to pass by luck.

Unless someone does that, all we can do is give a lower limit for the chance, that no one passed so far.

Formulas:

chance of no applicant passing = chances of each applicant multiplyed =(in case of all having same chance) chance**number of applicants

number of applicants so that chance of random event is below x= ln(chance)/ln(x)

Carn

[Gr8wight]

Originally posted by Carn


3.The 352 application are just that, 352 applications. From the challenge application section, we can see, that at least in the past year most applications do not result in any tests done. Therfore 352 applications likely mean less than 150 tests done.



Carn

In fact, Kramer said to me that in the one year he has been at the JREF, only one preliminary test has been undertaken. That was prior to the Yellow Bamboo test. He was not able to give me data for the previous five or six years elapsed since the challenge was formalized at $1,000,000, but it seems reasonable to assume it wouldn't be higher by an order of magnitude. So, of those 352 applications, it is more likely that fewer than 15-20 preliminary tests have been done. Planning to live a long, long time, are we geni?


(edited to correct the name of the poster addressed)

[Starrman]

And some challenges don't require chance at all. Yellow Bamboo, for instance, doesn't have a 1 in 1000 chance of succeeding - they can either do it or they can't.

[Carn]

Originally posted by Starrman
And some challenges don't require chance at all. Yellow Bamboo, for instance, doesn't have a 1 in 1000 chance of succeeding - they can either do it or they can't.

Depends upon the exact protocol.

An earthquake or a small meteorid might drop the attacker, if the protocol does not exclude this.

Also all attackers might just suffer at the right time during experiment some kind of heart or brain circulatory problem, which causes them to loose concious for half a second. It would be pretty difficult to notice this and it would look like yello bamboo passed the test. Though it is very unlikely.

And finally, though even more unlikely, my favourite one will work as well, if all the the air molecules right before the attacker suddenly just by chance move against him, he will be knocked down at least.

Carn

[geni]

Originally posted by Gr8wight
Planning to live a long, long time, are we geni?


yep and my life expectancy gives me another 60 years anyway.

[marting]

Originally posted by Carn
Formulas:

chance of no applicant passing = chances of each applicant multiplyed =(in case of all having same chance) chance**number of applicants

Carn

Yoiur number appears incorrect.
[Retracted. Oops , my error, Your number used was 3000, not 4000]

Assuming there is 1 chance in 1000 of passing then failure is the other 999. Thus the probability that all 4000 fail is .999**4000.

This is approx .018, and thus the probability of at least one passing is 1 - .018 or .982.

marty

[Gr8wight]

Originally posted by geni
yep and my life expectancy gives me another 60 years anyway.


Unfortunately, you would have to live almost six times that long to witness 1000 preliminary tests at the current frequency. I suspect you are unlikely to see that random pass.

[T'ai Chi]

Originally posted by Starrman
And some challenges don't require chance at all. Yellow Bamboo, for instance, doesn't have a 1 in 1000 chance of succeeding - they can either do it or they can't.

Possibly. Not even the best baseball player in the world can hit a homerun every time they are up to bat.

[Zep]

Originally posted by jzs
Possibly. Not even the best baseball player in the world can hit a homerun every time they are up to bat. If they only ever step up to the plate ONCE in their entire career and belt a homer off some underarm pitcher, what does that say about their "perfect" batting stats?

[T'ai Chi]

Originally posted by Zep
If they only ever step up to the plate ONCE in their entire


Ganzfeld, RNG, and some others have 'hit' much more than once.

Take dowsing. Either they can or can't find it. But wait, when we design an experiment, we have to keep in mind that they are able to 'hit' by chance alone. So our experiment becomes statistical in nature, even though if they can dowse or not is a purely binary thing.

[CFLarsen]

Originally posted by jzs
I wonder what alpha is typically set to for these preliminary tests? Does anybody know?

It doesn't make sense to calculate "odds" this way: Each claim is different than the others:

Since claims vary greatly in character and scope, specific rules must be formulated for each applicant.
Rules (http://www.randi.org/research/challenge.html)

Either people can do it, or they can't. End of story.

[T'ai Chi]

Originally posted by CFLarsen
It doesn't make sense to calculate "odds" this way: Each claim is different than the others:


http://www.randi.org/jr/08-24-01.html

"As always, as described in the rules, a preliminary test for the JREF prize would be performed. That test would have odds of only 1 in 1,000 against the results being positive by chance alone. "

It doesn't matter a bit that "each claim is different" when setting an alpha.


Either people can do it, or they can't. End of story.

Can Larry Bird make a 3 pointer each time he atttempts one? Yes or no?

[Carn]

Originally posted by jzs

Can Larry Bird make a 3 pointer each time he atttempts one? Yes or no? [/B]

You are pointing at a problem, that i thought of as well, though i do not think it's the concern of JREF:

What if people do not know how good they are?
What if the yello bamboo guys suceed only 1/10 of the time when stoppiing someone with screaming?
Then they might possess a paranormal ability, but would still fail the JREF test, if unlucky or having a bad day.
Couldn't JREF miss some real para ability due to applicants misjudging their ability?

Yes it could happen, but it takes exceptional stupid applicants and therefore no one has to care.
What sane person would try to win a million with some skill and not extensively training the skill under the circumstances he is required to perform? But if you train extensively and miss, that you are performing far worse, than you think, then you are just stupid.
When training to throw darts to prepare for a demonstration, where you claim to get at least 50 points per 3 shots, wouldn't you notice if you miss 2/3 of the time the dart board?

Carn

[T'ai Chi]

Claus, if someone can do it or they can't, that doesn't mean that statistics aren't involved.

As I mentioned, dowsers can either do it or they can't. However, their claim is treated in a statistical manner. They have to get x correct out of 20, or whatever.

So I'm not sure where you are going when you say someone can either do it or they can't.

[CFLarsen]

Originally posted by jzs
http://www.randi.org/jr/08-24-01.html

"As always, as described in the rules, a preliminary test for the JREF prize would be performed. That test would have odds of only 1 in 1,000 against the results being positive by chance alone. "

It doesn't matter a bit that "each claim is different" when setting an alpha.

Randi is talking about one specific experiment! You cannot simply take that as a general example.

Originally posted by jzs
Can Larry Bird make a 3 pointer each time he atttempts one? Yes or no?

I have no idea who Larry Bird is, or what a 3 pointer is.

[CFLarsen]

Originally posted by jzs
Claus, if someone can do it or they can't, that doesn't mean that statistics aren't involved.

As I mentioned, dowsers can either do it or they can't. However, their claim is treated in a statistical manner. They have to get x correct out of 20, or whatever.

So I'm not sure where you are going when you say someone can either do it or they can't.

But you need to know the exact number of necessary correct hits out of how many possible for each and every test, before you can start doing anything.

As it is, you have taken one experiment and extrapolated that into encompassing all experiments. You are fumbling in the dark, Justin.

[Jono]

The discussion above regarding statistics, or the application of it seems familiar.

I remember it during a group discussion with our biology teacher, he then quoted a swedish saying about the spurious nature behind many claimed stats.

"there are lies, then there are damn lies, then there are statistics"

[drkitten]

Originally posted by jzs


So I'm not sure where you are going when you say someone can either do it or they can't.

There are some claims where someone proposes an effect that is so overwhelmingly unlikely in a non-paranormal context, but so likely in the claimed paranormal context, that formal statistics are unnecessary. The applicant either succeeds or fails.

Larry Bird may not be able to hit a three point shot every time he tries, but he can certainly lift a basketball three feet any time he tries. If you claims to be able to lift a basketball purely by mental effort -- telekinetically -- what's the "chance" that you could do it by pure luck alone? Almost zero.

So if you can actually lift a basketball telekinetically, we don't need to worry about formal alpha cutoffs. Just lift the thing.

[edthedoc]

Surely the same as spoon bending though: there's a large possibility of fraud or cheating as Uri Geller knows so well.

[The Don]

Originally posted by jzs
So I'm not sure where you are going when you say someone can either do it or they can't.
It depends on my claim.

If I claim to be able to predict with 100% certainty the outcome of a coin then 10 tosses should be sufficient to establish whether or not I can to 0.001 probability.

If I'm claiming 55% certainty then I'd have to do many many more tests, but after sufficient coin tosses we could again test my assertion.


To return to your baseball analogy. How many at-bats I need to support my claim is dependent upon how different from chance my claim is.

[Starrman]

So if you can actually lift a basketball telekinetically, we don't need to worry about formal alpha cutoffs. Just lift the thing.

Exactly - I'm sure Randi would allow someone as many attempts or as much time as they felt they needed in order to do something like this. This is why the applicant has a say in the design of the test. If he says he can lift the basketball with his brain if he has 4 hours to try, do we say he lost due to chance?

If you ask Larry Bird to make a 3-pointer and he misses the first one - it is stupid to say Larry Bird cannot ever make three pointers. But, if you give him a half hour to hit one he probably will. But how do you set up a test for this with 1000 to one odds? You can't - which is (one of the many reasons) why jsz's argument is wrong.

[drkitten]

Originally posted by edthedoc
Surely the same as spoon bending though: there's a large possibility of fraud or cheating as Uri Geller knows so well.

But that's irrelevant to the statistical question. If I can cheat (and bend a spoon, or lift a basketball) doing it once, I can certainly cheat a second time.

Repeating an experiment protects Randi against people accomplishing tasks by "dumb luck." His other protocols guard against deliberate deception, misdirection, and cheating. But only some kinds of task can be accomplished by dumb luck -- I can perhaps "get lucky" and guess a coin flip, or beat Randi at rock-paper-scissors, or predict the next day's lottery numbers. I can't, however, "get lucky" and lift a basketball telekinetically.

Originally posted by Starrman

If you ask Larry Bird to make a 3-pointer and he misses the first one - it is stupid to say Larry Bird cannot ever make three pointers. But, if you give him a half hour to hit one he probably will. But how do you set up a test for this with 1000 to one odds? You can't - which is (one of the many reasons) why jsz's argument is wrong.


It depends on the claim. If I claim that I can make a single three point shot -- that's simply not paranormal. Lots of people can do that. If I say I can make five hundred three point shots in a row, that may or may not be paranormal (it may simply be "incredible" skill), and therefore the JREF may or may not accept it.

But it's certainly unlikely. We could, if we like, formalize the idea of just how unlikely it is by taking the shooting percentages of the best in the league, and then crunching the numbers to calculate the chance of a "hot streak" lasting five hundred shots. Or we could just accept that the chances are near enough to zero -- and if I can do what I claim, I get the money.

[drkitten]

Originally posted by The Don
It depends on my claim.

If I claim to be able to predict with 100% certainty the outcome of a coin then 10 tosses should be sufficient to establish whether or not I can to 0.001 probability.

If I'm claiming 55% certainty then I'd have to do many many more tests, but after sufficient coin tosses we could again test my assertion.



And if I claim to be able to set the coin on fire by staring at it, we probably only need one success, no matter how many tests get run. Because that's the sort of thing I that won't happen by chance at all.

[Paul2]

Originally posted by WhiteLion
The discussion above regarding statistics, or the application of it seems familiar.

I remember it during a group discussion with our biology teacher, he then quoted a swedish saying about the spurious nature behind many claimed stats.

"there are lies, then there are damn lies, then there are statistics"

Please excuse the American chauvinism, but I think that quote is correctly attributed to Mark Twain.

[PixyMisa]

Originally posted by Paul2
Please excuse the American chauvinism, but I think that quote is correctly attributed to Mark Twain.
*cough* Disraeli *cough*

Mark Twain used it, but he attributed the quote to Benjamin Disraeli.

[T'ai Chi]

Originally posted by CFLarsen
Randi is talking about one specific experiment!

No, he wasn't.

Read his quote again

--
As always, as described in the rules, a preliminary test for the JREF prize would be performed. That test would have odds of only 1 in 1,000 against the results being positive by chance alone. "
--

[Carn]

Originally posted by jzs
No, he wasn't.

Read his quote again

--
As always, as described in the rules, a preliminary test for the JREF prize would be performed. That test would have odds of only 1 in 1,000 against the results being positive by chance alone. "
--

jzs, is there anything you want to discuss in this topic, except for trying to paint CFLarsen as an illiterate?(Which by the way you do not do very well so far, mind my above emphasize.)

I'm just locking for a topic, you know.

Carn

[CFLarsen]

Originally posted by jzs
No, he wasn't.

Read his quote again

--
As always, as described in the rules, a preliminary test for the JREF prize would be performed. That test would have odds of only 1 in 1,000 against the results being positive by chance alone. "
--

Yeah, why don't you read his quote again?

That test would have odds of only 1 in 1,000 against the results being positive by chance alone.

(Emphasis mine)

Randi is talking about ONE SPECIFIC test!

[T'ai Chi]

Originally posted by CFLarsen

Randi is talking about ONE SPECIFIC test!

Set alpha to whatever you want. The idea still stands that someone could pass by chance, right?

Say the average alpha is p. If no one has passed the preliminary after there have been 1/p or more applicants, what would you think?

[CFLarsen]

Is Randi talking about one specific test, yes or no?

[T'ai Chi]

Originally posted by CFLarsen
Is Randi talking about one specific test, yes or no?

I certainly read it as one. I've emailed Randi for clarification, and to ask if alpha varies on preliminary tests, to give a range.

Now, Claus, please answer some questions"

Set alpha to whatever you want. The idea still stands that someone could pass by chance, right?

Say the average alpha is p. If no one has passed the preliminary after there have been 1/p or more applicants, what would you think?

[CFLarsen]

One thing at a time.

I'll await Randi's reply, then.

[T'ai Chi]

Originally posted by CFLarsen
One thing at a time.

I'll await Randi's reply, then.

You don't even want to comment based on the average alpha, or based on any alpha?

The idea still stands that someone is expected to pass the prelim by chance, right? Given a number of prelim tests greater than 1/(min alpha), what would you think if no one has passed the prelim?

[drkitten]

Originally posted by jzs
I certainly read it as one. I've emailed Randi for clarification, and to ask if alpha varies on preliminary tests, to give a range.


You needn't have wasted your bytes. There's enough information available in the challenge thread to demonstrate that the answer to your question is "yes." For example, the person who claims to be able to make it snow in Oakland, CA, during a month that has never been recorded as being colder than forty degrees. I can't accurately estimate the alpha for that, given that we have about a hundred years of records of thirty day months, we can say that the alpha cutoff for being below fifty degrees is about one in three thousand -- and the applicant claims to produce not only a temperature at least eight degrees lower than that, but also produce precipitation during that day.

[drkitten]

Originally posted by jzs

Say the average alpha is p. If no one has passed the preliminary after there have been 1/p or more applicants, what would you think?

Not much. If the chance of success is 1/p, then the chance of someone succeeeding "by chance" in p attempts is approximately 70%.

Or in other words, there's about one chance in three of no one winning the event by chance in that case.

There's about one chance in ten of no one winning after 2p attempts, and about one chance in thirdy after 3p attempts. Come back after 3p attempts and we'll definitely talk, because I'd definitely be willing to suspect chicanery after 3p attempts.

[Carn]

Originally posted by jzs
Set alpha to whatever you want. The idea still stands that someone could pass by chance, right?

Say the average alpha is p. If no one has passed the preliminary after there have been 1/p or more applicants, what would you think?

If you had minded one of my above posts you would know, that 1000 times trying a 1/1000 chance has a 36.7% chance of not succeding, so it wouldn't show anything.

BTW, the chance for non of 1/p tests being sucesful in case of alpha of p is (1-p)**1/p=~1/e=0.367879 in case of large p(>100).


And keep in mind that we actually only no the maximum p, the average p is rather diffcult to determine and we only have a maximum for the number of tests, roughly 1000.


Wait a few decades before you try to prove Randi is cheating that way.

Carn

[T'ai Chi]

Originally posted by new drkitten
Not much. If the chance of success is 1/p, then the chance of someone succeeeding "by chance" in p attempts is approximately 70%.

Or in other words, there's about one chance in three of no one winning the event by chance in that case.

There's about one chance in ten of no one winning after 2p attempts, and about one chance in thirdy after 3p attempts. Come back after 3p attempts and we'll definitely talk, because I'd definitely be willing to suspect chicanery after 3p attempts.

I used p as shorthand for alpha, the probabiltiy of commiting a type I error in a hypothesis test.

[drkitten]

Originally posted by jzs
I used p as shorthand for alpha, the probabiltiy of commiting a type I error in a hypothesis test.

Yes, I know. So what?

If the alpha cutoff is uniformly 1/p, then the chance of getting a type I error in p replications is about 70%. NOT 100%, as you seem to think. There's about a 30% chance of seeing no type I errors at all in a run of p trials.

Here's an example, from a simulated run of 100 "random" experiments with a 5% alpha cutoff. The 'Y's indicate where the random number generator created a "significant" result.

NNNNNNNNYNNNNNNNNNNNNNNNNYNNNYNNNNNNNNNNNNNNNNNNNN NNNNNYNNNNNNNNNNNNNNNNNYNNNNNNNNYNNNNNNNNNNNNNNNNN

Notice first that there are six successes in a hundred trials, which is about what we would expect. But two of the successes are within five trials of each other, and conversely there is a run of 26 failures between the third and fourth. It's not that unusual to see a run of more than p replications between spurious successes (type I errors).

You can confirm this yourself with a simple die experiment. Roll a die repeatedly and see if you can roll it seven or more times before an ace comes up.

[T'ai Chi]

[QUOTE]Originally posted by new drkitten
Yes, I know. So what?
[quote]

So, saying things like "Come back after 3p attempts and we'll definitely talk" doesn't make sense. If p = .001, you are suggesting I come back after .003 attempts??

[T'ai Chi]

Originally posted by Carn

Wait a few decades before you try to prove Randi is cheating that way.


I don't much care for your suggestion that I am trying to prove cheating, or prove anything, for that matter.

[drkitten]

Originally posted by jzs


So, saying things like "Come back after 3p attempts and we'll definitely talk" doesn't make sense. If p = .001, you are suggesting I come back after .003 attempts??

Here's what I said :


If the chance of success is 1/p, then the chance of someone succeeeding "by chance" in p attempts is approximately 70%.

Come back after 3p attempts and we'll definitely talk.


If the chance of success is 0.001, then it's 1/p, meaning p in the context of the quoted post is 1000. So after three thousand attempts, suspecting chicanery is legitimate if the alpha cutoff were uniformly 0.001.

Of course, the alpha cutoff is not uniformly 0.001, so a much more detailed analysis would still need to be done even after 3000 unsuccessful trials in order to establish what the actual expected number of type I errors would be.

[T'ai Chi]

Originally posted by CFLarsen
One thing at a time.

I'll await Randi's reply, then.

Chip Denman responded.

To paraphrase, he said that alpha for the the preliminary tests has varied, and to the best of his knowledge alpha has always been .001 or less.

Now that you've "awaited" the reply, what will you do about it?

[T'ai Chi]

Originally posted by Carn

Couldn't JREF miss some real para ability due to applicants misjudging their ability?


That could be an issue. JREF says that no judging will be done in the test because the results are self evidence to any observer. But if they test the applicants judgement of being so and so % successful, they have already introduced judgement into the picture.

I think another issue is that if there is a small effect, the sample sizes involved are way too tiny to detect it.

[Walter Wayne]

Originally posted by jzs


That could be an issue. JREF says that no judging will be done in the test because the results are self evidence to any observer. But if they test the applicants judgement of being so and so % successful, they have already introduced judgement into the picture.

I think another issue is that if there is a small effect, the sample sizes involved are way too tiny to detect it. [/B] If the effect is too tiny to detect with a small sample size, the individual wouldn't be aware that he is psychic, able to dowse, ...

That has always amused me about some of those with paranormal abilities. They are aware they have them, but yet you need to do 10 000 trials to detect it. How the heck do they know they have them?

Anyways, if I believe that I can predict a coin flip result 90% of the time, I haven't gotten that impression because I can predict it 50.01% of the time. Igot that impression from being tricked by others or myself.

Walt

[Carn]

Originally posted by jzs
I don't much care for your suggestion that I am trying to prove cheating, or prove anything, for that matter.

Sorry for misguessing your intent.

So we can conclude from this thread without any great difference in opinion:

-if too many, relativ to the average chance value for passing, applicants fail to pass prelim, we have to start suspecting fraud on JREF's behalf. But the number of applicants so far is at most 1/3 of the number we need to have before starting to suspect fraud.

-if the applicants idea, what they can do, is very far from what they realy can do(e.g. dowser thinks he is 50% better than chance, while he is actually only 5% better than chance), they are likely to fail the JREF prelim, although they have an ability.

Any objections?

Carn

[Carn]

Originally posted by jzs



I think another issue is that if there is a small effect, the sample sizes involved are way too tiny to detect it. [/B]

jzs, a effect can always be so small, that some test is unable to detect it. If every human posseses the ability to influence a coin flip with his thoughts so, that it will have 50,00000000000000000000000000000000000000000000000 0000
000000000000000000000000000000000000000000001% to show one side, then we all have paranormal abilities, but no test will ever detect it.
It would take more than 10**95 coin throws to detect it, there is no test that would prove or disprove the existance of such an ability and likely there never will be. But still it would mean that para abilities exists.
So never we can be certain that no para ability exists, but we maybe one day be certain and are already with specific claims, that no para ability exists currently, that does influence the live of the vast majority of humans.
E.g. if we can all predict coin throws with a probabilty of 50,1%, this will be irrelevant for practical purposes.
E.g. if there is only one dowser, who is actually realy able to dowse and is 90%+ of the time correct, that is irrelevant, because you are unlikely to get hold of him, there are several million other potential customers and he can only dowse 4-10 different locations per day.
Both things are not yet tested(no psi test goes for prediction of 0.1% above chance, not all dowsers have yet been tested), but already tested is, that we all cannot predict coin throws 55% of the time right and that there are not thousands of dowsers, who are 90%+ of the time correct.

The gap for any practially relevant para ability is getting closer and the more closer it gets, the more certain we can be that most things told about para abilities are wrong, because nearly all para abilities are said to be detected and used in practical life. For that purpose, the JREF test are fine.

As said always there could be weak para abilities existing, but the only support for this specualtion are tellings of practial relevant para abilites. But if these are mostly or all bogus, then there is no support for suspecting weak para abilities, it the is only idle speculation as useful as thinking about pink invisible unicorns.

Carn

[drkitten]

Originally posted by Walter Wayne
If the effect is too tiny to detect with a small sample size, the individual wouldn't be aware that he is psychic, able to dowse, ...


You're assuming that these people have determined their psychic abilities by scientific testing. I suspect a number of them "found" their powers by being told about them -- either by a person that they trusted (such as a guru or priest), or else by a spirit guide or something.

You know the drill -- "And the angel came in unto her, and said, Hail, thou that art highly favoured, the Lord is with thee: blessed art thou among women. And when she saw him, she was troubled at his saying, and cast in her mind what manner of salutation this should be. And the angel said unto her, Fear not, Mary: for thou hast found favour with God. And, behold, thou shalt have the ability to predict lottery numbers, and to find cell-phones underneath buckets, and to predict the future in extremely vague terms that can never be disproven, and to speak to the dead in such a way that the multitude believe that thou art answered therefrom."

I believe that this particular section of the book of Luke was substantially edited by later redactors.

[CFLarsen]

Originally posted by jzs
Chip Denman responded.

To paraphrase, he said that alpha for the the preliminary tests has varied, and to the best of his knowledge alpha has always been .001 or less.

Now that you've "awaited" the reply, what will you do about it?

Not so fast. You claimed that:

Originally posted by jzs
It doesn't matter a bit that "each claim is different" when setting an alpha.

Now, Chip Denman has responded that:

1) alpha for the the preliminary tests has varied
2) alpha has always been .001 or less

Do you admit that you were wrong?

Do you also admit that you can't extrapolate from just one test?

[Walter Wayne]

Originally posted by new drkitten
You're assuming that these people have determined their psychic abilities by scientific testing. I suspect a number of them "found" their powers by being told about them -- either by a person that they trusted (such as a guru or priest), or else by a spirit guide or something. I'm not assuming they found it by scientific testing. After the virgin Mary had been informed she would bare a son who would predict lottery numbers she later observed the ability in her son. So did her son.

My assumption is that when she and her son observed it in action, it wasn't because they saw a statistical proof that he could predict the lotto 6/49 number 1 in 13983815 times, instead of the 1 in 13983816 expected by chance.

All these people claim to observe the powers effects in their lives after they have been informed of it. If this is the case we don't need the ability to resolve such minute deviation from chance.

Walt

[T'ai Chi]

Originally posted by CFLarsen

Now, Chip Denman has responded that:


You're slipping Claus! ;) You take what I said at face value, without the evidence in the form of the actual email that Denman sent me. It must support your argument, so you don't care about the evidence..


1) alpha for the the preliminary tests has varied
2) alpha has always been .001 or less

Do you admit that you were wrong?


I also ask you, later on, to consider my argument using the average alpha. So if I talk about an average alpha, Claus, I am acknowledging that there is the possibility that they can differ- else why bring up the average, Claus?

Actually determining alpha depends upon an analysis of the relative costs of the two types of errors (type 1 and type 2).

Now, Claus do you wish to address the actual argument I put to you, that of: what would you think if after 1/(min alpha) trials still no one has beaten the prelim?

[drkitten]

Originally posted by Walter Wayne
I'm not assuming they found it by scientific testing. After the virgin Mary had been informed she would bare a son who would predict lottery numbers she later observed the ability in her son. So did her son.


This is where psychological issues like confirmation bias come in. If (for whatever reason) someone believes that they are "lucky" or "a good coin-tosser" or "a dowser," or whatever, then they will experience times both when they are successful and when they are unsuccessful. If they're human, they're likely to attribute their successes to their innate abilities (and to remember them), while attributing their failures to some sort of external circumstance (and then to forget them), without ever bothering to do any sort of formal numerical reality check.

One doesn't even need to invoke the paranormal to see this effect. Most of us, in fact, the overwhelming majority, believe that we have a better than average sense of humor (and for that matter are better than average drivers). Who among us has actually tested our sense of humor against a control population? But we all remember telling jokes that people laugh at, and we (mercifully) forget the ones that just got us dumb looks.....

[drkitten]

Originally posted by jzs

I also ask you, later on, to consider my argument using the average alpha. So if I talk about an average alpha, Claus, I am acknowledging that there is the possibility that they can differ.

Now, do you wish to address the actual argument I raised, that of: what would you think if after 1/(min alpha) trials still no one has beaten the prelim?

Without further information about the distribution of the alpha cutoffs, no further analysis is possible. The word "average" is insufficiently descriptive (what sort of average? what's the underlying distribution?).

But even in a best possible description, 1/(min alpha) trials is insufficient. We would still expect no type I errors in one out of three such runs.

[T'ai Chi]

Originally posted by new drkitten
Without further information about the distribution of the alpha cutoffs, no further analysis is possible.


Well, again, since they don't have/know, or won't release, or don't make the data easily available, I agree that we have insufficient info for complete inquiry.


The word "average" is insufficiently descriptive


Without qualifiers, "average" is the arithmetic mean.

Well, we know that alpha has an upper bound of .001, so 1/alpha has a lower bound of 1000. I'm not sure, at the moment, what else one can get from that.

[Walter Wayne]

Originally posted by new drkitten
This is where psychological issues like confirmation bias come in. If (for whatever reason) someone believes that they are "lucky" or "a good coin-tosser" or "a dowser," or whatever, then they will experience times both when they are successful and when they are unsuccessful. If they're human, they're likely to attribute their successes to their innate abilities (and to remember them), while attributing their failures to some sort of external circumstance (and then to forget them), without ever bothering to do any sort of formal numerical reality check. This is what I'm talking about. We don't need to have the ability to test claims below a certain threshold, but cause gods, angels and exposure to nuclear waste don't give us powers to do significantly better than chance over a million trials. They realize that would be useles in our life and are nice enough to give us significant powers to deceive the public or ourselves with.

Walt

[drkitten]

Originally posted by jzs


Without qualifiers, "average" is the arithmetic mean.


... which is insufficiently descriptive. In order to provide a meaningful estimate of the expected type I errors, I would need information about the distribution of alpha cutoffs about the mean.

The average yearly temperature in Astoria, OR is about the same as the average yearly temperature for Buffalo, NY. Does this mean I should pack identically for a trip to Astoria as to Buffalo? Even if they're in the same month?

[Metullus]

Originally posted by new drkitten
... which is insufficiently descriptive. In order to provide a meaningful estimate of the expected type I errors, I would need information about the distribution of alpha cutoffs about the mean.

The average yearly temperature in Astoria, OR is about the same as the average yearly temperature for Buffalo, NY. Does this mean I should pack identically for a trip to Astoria as to Buffalo? Even if they're in the same month?

Well, yeah!

Shorts and a t-shirt.

No matter what month.

[T'ai Chi]

Originally posted by new drkitten
... which is insufficiently descriptive. In order to provide a meaningful estimate of the expected type I errors, I would need information about the distribution of alpha cutoffs about the mean.


We don't have the alpha values, that is the whole point! JREF doesn't have or know or want to provide them.

[Walter Wayne]

Originally posted by jzs


We don't have the alpha values, that is the whole point! JREF doesn't have or know or want to provide them. [/B]JREF can't provide alpha values because every test has a different alpha value Depending on the claim it may be simpler to use a different alpha value. A lower limit is worth noting, but providing an average on future tests is ridiculous.

[T'ai Chi]

Originally posted by Walter Wayne
JREF can't provide alpha values because every test has a different alpha value


Some are different. But why couldn't they be provided?

[Winny]

Originally posted by jzs


We don't have the alpha values, that is the whole point! JREF doesn't have or know or want to provide them. [/B]
Isn't the alpha a bit hard to establish for some of these events?

For example, somebody is presently predicting that an asteroid is going to hit Nowheredoon, Scotland. What odds can you put on that?

Winny

[EHocking]

Originally posted by Carn
Sorry for misguessing your intent.

So we can conclude from this thread without any great difference in opinion:

-if too many, relativ to the average chance value for passing, applicants fail to pass prelim, we have to start suspecting fraud on JREF's behalf. But the number of applicants so far is at most 1/3 of the number we need to have before starting to suspect fraud.Carn, we cannot conclude this, since you have not determined what the "average chance value for passing" is. Rather the point of your post I think? That the proponents may not have sufficient mathematical knowledge to adjudge the probablility figures of a challenge?

-if the applicants idea, what they can do, is very far from what they realy can do(e.g. dowser thinks he is 50% better than chance, while he is actually only 5% better than chance), they are likely to fail the JREF prelim, although they have an ability.While it may be true that the dowser in question would lose the Challenge, part of the Challenge process, in this case, would be to determine what the chance probabilities for the protocol were. It is not unprecedented that a dowser's performance that has been "significant" is invited to continue with another set of trials. OK, no prize so far, but certainly not dismissed out of hand either.

Any objections?

Carn Uh, you probably need to change the opening statement, "So we can conclude from this thread without any great difference in opinion"?

[drkitten]

Originally posted by Carn

-if too many, relativ to the average chance value for passing, applicants fail to pass prelim, we have to start suspecting fraud on JREF's behalf. But the number of applicants so far is at most 1/3 of the number we need to have before starting to suspect fraud.


This is something of an understatement. According to the JREF records on another thread, there have been approximately 350 properly executed applications, most of which never come to the testing stage. In Kramer's tenure as Lord High Tester, he has actually executed one test (in a period of approximately a year). The actual number of tests conducted is, at a guess, fewer than fifty in total.

If ten thousand people apply, but only two actually proceed to the testing stage, then there's still no reason to suspect test fraud. The point of concern is not the number of applicants, but the number of tested applicants.

When that number gets to approximately 3000, I will be happy to investigate seriously the possibility of test fraud on the part of JREF. If I'm not too busy playing checkers in a nursing home, or as is even more likely, long since dead.