determining what constitutes a success

[latent aaaack]

This seems like a question that would've been posed here a lot but I don't see it on the first page here. Can anyone who's more familiar with the challenge comment on how it's determined that an event that a claimant causes is unlikely enough due to chance to constitute a success?

For example if someone claims to be able to predict outcomes of coin flips obviously it wouldn't be considered a success if they successfully predicted 5 coin flips, but how many flips would have to be accumulated and what would the final probablity of the guesses/outcomes have to be (after all the flips) to be considered a success?

And if the claimant one time successfully guesses a 10 digit number I'm assuming that wouldn't be considered a success either because it still could be a fluke, but how many times would he/she have to do that for it to be verified as a success?

Thanks to anyone with a better sense of this who answers.

[TobiasTheViking]

Depends on the odds.
But for coinflips it would probably be something like getting 80-90%+ of some 20 coin tosses correct.

10± a bit would be average just on chance. with the ± a bit guessing 12-13 correct isn't good enough, but go to 18 out of 20 then it is closer.


NOTE: i'm just some guy, and i'm sure Darat or someone else will correct me later. :D

[egslim]

The number of accurately predicted coinflips necessary to constitute a succes would be predetermined and mutually agreed upon by the JREF and the claimant.

The claimant will have to state his expected succesrate, then JREF will calculate the number of tries and successes necessary to meet the required odds. For example, it takes far fewer tries to demonstrate/disprove a 100% succesrate than a 70% succesrate.

[William Smith]

This seems like a question that would've been posed here a lot but I don't see it on the first page here. Can anyone who's more familiar with the challenge comment on how it's determined that an event that a claimant causes is unlikely enough due to chance to constitute a success?

For example if someone claims to be able to predict outcomes of coin flips obviously it wouldn't be considered a success if they successfully predicted 5 coin flips, but how many flips would have to be accumulated and what would the final probablity of the guesses/outcomes have to be (after all the flips) to be considered a success?

And if the claimant one time successfully guesses a 10 digit number I'm assuming that wouldn't be considered a success either because it still could be a fluke, but how many times would he/she have to do that for it to be verified as a success?

Thanks to anyone with a better sense of this who answers.

Welcome to the Forum, latent aaaack.

The success rate depends on the claim - and the cockyness of the applicant. Tobias already gave you the ballpark.

Perhaps you will find threads that spark your interest on other pages int he Forum. Enjoy the search function. Take your time.

For an overview and an insight into what success rate the JREF asks from applicants, check out the Challenge Application Forum. Look for the threads with several posts, as the negotiations between the JREF and the applicant have made at least some progress and therefore hold useful information for you.

The last two tests took place with applicants named Achau Nguyen and Angela Patel.

Enjoy the Forum.

[Thomas]

This seems like a question that would've been posed here a lot but I don't see it on the first page here. Can anyone who's more familiar with the challenge comment on how it's determined that an event that a claimant causes is unlikely enough due to chance to constitute a success?

For example if someone claims to be able to predict outcomes of coin flips obviously it wouldn't be considered a success if they successfully predicted 5 coin flips, but how many flips would have to be accumulated and what would the final probablity of the guesses/outcomes have to be (after all the flips) to be considered a success?

And if the claimant one time successfully guesses a 10 digit number I'm assuming that wouldn't be considered a success either because it still could be a fluke, but how many times would he/she have to do that for it to be verified as a success?

Thanks to anyone with a better sense of this who answers.
Here's a laws of chance table (http://www.automeasure.com/chance.html).

And welcome to the boards.

[T'ai Chi]

The claimant will have to state his expected succesrate,


I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

[Flange Desire]

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

Tai:
I claim I can view skullplates using xray vision,
but in fact can do no such thing.
(yes, in this example my belief is contrary to reality, same as all woos)
I state my expected success rate as 93.276%.
I can really perform at 0%.
When tested I have a success rate of around 10%.
I am performing acceptably!!!
Whould YOU say that I passed or failed?

[Rasmus]

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

You are assuming that just by chance they anyone would perform at around 50% or less?

I think a proper protocol could easily set a low enough margin, even though I think I have seen instances where this was doubtful.

But it really don't matter one way or another: They will not perform any better than chance, and if they do, they can still re-apply another year.

(Funny, someone is supposedly able to bend and break the known laws of the physical universe, but they mess up when it comes to simply calculating a lousy percentage ...)

[T'ai Chi]

You are assuming that just by chance they anyone would perform at around 50% or less?


No, I did not assume that.


They will not perform any better than chance, and if they do, they can still re-apply another year.


"They will not" and "if they do" are contradictory.

[Donks]

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.
To the best of your knowledge, has any claimant performed significantly better than random chance? You decide how to define significantly, just state what your definition is.

[nathan]

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

The claimant would also say they failed -- their claim was 'predict 80%'.

[William Smith]

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

In recent protocol negotiations, 70% accuracy was considered an acceptable performance for a preliminary test. KRAMER himself brought it up during correspondence with applicants.

[Hellbound]

Well, back on the original topic (and ignoring the troll and it's baiters :)), I believe it's been stated that the odds of winiing by chance are adjusted to be less than 1 in 1000 for the preliminary and 1 in 1,000,000 for the final test. You can use something like the aforementioned laws of chance table to determine what it would take for that.

If I claim to be able to predict coinflips at 100% accuracy, then I'd have to be able to correctly predict 10 out of 10 flips for the preliminary, and 17 out of 17 for the final. Now, the JREF will tend to vary this so you can miss a few, but I'm not sure how to calculate "X out of Y" hits chances, but that should give you an estimate.

[T'ai Chi]

To the best of your knowledge, has any claimant performed significantly better than random chance?

Who knows, datasets on performance apparently aren't released.

[CFLarsen]

Note that T'ai Chi avoided Flange Desire's question.

[CFLarsen]

Who knows, datasets on performance apparently aren't released.

Anyone can go to Florida to see all the records. You are perfectly aware of that.

[William Smith]

Note that T'ai Chi avoided Flange Desire's question.

It should read: "Note that T'ai Chi ignored Flange Desire's order." :rolleyes:

[Thomas]

Who knows, datasets on performance apparently aren't released.
Or apparently they are, the problem is just that very few people actually show up for the preliminary test. Here's from a few cases where the applicants decided to show up:


Result of the Hans-Peter Borer test
This is to inform you about the test of the Swiss candidate Hans-Peter Borer, which took place last Saturday. He claimed to find a (switched-on) mobile phone with the help of a pendulum.

We gave him our standard test, consisting of 13 trials which in turn consist of a 1:10 set up each.

His result: 2 hits.
http://forums.randi.org/showthread.php?t=29864



Result of the Achau Nguyen test
To tabulate the results, we brought out a white scoreboard which displayed columns for “Word Sent,” “Word Received,” and “Running Score”. The sequence “sent,” “received,” “score” was read for all 20 words. No “Sent” words were even close to the “Received” words. (e.g. The first word he sent was “ovary”, though E. received “shopping mall.”) The running score became a column of zeros.
http://forums.randi.org/showthread.php?t=28936
http://www.csicop.org/specialarticles/nguyen.html



Result of the Angela Patel test
When the time then came to reveal the results there will be no surprises when I say that Angela scored zero out of three. She was keen to see if the real addresses were close to what she had discovered, but she remarked with disappointment, "they weren't even on the same page!"
http://forums.randi.org/showthread.php?t=34609



Look, I know you're just a troll, but we have to take turns in exposing your lies sometimes.

ETA: Ooh, Claus was there before me I see now, who would have known ;)

[Ririon]

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.
Claiming ignorance about basic statistics and having a homepage named statisticool? Cute...

[SkepticScott]

I think the answer depends on what the claim is and what the chance of it happening by chance is. If you claim to be able to dowse water with a W% success rate, then the test will be constructed with X trials and Y containers, one of which contains water, and you will have to succeed in Z of the trials. The exact values for W, X, Y and Z would be probably be determined by JREF's statistical expert(s).

I think the chances mentioned above are correct, the chance of passing the preliminary test by chance is 1 in 1 000, and the chance of passing the challenge by chance would be 1 in 1 000 000.

Even if you fail you could "win". Say you claim to be able to predict coin flips 100% of the time but only succeed 80% of the time, that would be enough to get some scientists interested in studying your ability. You could also re-apply in a year, claiming a 70% chance of success, and you'd win the challenge then.

[Flange Desire]

It should read: "Note that T'ai Chi ignored Flange Desire's order." :rolleyes:

"Whould YOU say that I passed or failed?" looks a lot like a question to me.

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

My concern is the implication of unfairness in Tais last sentence.
Tai says that the claimant is really perfoming acceptably, because they were incorrect about their own assesment.
I say that if the ball does not go through the goalposts then it is not a goal,
regardless of how well you think you can kick a ball, and regardless of how much you miss by.

[matt-r]

T'ai, when you say you want to know what the claimants can do, rather than whether they can do what they claim, are you saying that the JREF should test them for any level of paranormal ability? Because if that's what you're saying then (from my limited statistical knowledge) you are massively increasing the practical difficulties of running tests.

e.g

Someone claims to be able to consistently predict 80% of the tosses of a perfectly fair coin. Now, we all know that if he could actually consistently predict 50.000001% of coin tosses (in a test which absolutely excludes any other factor known to science) then he would have a small measure of paranormal ability. But if you want to test whether someone is able to predict 50.000001% of coin tosses then you have to do a HUGE number of tests in order to be comfortable (with a reasonable measure of confidence) that someone can in fact do that. I'm sure someone here, whose statistical education is more recent than mine, could calculate how many tests would be required to know with 95% confidence (which may be too small for JREF purposes) that someone can predict 50.000001% of coin tosses, assuming they get the bare minimum of correct predictions. I suspect it runs to the millions.

On the other hand, it's remarkably easy to test someone's claim of 80% correct predications. The number of trials needed for 95% confidence is probably well under a hundred (that's just a guess - it's nearly 10 years since I did my A-level in statistics, and I've since gone into law where there's little call for it!).

Now of course, the example you gave cited someone who could actually predict 60% of coin tosses (which I suspect would still be testable with a non-ludicrous number of trials), but your argument would seem to extend right down to my 50.000001% example. Any boundary you draw that is below the claimant's claimed success rate is arbitrary.

Now of course, the above doesn't really help the JREF if they run up against an applicant who actually claims to be able to predict coin tosses with a 50.000001% success rate - but there are remarkably few of them about!

Matt

[William Smith]

"Whould YOU say that I passed or failed?" looks a lot like a question to me.
...



Come on TC, answer the direct question.

I don't take orders.
Try again.

QED. :D

[jmercer]

Success or failure is based on a customized set of parameters agreed upon during each individual protocol negotiation, is it not? There are some basic standards that JREF uses for repetitive claims - no need to re-invent the wheel - but even those are subject to negotiation as long as the agreement doesn't compromise the measure of success/failure to the point where random chance results would be indistinguishable from a real success.

Besides, even random chance won't cut it - you need to succeed with the preliminary test, then do a repeat performance for the final test. The odds of random chance "winning" consecutively like that have got to be astronomical.

[Bindamel]

T'ai, when you say you want to know what the claimants can do, rather than whether they can do what they claim, are you saying that the JREF should test them for any level of paranormal ability? Because if that's what you're saying then (from my limited statistical knowledge) you are massively increasing the practical difficulties of running tests.

e.g

Someone claims to be able to consistently predict 80% of the tosses of a perfectly fair coin. Now, we all know that if he could actually consistently predict 50.000001% of coin tosses (in a test which absolutely excludes any other factor known to science) then he would have a small measure of paranormal ability. But if you want to test whether someone is able to predict 50.000001% of coin tosses then you have to do a HUGE number of tests in order to be comfortable (with a reasonable measure of confidence) that someone can in fact do that. I'm sure someone here, whose statistical education is more recent than mine, could calculate how many tests would be required to know with 95% confidence (which may be too small for JREF purposes) that someone can predict 50.000001% of coin tosses, assuming they get the bare minimum of correct predictions. I suspect it runs to the millions.

On the other hand, it's remarkably easy to test someone's claim of 80% correct predications. The number of trials needed for 95% confidence is probably well under a hundred (that's just a guess - it's nearly 10 years since I did my A-level in statistics, and I've since gone into law where there's little call for it!).

Now of course, the example you gave cited someone who could actually predict 60% of coin tosses (which I suspect would still be testable with a non-ludicrous number of trials), but your argument would seem to extend right down to my 50.000001% example. Any boundary you draw that is below the claimant's claimed success rate is arbitrary.

Now of course, the above doesn't really help the JREF if they run up against an applicant who actually claims to be able to predict coin tosses with a 50.000001% success rate - but there are remarkably few of them about!

Matt

Coin flip requirements based on requiring the test to be at the 1 in a million range of chance occurrence:

84% -- 50 trials
61% -- 500 trials
54% -- 5,000 trials
51% -- 50,000 trials
50.4% -- 500,000 trials
50.1% -- 5,000,000 trials

Caveat: I think I did this right, but I would be happy to stand corrected if I did not.

[matt-r]

Thanks Bindamel, that's just what I needed!

T'ai, doesn't that answer why the JREF tests claims largely based upon the success rate claimed, rather than in an effort to establish whether there is a minute, unexplained effect? Even if you try to test for a 51% success rate, you're massively increasing the number of tests required.

In any event, if someone claims 80% success and it's clear that they cannot achieve that rate (or anywhere near that rate) of success, why shouldn't the JREF, quite reasonably, conclude that the applicant is probably a fantasist and that to go ahead and test for a very small paranormal ability is a waste of their time? Is it really very likely that the applicant is someone with the true ability to predict 51% of coin tosses, but they are simply so self-deluded that they claim an 80% success rate? I'm far more inclined to conclude that they are totally deluded, and have no paranormal ability whatsoever. But hey, that's what makes me a skeptic. :D

Matt

P.S. As a semi-related aside, many people on this board at many times have stated that, despite being skeptics, they think it would be really cool if there really were people out there with paranormal powers (a sentiment with which I wholeheartedly concur). But wouldn't it just be a real kick in the teeth if there were people out there with paranormal powers, but that they were so slight as to serve absolutely no useful function within human society? What if we discovered that the only paranormal ability that REALLY exists is spoon-bending (and it doesn't work on forks or knives, or even large spoons for that matter). That would be the most depressing thing ever - far more depressing to my mind than the likely reality that no one has cool psychic powers. I think I'd prefer not to know if that was the case!:p

[SkepticScott]

The other question that this brings up is if someone thinks they have a minute ability, like being able to predict coin flips 51% of the time, how did they discover it in the first place? No one records and analyzes 50,000 coin flips for no reason.

[Ririon]

The other question that this brings up is if someone thinks they have a minute ability, like being able to predict coin flips 51% of the time, how did they discover it in the first place? No one records and analyzes 50,000 coin flips for no reason.
Why not? You could do that in less than a week. And then you would have to replicate it to be sure... People do stranger things than that. Maybe not more boring, though. If I was volunteering to conduct a trial, I would not be happy if Randi agreed to a protocol like that!

[drkitten]

Coin flip requirements based on requiring the test to be at the 1 in a million range of chance occurrence:

84% -- 50 trials
61% -- 500 trials
54% -- 5,000 trials
51% -- 50,000 trials
50.4% -- 500,000 trials
50.1% -- 5,000,000 trials

Caveat: I think I did this right, but I would be happy to stand corrected if I did not.

There's a minor issue that sort of damns this whole enterprise. I didn't bother to check your math -- but as near as I can tell, you're doing the sums based on the idea that a coin flip is a 50/50 proposition? (In other words, your null hypothesis is a 50/50 split?)

This is where part of the problem with T'ai's whole approach comes in. If I claim to be able to predict coin flips with 51% accuracy -- well, THAT may not be paranormal. Coin flips aren't actually 50/50, depending upon how you flip them. Even a "fair" flip is biased to land 'tails' (typically) because the weight distribution of the coin is uneven. Try it yourself with 50,000 flips and see!

Unfortunately, this means that T'ai's hope to do a grand metaanalysis simply won't work, because we don't know the proper null hypothesis to apply for many of the tests that have been done.

[T'ai Chi]

T'ai, when you say you want to know what the claimants can do, rather than whether they can do what they claim, are you saying that the JREF should test them for any level of paranormal ability? Because if that's what you're saying then (from my limited statistical knowledge) you are massively increasing the practical difficulties of running tests.


See my example why. The person could be mistaken about their beliefs (in fact, it is known that people are regularly mistaken, so this seems an obvious point).

The scenarios don't make difficulties go away. Someone can simply say 'I believe I can get 50.000000001%' and then what? Obviously, in the real world we live in, there are limited resources, and either way we'd have to take those into account.

[T'ai Chi]

In any event, if someone claims 80% success and it's clear that they cannot achieve that rate (or anywhere near that rate) of success, why shouldn't the JREF, quite reasonably, conclude that the applicant is probably a fantasist and that to go ahead and test for a very small paranormal ability is a waste of their time?


If someone is tested for 80% success and they don't pass, I doubt they'd then claim a smaller percent.

But in these hypothetical scenarios, your guess is as good as mine.

[petre]

See my example why. The person could be mistaken about their beliefs (in fact, it is known that people are regularly mistaken, so this seems an obvious point).

The scenarios don't make difficulties go away. Someone can simply say 'I believe I can get 50.000000001%' and then what? Obviously, in the real world we live in, there are limited resources, and either way we'd have to take those into account.

Well, we'd have to take those into account if our intent is to demonstrate conclusively that there is nothing paranormal about applicants or their abilities.

However, if someone were to create an orginization with the intent to simply TEST WHAT THEY CLAIM, then perhaps we could ignore such possibilites and simply TEST THEIR CLAIM.

Perhaps someone with more resources at their disposal will wish to contact such people to examine whether or not there might still be something paranormal, even if they themselves cannot identify it within a degree of success. If you know of any such individuals with an interest in doing so and resources to expend, perhaps you could direct them to contact JREF to put them in touch with interested applicants.

[matt-r]

Well, we'd have to take those into account if our intent is to demonstrate conclusively that there is nothing paranormal about applicants or their abilities.

However, if someone were to create an orginization with the intent to simply TEST WHAT THEY CLAIM, then perhaps we could ignore such possibilites and simply TEST THEIR CLAIM.


You're right of course - but I think it's valid to ask "Why is the JREF's aim to simply test specific claims rather than to be a seeker of ultimate truth?". Superficially, seeking the ultimate truth is an attractive idea. Superficially, it would be good to test coin-tossers to see if they have any ability to predict coin tosses beyond chance and award the million on that basis. T'ai seems to be suggesting that that would be a better goal. Whether he thinks it's a better goal for the community in general, or for the JREF in particular he doesn't make clear, but I've tried to fill in the gaps :D

However, when you get down to it, seeking ultimate truths is damn impractical. One problem is the statistical one I set out above. DrKitten's point is also a good one - sometimes (often?) we simply do not know what the null hypothesis is, and so the JREF has to plump for a suitably high threshold which is intuitively beyond the realms of mere chance.

[CFLarsen]

Tai:
I claim I can view skullplates using xray vision,
but in fact can do no such thing.
(yes, in this example my belief is contrary to reality, same as all woos)
I state my expected success rate as 93.276%.
I can really perform at 0%.
When tested I have a success rate of around 10%.
I am performing acceptably!!!
Whould YOU say that I passed or failed?
This is still ignored by T'ai Chi.

It's "Yes" or "No" question, but apparently that is too hard.

[Cuddles]

You're right of course - but I think it's valid to ask "Why is the JREF's aim to simply test specific claims rather than to be a seeker of ultimate truth?". Superficially, seeking the ultimate truth is an attractive idea. Superficially, it would be good to test coin-tossers to see if they have any ability to predict coin tosses beyond chance and award the million on that basis. T'ai seems to be suggesting that that would be a better goal. Whether he thinks it's a better goal for the community in general, or for the JREF in particular he doesn't make clear, but I've tried to fill in the gaps :D

However, when you get down to it, seeking ultimate truths is damn impractical. One problem is the statistical one I set out above. DrKitten's point is also a good one - sometimes (often?) we simply do not know what the null hypothesis is, and so the JREF has to plump for a suitably high threshold which is intuitively beyond the realms of mere chance.

There are many people devoted to finding any evidence of the supernatural. Some of them are genuine scientists working in respectable labs/universities and not just woos who already believe. Randi just takes a different approach that is as much to do with discrediting the more outspoken frauds as it is to do with providing evidence for anything.

As you have said, seeking ultimate truths is damn impractical. Presumably Randi feels this is best left to professional scientists, while his prize is more suited to targeting people who claim easily observable results. Offering a large sum of money is also more likely to appeal to those who want publicity and fame rather than those who want the ultimate truth, which is exactly the sort of person Randi wishes to debunk (unless they are right of course :p).

[Jehanne]

There's a minor issue that sort of damns this whole enterprise. I didn't bother to check your math -- but as near as I can tell, you're doing the sums based on the idea that a coin flip is a 50/50 proposition? (In other words, your null hypothesis is a 50/50 split?)

This is where part of the problem with T'ai's whole approach comes in. If I claim to be able to predict coin flips with 51% accuracy -- well, THAT may not be paranormal. Coin flips aren't actually 50/50, depending upon how you flip them. Even a "fair" flip is biased to land 'tails' (typically) because the weight distribution of the coin is uneven. Try it yourself with 50,000 flips and see!

Unfortunately, this means that T'ai's hope to do a grand metaanalysis simply won't work, because we don't know the proper null hypothesis to apply for many of the tests that have been done.

I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

1) I will pray to my deity and let him know that we are going to conduct an experiment to prove his existence. He will then inform me, in advance, of the outcome of that experiment. I will write the 100 "heads & tails" results (that is, the exact sequence of "heads & tails" that will occur) down on a paper of paper, seal that in an envelope, and then mail it to your office.

2) You conduct the "coin toss" experiment with a coin of your choosing and at a time and place of your choosing, recording the results.

3) If the results of your experiment agree exactly with my a prior claims, then we replicate the experiment again. I pray to my deity a second time, find out from him/her/it what the second sequence of "Heads & Tails" will be, write that information down, seal it in an envelope, mail it to your office, etc.

4) If the replication agrees with my second results, I win the $1 million dollars (which I will donate, in its entirety, to charity)!

5) If not, we agree that I will never perform this experiment again nor will the James Randi foundation agree to a "third" attempt.

[TobiasTheViking]

I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

1) I will pray to my deity and let him know that we are going to conduct an experiment to prove his existence. He will then inform me, in advance, of the outcome of that experiment. I will write the 100 "heads & tails" results (that is, the exact sequence of "heads & tails" that will occur) down on a paper of paper, seal that in an envelope, and then mail it to your office.

2) You conduct the "coin toss" experiment with a coin of your choosing and at a time and place of your choosing, recording the results.

3) If the results of your experiment agree exactly with my a prior claims, then we replicate the experiment again. I pray to my deity a second time, find out from him/her/it what the second sequence of "Heads & Tails" will be, write that information down, seal it in an envelope, mail it to your office, etc.

4) If the replication agrees with my second results, I win the $1 million dollars (which I will donate, in its entirety, to charity)!

5) If not, we agree that I will never perform this experiment again nor will the James Randi foundation agree to a "third" attempt.
Well, the test would have to be performed in person. With all individuals present(both you and whoever performs the toss). But besides for that it should be a valid claim for the challenge.

Sincerely
Tobias

[Rasmus]

I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

I doubt that the protocol would be accepted like that. You would have no way to control whether the tossing of the coins was done fairly or if the JREF cheated during the control. So, for a final test at least (and probably for the preliminary test, too) you would have to be present at a test-site.

But I am sure that sending an application that contains a prediction of 10 or 20 correct predictions would certainlyx draw their attention ...

Also, the JREF probably won't care what you do with the money, and if you fail you could still re-apply after a year. There's no point in changing the rules here; if you want to give to charity: great! If you never want to apply again ... well, that would be great if it meant you understood that you have no super-powers.

[William Smith]

I am new here and have been following this thread with much interest! Sorry about "jumping in" right of the bat before introducing myself. But, how about this challenge:

If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:

1) I will pray to my deity and let him know that we are going to conduct an experiment to prove his existence. He will then inform me, in advance, of the outcome of that experiment. I will write the 100 "heads & tails" results (that is, the exact sequence of "heads & tails" that will occur) down on a paper of paper, seal that in an envelope, and then mail it to your office.

2) You conduct the "coin toss" experiment with a coin of your choosing and at a time and place of your choosing, recording the results.

3) If the results of your experiment agree exactly with my a prior claims, then we replicate the experiment again. I pray to my deity a second time, find out from him/her/it what the second sequence of "Heads & Tails" will be, write that information down, seal it in an envelope, mail it to your office, etc.

4) If the replication agrees with my second results, I win the $1 million dollars (which I will donate, in its entirety, to charity)!

5) If not, we agree that I will never perform this experiment again nor will the James Randi foundation agree to a "third" attempt.

First of all: Welcome to the Forum, Jehanne.

If you are serious about your claim - which I also consider both easily testable and meeting the Challenge's criteria - I suggest you start a seperate thread in this subforum in which we can discuss solely your claim.

Again: Welcome.

[jmercer]

You're right of course - but I think it's valid to ask "Why is the JREF's aim to simply test specific claims rather than to be a seeker of ultimate truth?".
It is a valid question - and one with a valid answer. :)

JREF was formed by people who do not believe in the supernatural. The reason they test specific claims is because that's what they do best by design. Seeking ultimate truth falls within the realm of science and/or philosophy; debunking fraudulent claims of the supernatural by JREF is a complementary activity which happens to benefit those who are seeking the ultimate truth.

[Mojo]

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

See my example why. The person could be mistaken about their beliefs (in fact, it is known that people are regularly mistaken, so this seems an obvious point). Yes, it's entirely possible that they are mistaken about their abilities.

If you've followed threads in the Challenge forum, you will no doubt be aware that applicants and potential applicants are routinely advised to perform tests of their own, under similar conditions to an actual challenge test, so that they can figure out for themselves whether they really have the ability they claim, and what success rate they can expect. If they carry out the tests as they are recommended to do, and they actually do have an ability, they should be able to figure out what their success rate is likely to be and to negotiate an appropriate success rate for the challenge.

If they don't bother to test themselves properly before negotiating a target success rate for the challenge, then that's their problem, isn't it?

[SkepticScott]

I am new hereWelcome, Jehanne!
If I claimed that I could predict the exact outcome (that is, the exact sequence of Heads & Tails) of 100 tosses of a "fair coin" of your choosing, would you accept that as evidence of the existence of the paranormal and/or supernatural? The following protocol would apply:
(snip)I can't speak for JREF, but it looks OK to me. You'd have to fill out an challenge application though. Your 'donate to charity' and 'never reapply' might be better kept as private promises to yourself, and you shouldn't try to change the challenge rules for even such terms.

1) I will pray to my deity and let him know that we are going to conduct an experiment to prove his existence.As with any claim, claimants may want to conduct an experiment themselves to see if they might be deceiving themselves about the power. This claim is very easy to test like that.

[Flange Desire]

QED. :D

Aye! Ye canna gae ordering thon wee jock aboot.
Tae fick with he.

[William Smith]

Aye! Ye canna gae ordering thon wee jock aboot.
Tae fick with he.

Oh [Expletive Deleted], I pissed off an Aussie again. :)

Please, please forgive a celtic caveman, oh honourable superhero from the continent that blessed us with the great Virgil Donati but also spat out Columcille Gerard Gibson so you probably made just up for that screw-up I mean I liked Lethal Weapon II III also but that passion stuff and the alleged antisemitc rant pfui can you believe how bad this guy acts and what about all the violence does he get off on it does he get paid by the government to desensitize the population for present and future wars I mean the bringing of peace and democracy did he go weewee in his bed and his mommy beat the living piss out of him or some other quasi-psychodiddlyological ramalamadingdong I mean can you believe this nonsense you know Danny Glover really carried him the whole time and Pesci and Russo well at least the Simpsons parlayed his sorry tuchus into a great episode you know he's just a dummy I know but he sells tickets yeah mad max schmad schmax lethal weapon schmethal schmeapon Air America Schmair Schmamerica Maverick yeah Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Dallas Mavericks ruleDallas Mavericks rule Dallas Mavericks rule Dallas Mavericks rule Maverick Schmaverick Braveheart Schmaveschmart Pocahontas Pukahontas Schmukaschmonschmas Ransom Schmanschmom Payback Schmayback Million Dollar Hotel Dime Brothel Chicken Run Schmicken Schmun Patriot Oh say can you see What Women Want well who knows when they mostly do not even know themselves what they frickin want am I right am I right tough crowd Schmat Schmomen Schmant brrrzzzzz brrrrRRRRR RRzzzzZZZZZ and the weather forecast for Monday: Buy waffles tasty waffles with plenty of maple sirup and schmmmmm brrrzzzzrrrrrbrprrzz ZZZZschmschm schmzzzzz how does this promote the JREF Mission I guess you will have to answer that for yourself what I have learned is that this Forum has a great wingspan tremendous upside potential and freakish atleticism it will have enough chances to prove its abilities in the future as it will face beaucoup challenges every single day every single day every single day bbbrrrmmmschm zzzZZZp pffsssSSSSSss [insert noise of my cat snoring (!) on her pillow, breathing very audible. probably chasing a big fat mouse in her mind, oh how cute, oh my, if this is love it feels great, oh.]
And that's the Wørd.
We'll be right back.
(Fade in music.)

[Flange Desire]

My ire was aimed at tai, not you, you big silly GzuzKryzt!
Yes, old Mel is a bugger for the bottle.

[AgingYoung]

...For example if someone claims to be able to predict outcomes of coin flips obviously it wouldn't be considered a success if they successfully predicted 5 coin flips, but how many flips would have to be accumulated and what would the final probablity of the guesses/outcomes have to be (after all the flips) to be considered a success?From what I understand Randi does a card trick. Although he's merely normal he can do that trick with 100% accuracy. I think Randi would expect that anyone who claims to have a paranormal ability to at least be able to demonstrate it to the extent a normal person can do something. Anything less than 100% accuracy is less than what a normal person can do. It shouldn't be considered paranormal.

Gene

[SkepticScott]

(snip)Anything less than 100% accuracy is less than what a normal person can do.A normal person can't do everything 100% of the time unless it is very simple. For instance, recognizing a word whispered from a few feet away has a success rate that is much better than chance, but less than 100%.

[T'ai Chi]

If they don't bother to test themselves properly before negotiating a target success rate for the challenge, then that's their problem, isn't it?

If they are suspected of not having critical thinking skills, they how can they be expected to carry out their own tests and have a good notion of their own skill assessment?

[William Smith]

If they are suspected of not having critical thinking skills, they how can they be expected to carry out their own tests and have a good notion of their own skill assessment?

"They" can inquire in this very Forum and "they" will most likely find step-by-step help and guidance. The only requirements for "them" are basic social and communication skillz.

[robinson]

Seems true. From reading the FAQ page, - .../research/faq.html#4.2

... you need to have your health checked, you need to discuss your ability/power/whatever with your Doctor, demonstrate it to them and other observers before you even apply. Seems simple to me. Then, if you have already demonstrated something to rational, healthy, scientific minds, you could apply for the challenge.

To quote from the FAQ, -"If you can convince a brutally honest friend that you can do something paranormal, then keep going. Otherwise, stop; you will have no chance convincing the JREF's investigators."

Forgive me if I missed this in some other thread here. While I have not read everything, I have studied the challenge since it was only $10,000 years ago, and I love it!

Not just the big money aspect, but the entire idea of it.

And hello, my first commentary here. I'm a short tempered, crusty old skeptic, so don't worry about hurting my feelings.

[Legalduck]

I'm more interested in what the claimant can do, rather than what they believe they can do. What if they are incorrect about their own assesment. For example, they state their expected success rate of 80%, but they can really perform at 60%.

Then if they get 70%, they are really performing acceptably, but you would say they'd fail.

1) The successes needed is mutually agreed upon. If the applicant doesn't believe they can meet that standard they shouldn't agree to it.

2) This is a challenge. Its up to the applicant to meet the specified goals.

3) What you're proposing is a variant of texas sharpshooter's fallacy (fire your shots, then draw the bullseye). The reason the challenge uses very specific pass criteria is to avoid complaints by applicant's that they "performed acceptably."

[robinson]

I could win the MDC if they changed the rules.



Or if I had some powers science couldn't explain.


Or an ultra cool super secret mind ray hidden on my person.

[William Smith]

Just for kicks: http://www.youtube.com/watch?v=fdVucvo-kDU

[Bryan Shelton]

...I believe it's been stated that the odds of winiing by chance are adjusted to be less than 1 in 1000 for the preliminary and 1 in 1,000,000 for the final test.

I'm new here, and I've been looking around trying to find out what the required odds are for that Million Dollar Challenge. I've read the official rules for the challenge and I've read a few threads in the forums having to do with it, but I haven't seen anything yet stating that the required odds that the results are from chance alone must be 1 in a million, or worse. Can you point me to any official statement of that?

Bryan

[Hellbound]

I'm new here, and I've been looking around trying to find out what the required odds are for that Million Dollar Challenge. I've read the official rules for the challenge and I've read a few threads in the forums having to do with it, but I haven't seen anything yet stating that the required odds that the results are from chance alone must be 1 in a million, or worse. Can you point me to any official statement of that?

Bryan

Nope, no idea :)

I'll look through a bit, see what I can find though. I recall reading it somewhere here, but that's just my recollection. And we all know how reliable memory is...

[Jekyll]

Nope, no idea :)

I'll look through a bit, see what I can find though. I recall reading it somewhere here, but that's just my recollection. And we all know how reliable memory is...

I don't think it's explicitly part of the challenge. It may well be a guideline Randi et al. use in determining if a test is acceptable, but I doubt they'd put it down on the challenge FAQ, there's far to much room for misinterpretation. Can you imagine the woo's? "There's no point entering; Randi won't let anyone compete unless they have less than a 1 in a million chance of winning!"

[drkitten]

I'm new here, and I've been looking around trying to find out what the required odds are for that Million Dollar Challenge. I've read the official rules for the challenge and I've read a few threads in the forums having to do with it, but I haven't seen anything yet stating that the required odds that the results are from chance alone must be 1 in a million, or worse. Can you point me to any official statement of that?

There's no official statement of that nature, partly because for many of the proposed abilities/challenges, calculating probabilities in such a manner is problematic. (For example, one person claimed to be able to make UFOs -- flying saucers -- appear on command. Another person claimed to be able to make it snow in july in Oakland, CA. Can you give me a meaningful odds assessment for that?)

Also, Randi has from time-to-time bent his own rules regarding the strictness of the test criteria in order to make specific offers to specific high-profile charlatans (such as Sylvia Browne).

[Cuddles]

Probability isn't always relevant anyway. If someone claims to be able to levitate all they need to do is show that they can levitate in an environment where they can't be cheating. There is no possibility of this happening by chance, so this criteria (criterion?) wouldn't apply. I assume that since every claim is different Randi feels there is no point trying to fix the odds, as long as the possibility of a chance win is so small that it can be ignored.

[Bryan Shelton]

I find it very odd that they wouldn't explicitly state something as important as that right up front for everyone to see. If the average "woo" (I can guess what that means, from the context) doesn't understand the nuances, that's HIS problem, not JREF's.

Bryan

[SkepticScott]

Ithere's far to much room for misinterpretation. Can you imagine the woo's? "There's no point entering; Randi won't let anyone compete unless they have less than a 1 in a million chance of winning!"And it's so easy to mis-interpret. Most people wouldn't catch the distinction between "winning" and "winning by chance".

[drkitten]

I find it very odd that they wouldn't explicitly state something as important as that right up front for everyone to see.

It's not as important as you seem to think, though.

The whole point of the MDC challenge is that if you have the claimed ability, your chance of succeeding is near-100%. If you do not have the claimed ability, your chance of succeeding is near 0%. The exact meaning of "near" isn't really that relevant. More important are the countermeasures that an experienced magician such as Randi takes to prevent explicit cheating. A conspiracy of genuinely random numbers is not the sort of thing that anyone really needs to worry about. A thousandth, a millionth, a billionth -- they're all pretty good operational definitions of "ain't gonna happen."

[Bryan Shelton]

Probability isn't always relevant anyway. If someone claims to be able to levitate all they need to do is show that they can levitate in an environment where they can't be cheating. There is no possibility of this happening by chance, so this criteria (criterion?) wouldn't apply.

I understand that, but of course I'm referring to claims in which there IS a strong element of chance, like if someone were to claim that they can predict the outcome of coin tosses with a 60% accuracy.

I assume that since every claim is different Randi feels there is no point trying to fix the odds, as long as the possibility of a chance win is so small that it can be ignored.

Yes, but what is "small enough to be ignored"?? There still needs to be an acceptable level of risk that's clearly stated in advance. It should be the same for every test. I see no reason why it isn't right upfront for everyone to see.

Bryan

[Bryan Shelton]

It's not as important as you seem to think, though.

The whole point of the MDC challenge is that if you have the claimed ability, your chance of succeeding is near-100%. If you do not have the claimed ability, your chance of succeeding is near 0%. The exact meaning of "near" isn't really that relevant.

Oh really?? What if I claim that I have a remarkable ability to predict the outcome of coin tosses with a 60% accuracy? That suddenly becomes a HIGHLY STATISTICAL claim, one requiring a great many trials to prove or disprove with a high degree of certainty.

More important are the countermeasures that an experienced magician such as Randi takes to prevent explicit cheating. A conspiracy of genuinely random numbers is not the sort of thing that anyone really needs to worry about. A thousandth, a millionth, a billionth -- they're all pretty good operational definitions of "ain't gonna happen."

Do you think Randi is going to risk a million bucks on a 1-in-1,000 shot? That's not the impression I've gotten here. Two or three others have said that for the Big Test, it's going to be 1-in-a-million.

Bryan

[drkitten]

Yes, but what is "small enough to be ignored"?? There still needs to be an acceptable level of risk that's clearly stated in advance.

No, it need not be stated in advance. Since the "risk" involved is purely Randi's (it's the risk that he will pay out his money for a negative result), he can set it at his sole discretion, as long as the applicant agrees.

It should be the same for every test.

Again, no. Depending upon the type of test to be run, it may not be practical or cost-effective to adjust the risk that exactly.

As long as the risk is "acceptable" to Randi (and secondarily to the claimant, although there's little reason for the claimant to care), the test remains valid.

[drkitten]

Oh really?? What if I claim that I have a remarkable ability to predict the outcome of coin tosses with a 60% accuracy? That suddenly becomes a HIGHLY STATISTICAL claim, one requiring a great many trials to prove or disprove with a high degree of certainty.

I can't tell if you're deliberately misreading or not. But you're confusing the odds named in the claim with the odds of passing.



Do you think Randi is going to risk a million bucks on a 1-in-1,000 shot?

He has. Or, at least, he's offered to. Sylvia didn't accept the claim (for obvious reasons -- a 1 in a 1,000 shot still leaves her with scarily high chance of being the Uri Geller of the early 21st century).

[Bryan Shelton]

I can't tell if you're deliberately misreading or not. But you're confusing the odds named in the claim with the odds of passing.

What makes you think I'm confusing the two? I am not.

He has. Or, at least, he's offered to. Sylvia didn't accept the claim (for obvious reasons -- a 1 in a 1,000 shot still leaves her with scarily high chance of being the Uri Geller of the early 21st century).

Oh, I'm not all that surprised that he might do it on occasion with a very high-profile Sylvia Browne or a Uri Geller type. But he's not going to do it ROUTINELY with the average John Doe who walks in the door. As others have made clear in this thread, his requirements apparently snap right back to a much more stringent level for run-of-the-mill candidates.

Bryan

[Jekyll]

Oh, I'm not all that surprised that he might do it on occasion with a very high-profile Sylvia Browne or a Uri Geller type. But he's not going to do it ROUTINELY with the average John Doe who walks in the door. As others have made clear in this thread, his requirements apparently snap right back to a much more stringent level for run-of-the-mill candidates.

Bryan

Yup. This makes perfect sense, there's not a lot of risk in offering 1:1000 long shots to a couple of people who probably wont take it, but if you allow every crazed lunatic on the internet to have a go the odds would soon stack up against you.

[IXP]

Remember also that in order to win the prize, you have to perform the stunt twice. If the odds of winning by chance are 1 in 1000, there is a 1 in 1,000,000 chance that you will do in in both trials.

Or to look at it another way, in order for the there to be a 1 in 1000 chance to win, you chance in each trial has to be better than 1 in 32. You will not see these odds offered for a preliminary trial.

IXP

[drkitten]

Remember also that in order to win the prize, you have to perform the stunt twice. If the odds of winning by chance are 1 in 1000, there is a 1 in 1,000,000 chance that you will do in in both trials.

Or to look at it another way, in order for the there to be a 1 in 1000 chance to win, you chance in each trial has to be better than 1 in 32. You will not see these odds offered for a preliminary trial.


Again, I'm not sure what the point you're trying to make is.

A better way of looking at it (in practical terms) is that to win the money, you need to perform twice. The probability (of passing by chance) is irrelevant; it's set low enough that you almost certainly can't do it once -- but if you can do what you claim, doing it once, twice, or fifty times is easy.

My "paranormal" power is the ability to burn anything in the kitchen. Even water. Even rocks. Even abstractions like the number seven, or heroism, or gravity. But my cooking skill is what it is -- if I can burn water once, I can burn it twice.

[Hellbound]

Also, I'm pretty sure you're confusing the 1/1000 1/1000000 chances for preliminary/final.

Each test, individually, has a 1 in 1000 chance. So the odds of passing both the preliminary and the final are 1 in 1,000,000 combined (this is my understanding). So when Randi made the offer to Sylvia, he was immediately giving her 1000 times the chance to win.

[Almo]

Note that T'ai Chi avoided Flange Desire's question.

I don't think FD's question fits T'ai's assertion anyway.

[drkitten]

Each test, individually, has a 1 in 1000 chance. So the odds of passing both the preliminary and the final are 1 in 1,000,000 combined (this is my understanding).

My understanding differs somewhat. The target probability for the preliminary is 1:1000. The target probability for the final is a more stringent 1:1000000. So to win both requires a 1:1,000,000,000 (one in a b-for-billion) chance.

[Hellbound]

My understanding differs somewhat. The target probability for the preliminary is 1:1000. The target probability for the final is a more stringent 1:1000000. So to win both requires a 1:1,000,000,000 (one in a b-for-billion) chance.

Do you remember where those numbers came from? I recall coming across them, but I don't remember where, either :)

[drkitten]

Do you remember where those numbers came from? I recall coming across them, but I don't remember where, either

Randi has mentioned them from time to time in his commentaries. However, as I pointed out upthread, they're not written in stone or part of the "official" challenge rules, partly for tactical reasons so that he can continue to yank Sylvia's chain....

[T'ai Chi]

Tai says that the claimant is really perfoming acceptably, because they were incorrect about their own assesment.


No, I am not saying that they are performing acceptably because they were incorrect about their own assessment. I am saying they are performing acceptably because they are performing above their real ability not their claimed/believed ability.


I claim I can view skullplates using xray vision,
but in fact can do no such thing.
(yes, in this example my belief is contrary to reality, same as all woos)
I state my expected success rate as 93.276%.
I can really perform at 0%.
When tested I have a success rate of around 10%.
I am performing acceptably!!!
Whould YOU say that I passed or failed?

It all depends on the specifics of course. If you can frame your test of skill in statistics, one could see if 10% is very statistically significant or not.

[petre]

No, I am not saying that they are performing acceptably because they were incorrect about their own assessment. I am saying they are performing acceptably because they are performing above their real ability not their claimed/believed ability.



It all depends on the specifics of course. If you can frame your test of skill in statistics, one could see if 10% is very statistically significant or not.

Note that though the true reality is explicitly stated ("but in fact can do no such thing"), that is unimportant so long as you can find statistics to support a different conclusion.

[T'ai Chi]

Note that though the true reality is explicitly stated ("but in fact can do no such thing"), that is unimportant so long as you can find statistics to support a different conclusion.

If they "can do no such thing", but their performance (always the evidence!) says otherwise and is significantly beyond what chance dictates, then they win according to the challenge rules. I don't like that either, but that would be a fact.

[Bryan Shelton]

My understanding differs somewhat. The target probability for the preliminary is 1:1000. The target probability for the final is a more stringent 1:1000000. So to win both requires a 1:1,000,000,000 (one in a b-for-billion) chance.

Good god (pardon the expression)! One person's "understanding" differs from another person's "understanding" by three orders of magnitude! :-)

I find it ironic that such uncertainty and lack of precision rears its ugly head on a site which ostensibly goes out of its way to be perfectly clear about the rules of the game in the Million Dollar Challenge. Once again, why in the blue blazes doesn't Randi simply state what the expected odds are going to be, in any test of a statistical nature? It's not like it's necessarily carved into stone tablets or anything...he could include a disclaimer to the effect that the odds _may_ be adjusted in certain cases, but they normally will be set to 1-in-a-million (or 1-in-a-billion, or whatever).

The more I hear about this apparent uncertainty, the more it leaves a rather bad taste in my mouth. If I personally wanted to submit myself for testing, I'd sure as heck want to know what was expected of me right up front, before going through even the early rounds of paperwork, negotiations, etc.

Bryan

[nathan]

I had understood that the preliminary and final tests were supposed to follow identical procedures. After all the contract defining that procedure only defines one procedure. I'm not aware of one that contains words like 'claimant will perform 10 trials in the preliminary and 100 trials in the final test'.

So that implies the odds of passing the preliminary *or* the final by chance alone can be 1/1000 and odds of passing the challenge *as a whole* is 1/1000,000.

There's some ambiguity in the statement 'passing the final challenge by chance is 1:1000000'. It can either mean 'getting to the end of the process having endured the preliminary and the final', or mean 'passing the final in its own right'. My understanding is that it is the former that is meant.

[Gr8wight]

The more I hear about this apparent uncertainty, the more it leaves a rather bad taste in my mouth. If I personally wanted to submit myself for testing, I'd sure as heck want to know what was expected of me right up front, before going through even the early rounds of paperwork, negotiations, etc.

Bryan

You could easily e-mail Mr Randi and ask him.

Is applying for the challenge something you had contemplated doing?

[Hellbound]

Good god (pardon the expression)! One person's "understanding" differs from another person's "understanding" by three orders of magnitude! :-)

I find it ironic that such uncertainty and lack of precision rears its ugly head on a site which ostensibly goes out of its way to be perfectly clear about the rules of the game in the Million Dollar Challenge. Once again, why in the blue blazes doesn't Randi simply state what the expected odds are going to be, in any test of a statistical nature? It's not like it's necessarily carved into stone tablets or anything...he could include a disclaimer to the effect that the odds _may_ be adjusted in certain cases, but they normally will be set to 1-in-a-million (or 1-in-a-billion, or whatever).

The more I hear about this apparent uncertainty, the more it leaves a rather bad taste in my mouth. If I personally wanted to submit myself for testing, I'd sure as heck want to know what was expected of me right up front, before going through even the early rounds of paperwork, negotiations, etc.

Bryan

Why are you even bothering to post? It's apparent you've made up your mind to dismiss the challenge anyway, so why are you here?

Three orders of magnitude, yes. It's a misunderstanding of a single aspect. I knew 1 in 1000 and 1 in 1000000; it was my mistake in assuming 1 in 1000000 was a cumulative rather than independent probability. A mistake akin to "forgetting to carry the one", and due to my ignorance. As I stated multiple times, I am not an authority on the challenege, and was going from memory. The fact that I am wrong is no reflection on the challenge, rather on myself.

Frankly, NO ONE in the forum is involved, in any official way, with the challenge itself. If you actually wanted answers to these questions you should have emailed randi himself, or the challenge email listed on the main page.

Your assertion that my poor understandign somehow means the challenge is dodgy is, frankly, ridiculous and smacks of your underlying bias. It's the same as declaring a doctor as incompetent because one of his patients mis-remembers the dosage on his last prescription.

I'm calling troll (not that this will suprise many).

[Bryan Shelton]

Why are you even bothering to post? It's apparent you've made up your mind to dismiss the challenge anyway, so why are you here?

DISMISSING a challenge? Who said anything about even ACCEPTING a challenge? Not me.

Three orders of magnitude, yes. It's a misunderstanding of a single aspect. I knew 1 in 1000 and 1 in 1000000; it was my mistake in assuming 1 in 1000000 was a cumulative rather than independent probability. A mistake akin to "forgetting to carry the one", and due to my ignorance. As I stated multiple times, I am not an authority on the challenege, and was going from memory. The fact that I am wrong is no reflection on the challenge, rather on myself.

Why do you assume that YOU are the one in error? Maybe you are correct, and "drkitten" is the one in error. The truth of this matter hasn't beeen determined, has it?

Frankly, NO ONE in the forum is involved, in any official way, with the challenge itself. If you actually wanted answers to these questions you should have emailed randi himself, or the challenge email listed on the main page.

I think I'll do that. But at the very least, I think everybody now sees that there's a lack of information in the stated rules.

Your assertion that my poor understandign somehow means the challenge is dodgy is, frankly, ridiculous and smacks of your underlying bias.

You're showing a bit of paranoia, friend. Nobody accused YOU specifically of being wrong.

I'm calling troll (not that this will suprise many).

There: do ya feel better? :-)

Bryan

[orpheus]

Not that this will make any difference to this discussion (which I've just now stumbled upon), but isn't it something of a moot point? I mean, in view of the fact that 1) the challenge is Randi's; he decides how it's structured and is under no obligation to specify odds up-front, and 2) for each applicant, the specific rules/protocol and what constitutes a win - including odds - if appropriate to the type of claim - must be agreed on by both parties.

[William Smith]

...
The more I hear about this apparent uncertainty, the more it leaves a rather bad taste in my mouth. If I personally wanted to submit myself for testing, I'd sure as heck want to know what was expected of me right up front, before going through even the early rounds of paperwork, negotiations, etc.

Bryan

As Gr8wight already said:

Bryan, If you state a claim and how you would like to prove it, a simple inquiry to challenge@randi.org will most likely dispart any clouds of uncertainty concerning the odds. Quick and efficent. No mouthwash needed.

[Bryan Shelton]

I understand what you're saying, but at the risk of beating a dead horse, isn't it a little odd that the standard (or typical) odds aren't already common knowledge? What's the Big Secret?

Bryan

[ObscureReferenceMan]

I've been looking around trying to find out what the required odds are for that Million Dollar Challenge.

I find it very odd that they wouldn't explicitly state something as important as that right up front for everyone to see.

But at the very least, I think everybody now sees that there's a lack of information in the stated rules.

Bryan,

I'm trying to understand what you're saying. You seem to think the JREF should post the "required odds" for a claim. Why? What is the point? It might be helpful for someone unfamiliar with the challenge coming to see it in "mid run". And if the claim were "statistically based" (i.e. predicting coin flips). Then this visitor would be told something like, "Psychic Fred's chances of succeeding by chance alone are X."

Also, the challenge rules are for the claimant; to guide him/her to putting together a proper statement of abilities, and move toward preliminary testing. And, as others have said, this is a challenge. It's for the claimant to show of his/her powers. Not for JREF to figure what the chances are of success.

Finally, since each claim is unique, having odds posted doesn't seem to make sense. And again, as others have pointed out, how does one determine the odds of levitation, or telekenisis?

Sorry if I am not understanding you. You seem to feel very strongly about this. I'm just trying to understand why.

[William Smith]

I understand what you're saying, but at the risk of beating a dead horse, isn't it a little odd that the standard (or typical) odds aren't already common knowledge? What's the Big Secret?

Bryan

Bryan, since there are no "standard (or typical)" paranormal claims, odds of claims cannot be "common knowledge".

For claims which pop up from time like "telepathy" or "mind reading" the odds used in a JREF protocol (see the Achau Nguyen thread http://forums.randi.org/showthread.php?t=28936) seem similar.

The very different nature of claims make it impossible to give standard odds up front. How for example, would you calculate the odds of "bending a key with your mind"? What would the standard odds be for UFO summoning?

The upcoming Carina Landin test http://forums.randi.org/showthread.php?t=63753 and http://forums.randi.org/showthread.php?t=32779 and http://forums.randi.org/showthread.php?t=58612 and http://forums.randi.org/showthread.php?t=47223 allows for a precise calculation of odds. Perhaps you find it interesting enough to follow.

[Hellbound]

I understand what you're saying, but at the risk of beating a dead horse, isn't it a little odd that the standard (or typical) odds aren't already common knowledge? What's the Big Secret?

Bryan

Why is it odd?

The only time these odds would even matter at all is if you planned on winning the challenge through chance.

If you actually have an ability, the chance odds never come into play.

What you're doing here would be like somone challenging you to throw a dart at a dartboard, and you complaining that something seems "fishy" because he doesn't tell you the odds of hitting the board if you were blindfolded, spun in place, and throw in whatever direction you end up facing.

THe chance odds are, really, inconsequential as long as they are low enough to rule out chance as a likely reason for a "win". Whether 1 in a million or 1 in a billion, I'm comfortable with either.

Agian, you're either ignorant or deceptive...and I am still fairly certain you aren't debating honestly. However, there are lurkers that might believe your argument has some bearing, so it needs to be responded to.

[petre]

I understand what you're saying, but at the risk of beating a dead horse, isn't it a little odd that the standard (or typical) odds aren't already common knowledge? What's the Big Secret?

Bryan

Would you find it odd if a Casino didn't post the odds for winning a game? Would their failure to do so make you feel they're witholding a Big Secret?

[Bryan Shelton]

Bryan,

I'm trying to understand what you're saying. You seem to think the JREF should post the "required odds" for a claim. Why? What is the point?

For maximum transparency, of course. Again I ask, what is the big secret? What is the point of obscuring information? Why not just be right up-front about it, from square-one? Does the JREF have anything to hide?

It might be helpful for someone unfamiliar with the challenge coming to see it in "mid run". And if the claim were "statistically based" (i.e. predicting coin flips). Then this visitor would be told something like, "Psychic Fred's chances of succeeding by chance alone are X."

Yes.

Also, the challenge rules are for the claimant; to guide him/her to putting together a proper statement of abilities, and move toward preliminary testing. And, as others have said, this is a challenge. It's for the claimant to show of his/her powers. Not for JREF to figure what the chances are of success.

HUH??

But OF COURSE it's for JREF to figure what the chances are of success!! If I told them that if they thought of a number between 1 and 10 and I could read their mind and tell them what that number was, do you think they'd let me get away with demonstrating it only one single time, and then making a million bucks if I get it right?? Of course not. They'd think about the challenge for about a nanosecond, and then make me do it successfully for several times in a row (probably at least 6-10 times)! :-)

You were obviously just talking off the top of your head when you said that it's not for JREF to figure the chances of success.

Finally, since each claim is unique, having odds posted doesn't seem to make sense. And again, as others have pointed out, how does one determine the odds of levitation, or telekenisis?

One DOESN'T determine the odds of levitation or telekinesis. As I've indicated before, I'm talking only about challenges which have an obvious statistical element involved.

Sorry if I am not understanding you. You seem to feel very strongly about this. I'm just trying to understand why.

I hope this clarifies things for you.

Bryan

[Bryan Shelton]

Bryan, since there are no "standard (or typical)" paranormal claims, odds of claims cannot be "common knowledge".

For claims which pop up from time like "telepathy" or "mind reading" the odds used in a JREF protocol seem similar.

The very different nature of claims make it impossible to give standard odds up front. How for example, would you calculate the odds of "bending a key with your mind"? What would the standard odds be for UFO summoning?

I addressed that in my previous post. I was referring only to challenges that involve an obvious statistical element, of course. For those, the odds can certainly be calculated. At the very least, in principle.

The upcoming Carina Landin test allows for a precise calculation of odds. Perhaps you find it interesting enough to follow.

Thanks for citing that for me! I'll definitely follow that.

Bryan

[Bryan Shelton]

Why is it odd?

The only time these odds would even matter at all is if you planned on winning the challenge through chance.

Yes. Or another possibility: you are sincere and well-meaning, but you don't really have the ability that you THINK you do.

If you actually have an ability, the chance odds never come into play.

What you're doing here would be like somone challenging you to throw a dart at a dartboard, and you complaining that something seems "fishy" because he doesn't tell you the odds of hitting the board if you were blindfolded, spun in place, and throw in whatever direction you end up facing.

I've already given you obvious examples where chance odds sure as hell DO come into play.

THe chance odds are, really, inconsequential as long as they are low enough to rule out chance as a likely reason for a "win". Whether 1 in a million or 1 in a billion, I'm comfortable with either.

Yes. And I'm curious to know exactly how stringent the JREF is when setting the requirements necessary to ensure that they don't lose their million bucks through chance alone. The fact that they seem rather cagey about that leaves a bad taste in my mouth, as I said before.

Agian, you're either ignorant or deceptive...and I am still fairly certain you aren't debating honestly.

Thanks for sharing that with me, but that's the very least of my concerns.

Bryan

[William Smith]

For maximum transparency, of course. Again I ask, what is the big secret? What is the point of obscuring information? Why not just be right up-front about it, from square-one? Does the JREF have anything to hide?
...


And again the answer is: There is no "big secret". I also do not see where the JREF "obscures information".

At the risk of sounding redundant, hardly any Challenge resembles the other. As JREF hence treats claims on a case-by-case basis, right from square one, the need for odds up-front is irrelevant.
As soon as there is a clearly stated claim, the odds get determined in the protocol. I have not yet witnessed the JREF hiding anything concerning the odds of a successful Challenge.

Bryan, may I suggest you read as many of the past Challenge Applications as possible? http://forums.randi.org/forumdisplay.php?f=43

Not only will it enlighten you as to the odds of claims of a statistical nature (e.g. Hans Peter Borer's claim + test http://forums.randi.org/showthread.php?t=29864 ), it will also show you how hard it seems for applicants to even state the claim precisely.

[Bryan Shelton]

And again the answer is: There is no "big secret". I also do not see where the JREF "obscures information".

It's obscure enough that at least two established posters here disagreed completely on what the odds are in these tests. One thought that there was a cumulative 1/1,000,000 chance through luck alone, whereas the other thought that it was 1/1,000,000,000. They both used the rather vague expression "my understanding is...", which sounds to me like people here aren't very clear about how the probabilities are adjusted in these tests that involve a statistical element. It sounds to me like the JREF doesn't take any great pains to make the odds clear and up-front.

At the risk of sounding redundant, hardly any Challenge resembles the other. As JREF hence treats claims on a case-by-case basis, right from square one, the need for odds up-front is irrelevant.

Why is it irrelevant? I don't see how that follows logically. Why aren't the odds of passing any test standardized, regardless of how disparate the claims themselves happen to be? For example, why wouldn't the required odds for success from sheer guesswork alone be adjusted to be 1-in-a-million (or whatever number JREF finds acceptable), regardless of whether I claim to be able to predict the results of coin-flipping trials, or whether I claim to be able to tell the sex of the author of a diary that I hold in my hands? Do you see what I'm saying? It's a matter of standardization.

As soon as there is a clearly stated claim, the odds get determined in the protocol.

Why on earth would the odds get determined in the protocol?? It's obvious that the protocol itself is going to change from one claim to another; however, I don't see why the actual ODDS would change (be re-adjusted) from one test to another, except for a special case which has been mentioned before (like if Randi wanted to relax the odds a bit just to entice a high-profile "psychic" like Sylvia to take the test).

Bryan, may I suggest you read as many of the past Challenge Applications as possible? http://forums.randi.org/forumdisplay.php?f=43

I read through the first page of applications, but didn't see anything of relevance to what we've been discussing. In only a couple of the threads did I see anything much more than just a casual discussion of the odds, and those were from the applicants themselves. I believe there was only one very brief and insignificant reference to some "acceptable" odds from someone associated with JREF.

Not only will it enlighten you as to the odds of claims of a statistical nature (e.g. Hans Peter Borer's claim + test http://forums.randi.org/showthread.php?t=29864 ), it will also show you how hard it seems for applicants to even state the claim precisely.

There's a brief section on the required odds for both the first and second parts of Borer's testing, but true-to-form, it's ambiguous and uncertain whether those are standardized requirements for all applicants, or just something negotiated between Borer and JREF.

Bryan

[ObscureReferenceMan]

I think I see your point, Bryan. But I suppose I just don't see it as urgently needed as you do.

Also, that's not what I'd call "transparency". It's more like "additional info". My take on it is, yes the JREF doesn't take "great pains" to make the odds clear. So what? It would be nice, but if not, I don't see how it matters.

[Cuddles]

It's obscure enough that at least two established posters here disagreed completely on what the odds are in these tests. One thought that there was a cumulative 1/1,000,000 chance through luck alone, whereas the other thought that it was 1/1,000,000,000. They both used the rather vague expression "my understanding is...", which sounds to me like people here aren't very clear about how the probabilities are adjusted in these tests that involve a statistical element. It sounds to me like the JREF doesn't take any great pains to make the odds clear and up-front.[quote]

And they don't take any pains to hide it either, they just don't say. Have you tried actually asking anyone in the JREF?

[quote]Why is it irrelevant? I don't see how that follows logically. Why aren't the odds of passing any test standardized, regardless of how disparate the claims themselves happen to be? For example, why wouldn't the required odds for success from sheer guesswork alone be adjusted to be 1-in-a-million (or whatever number JREF finds acceptable), regardless of whether I claim to be able to predict the results of coin-flipping trials, or whether I claim to be able to tell the sex of the author of a diary that I hold in my hands? Do you see what I'm saying? It's a matter of standardization.

Why should it be standard? As long as the odds are low it doesn't matter if the odds are different each time. It's Randi's challenge and he can do what he likes with it.

Why on earth would the odds get determined in the protocol?? It's obvious that the protocol itself is going to change from one claim to another; however, I don't see why the actual ODDS would change (be re-adjusted) from one test to another, except for a special case which has been mentioned before (like if Randi wanted to relax the odds a bit just to entice a high-profile "psychic" like Sylvia to take the test).

I claim I can predict a coin toss with 60% accuracy, someone else claims they can dowse for water with 80% accuracy. To test the dowsing claim would require maybe 20 buckets to be checked and could be done in a couple of hours. The coin tossing one could be tested with maybe 100 tosses. The odds of success by chance are completely different, but there is no point attempting to make them equal since it is easier for all concerned just to make up a test that give low odds.

I read through the first page of applications, but didn't see anything of relevance to what we've been discussing. In only a couple of the threads did I see anything much more than just a casual discussion of the odds, and those were from the applicants themselves. I believe there was only one very brief and insignificant reference to some "acceptable" odds from someone associated with JREF.

Exactly. Even with people who are actually trying to apply the issue of the odds does not come up much. Why do you amke such an issue of it when clearly the JREF and the applicants really don't care, as long as the odds of winning by chance are low enough to be ignored.

There's a brief section on the required odds for both the first and second parts of Borer's testing, but true-to-form, it's ambiguous and uncertain whether those are standardized requirements for all applicants, or just something negotiated between Borer and JREF.

Bryan

It's not uncertain at all. The test was conducted in Switzerland by someone trusted by the JREF to run a proper test. It never says specifically, but is very clearly implied, that Kramer didn't even know what the test consisted of, let alone what the odds were, until the test was complete and he was given the results.

[William Smith]

Bryan Shelton, I suggest you contact JREF directly, e.g.: challenge@randi.org

Only JREF Officials can give a definitive answer which will satisfy you.

I do not see what you see. Your view seems to overly simplify. I'm afraid I can be of no more assistance to you in this matter.

[petre]

The odds of winning the challenge by chance aren't clear and upfront. Why? It's isn't a lottery, it's a challenge. It's not intended that a challenger wins by chance. Informally, the odds in the past have tended to 1/1000 by chance. Perhaps if you were to apply, they would require performace of 1/1,000,000,000,000 by chance, that's their business. The only people that such a statistic is really important too are the people that intend to win by chance, and I doubt JREF really cares what such people think.

[Bryan Shelton]

I think I see your point, Bryan. But I suppose I just don't see it as urgently needed as you do.

Oh, it's not really that I see it as being all that "urgent". In the grand scheme of things, it's obviously not something to lose any sleep over!;)

I guess it's mainly just that I find it a little incongruous that despite the fact that they take such great care to spell out all the other rules of the challenge (sometimes in exquisite detail), they leave completely unspecified the odds required to pass the test(s), until they are hashed-out in an arduous process that can take many months.

And then combine that with the criticisms that come from the JREF people when an applicant overlooks one of the rules himself, or supposedly doesn't tow-the-line precisely enough on one of them for their taste, and that leaves a situation which in my opinion has a slight odor about it.

Bryan

[Bryan Shelton]

"There's a brief section on the required odds for both the first and second parts of Borer's testing, but true-to-form, it's ambiguous and uncertain whether those are standardized requirements for all applicants, or just something negotiated between Borer and JREF." --- Bryan

It's not uncertain at all. The test was conducted in Switzerland by someone trusted by the JREF to run a proper test. It never says specifically, but is very clearly implied, that Kramer didn't even know what the test consisted of, let alone what the odds were, until the test was complete and he was given the results.

So you're claiming that the odds WEREN'T standardized? The guy did say something about how he gave Borer "the usual test", if I remember the wording correctly. That seems ambiguous to me. That could conceivably also imply some "usual odds".

Bryan

[Rasmus]

Oh, it's not really that I see it as being all that "urgent". In the grand scheme of things, it's obviously not something to lose any sleep over!;)

I guess it's mainly just that I find it a little incongruous that despite the fact that they take such great care to spell out all the other rules of the challenge (sometimes in exquisite detail), they leave completely unspecified the odds required to pass the test(s), until they are hashed-out in an arduous process that can take many months.

Why is it incongruous? It's not like it would make a flaming difference to anyone who can really do what they claim they can do. And that much is clear from the rules and the entire setup of the challenge, too.

It is simple to win the challenge if you can do somethoing paranormal. You can predict coin flips? Great! What does it matter to you, if you have to do 5, 10 or even 100 predictions?

Also, it's not like the process of designing a protocol would take many months because the odds have to be determined. It takes many months because other things have to be taken care of.

And then combine that with the criticisms that come from the JREF people when an applicant overlooks one of the rules himself, or supposedly doesn't tow-the-line precisely enough on one of them for their taste, and that leaves a situation which in my opinion has a slight odor about it.

Bryan

Then why add further rules?

[Bryan Shelton]

Why is it incongruous? It's not like it would make a flaming difference to anyone who can really do what they claim they can do. And that much is clear from the rules and the entire setup of the challenge, too.

It is simple to win the challenge if you can do somethoing paranormal. You can predict coin flips? Great! What does it matter to you, if you have to do 5, 10 or even 100 predictions?

OH REALLY?? What if my paranormal ability were to be able to correctly predict the outcome of coin flips with an accuracy of 50.1%? Would THAT be provable with 5, 10, or even 100 trials?:D

There are obvious examples like that where the exact odds required by the JREF could make a HUGE difference in the amount of work required to meet a challenge.

Bryan

[petre]

OH REALLY?? What if my paranormal ability were to be able to correctly predict the outcome of coin flips with an accuracy of 50.1%? Would THAT be provable with 5, 10, or even 100 trials?:D

There are obvious examples like that where the exact odds required by the JREF could make a HUGE difference in the amount of work required to meet a challenge.

Bryan

Absolutely. Should you chance to run into such an applicant, or happen to be one yourself, I advise you to contact JREF staff to discuss the matter. Thank you for your concern.

[William Smith]

challenge@randi.org

[Cuddles]

OH REALLY?? What if my paranormal ability were to be able to correctly predict the outcome of coin flips with an accuracy of 50.1%? Would THAT be provable with 5, 10, or even 100 trials?:D

There are obvious examples like that where the exact odds required by the JREF could make a HUGE difference in the amount of work required to meet a challenge.

Bryan

And how many people are likely to have tossed enough coins to be able to say they can do that? People can only claim on what they believe themselves able to do. Someone could easily claim to do this, but unless they can offer proof that they have actually done a trial large enough to detect it they will simply be ignored.

In addition, who cares? If someone is willing to spend their time tossing millions of coins, why would they care if they have to do a few million more? Since they would agree the number beforehand anyway it's not like they'd waste their time or the JREF's time.
Edit : At least not any more than the usual applicants.;)