The Next Level

[Monster Machine]

I was thinking about this last night. What test is involved in phase 2 if the applicant passes phase 1?

Let's use this as an example:

I have a levitation claim. I claim I can float at least 2 feet off the ground for longer than 30 minutes. A Protocol is developed to test this claim. All parties agree. The preliminary test is carried out and I pass (i.e. I float in the air for 30 minutes as stipulated by the protocol) All parties agree I have passed.

What would the second phase entail in this example?

Monster

PS - Just for the record - I cannot levitate.

[IMST]

The JREF examines the results and looks for ways you may have cheated. They upgrade the controls to prevent this and you do the same test again.

[drkitten]

I was thinking about this last night. What test is involved in phase 2 if the applicant passes phase 1?

Let's use this as an example:

I have a levitation claim. I claim I can float at least 2 feet off the ground for longer than 30 minutes. A Protocol is developed to test this claim. All parties agree. The preliminary test is carried out and I pass (i.e. I float in the air for 30 minutes as stipulated by the protocol) All parties agree I have passed.

What would the second phase entail in this example?

Monster

PS - Just for the record - I cannot levitate.

This is one type of claim where the second phase is almost not needed. One of the major reasons for the two-phase design is that the statistics can be tighter on the second phase. While you might need to predict five cards in a row in the first phase, you'd need to predict ten or fifteen on the second phase. Where you'd need to find gold eight out of ten times on the first phase, you'd need to find it 14 out of twenty.

With a non-statistical claim like levitation, they'd basically just be setting up more elaborate equipment to make sure you weren't using wires.

[shadron]

I was thinking about this last night. What test is involved in phase 2 if the applicant passes phase 1?

Let's use this as an example:

I have a levitation claim. I claim I can float at least 2 feet off the ground for longer than 30 minutes. A Protocol is developed to test this claim. All parties agree. The preliminary test is carried out and I pass (i.e. I float in the air for 30 minutes as stipulated by the protocol) All parties agree I have passed.

What would the second phase entail in this example?

Monster


PS - Just for the record - I cannot levitate.

That would not, in all likelihood, how the protocol would be carried out. It would probably be more like: There are five 12' high columns in the room, each with a different single digit, 1-5, written on a notecard at the top of each column, face up. Without touching the columns, get all five of the numbers assigned correctly to each column.

The protocol for the final test would be similar, but not designed until the first test was completed, and could involve ten similar test done on consecutive days or some such; it would, as stated above, make a full, unabashed attempt to eliminate knowledge that did not come from actually looking down at the top of the columns from above. The test is whether or not he can float as claimed, not whether he can devise a method of cheating in any or all cases - that has been well established before, at least for many cases.

[drkitten]

That would not, in all likelihood, how the protocol would be carried out. It would probably be more like: There are five 12' high columns in the room, each with a different single digit, 1-5, written on a notecard at the top of each column, face up. Without touching the columns, get all five of the numbers assigned correctly to each column.

In my experience, neither Randi nor his agents make wholesale emendations to the proposed protocols in this way. If you claim that you can levitate, then you would be tested on your levitation ability. It's not fair to be tested on your ability to identify digits while levitating (as your protocol demands), and it appears to add needless complexity.

Randi would simply ask for a method by which you could be confirmed to be levitating. It could be as simple as waving someone's hand underneath you or holding a measuring rod -- once you had been demonstrated not to be attached to strings or concealed supports or such-like.

[Monster Machine]

In my experience, neither Randi nor his agents make wholesale emendations to the proposed protocols in this way. If you claim that you can levitate, then you would be tested on your levitation ability. It's not fair to be tested on your ability to identify digits while levitating (as your protocol demands), and it appears to add needless complexity.

Randi would simply ask for a method by which you could be confirmed to be levitating. It could be as simple as waving someone's hand underneath you or holding a measuring rod -- once you had been demonstrated not to be attached to strings or concealed supports or such-like.

So if that's the preliminary, what would the second test be?

I guess I am trying to determine how far things would go depending on the claim being made. For sure, anything that has a statistical chance of passing would be tested twice. But for things like levitation, I think only one test would be in order, no?

Monster

[IMST]

As I said, I believe they would look for cheating, try to improve controls and do exactly the same test again. Two tests for everyone.

[RemieV]

drkitten is correct. Identifying numbers placed on top of columns is not mentioned anywhere in the claim itself. Why would that be part of the protocol?

Take Rosemary Hunter, for instance. Her claim was the ability to make people urinate themselves with the power of her mind. Her test was an attempt to demonstrate the ability.

Anyone interested in this subject who is attending TAM should consider signing up for the Million Dollar Challenge Workshop with Chip Denman and myself. The workshop will take you through a hands-on demonstration of statistical significance, protocol development, and all the way up to running a mock-test in the workshop classroom.

~Remie

[rjh01]

From the FAQ (http://www.randi.org/site/index.php/component/content/article/37-static/254-jref-challenge-faq.html)

5.2 What happens between the preliminary test and the official test?

The protocol itself will not be changed, and neither will any of the documents you and the JREF have agreed upon. The final test may be longer, or require more conclusive results through more sets of the test to ensure that the preliminary test was not a fluke.

[Cuddles]

But for things like levitation, I think only one test would be in order, no?

No. Why would you think that?

[drkitten]

So if that's the preliminary, what would the second test be?

The same as the first test, but with more rigorous testing to make sure that you weren't on wires like a stage Peter-Pan.


I guess I am trying to determine how far things would go depending on the claim being made. For sure, anything that has a statistical chance of passing would be tested twice. But for things like levitation, I think only one test would be in order, no?

No. The JREF is very clear on this.

One major advantage of this approach is that it lets the JREF farm out most of the work while still reaping the glory. Remember that preliminary tests are not usually conducted at the JREF, and that Randi himself is not present and takes no part in the testing. This would not be true for the final test; it's conducted under the personal supervision of Randi himself.

If nothing else, this is good for the JREF in that they get the photo-op when the person ultimately passes. (hah!) Randi's an old showman and understands the value of publicity. It also allows Randi to guard against the incompetence of the South Hudson Institute of Technology Skeptics' Club; if they missed some obvious way to cheat, he has a chance to review the footage and what-not and figure out if there are any new countermeasures he needs to put in.

[RemieV]

The same as the first test, but with more rigorous testing to make sure that you weren't on wires like a stage Peter-Pan.



No. The JREF is very clear on this.

One major advantage of this approach is that it lets the JREF farm out most of the work while still reaping the glory. Remember that preliminary tests are not usually conducted at the JREF, and that Randi himself is not present and takes no part in the testing. This would not be true for the final test; it's conducted under the personal supervision of Randi himself.

If nothing else, this is good for the JREF in that they get the photo-op when the person ultimately passes. (hah!) Randi's an old showman and understands the value of publicity. It also allows Randi to guard against the incompetence of the South Hudson Institute of Technology Skeptics' Club; if they missed some obvious way to cheat, he has a chance to review the footage and what-not and figure out if there are any new countermeasures he needs to put in.

That, however, isn't precisely true. The JREF doesn't 'farm out' the work in order to reap the glory, but because most applicants aren't located in Ft. Lauderdale, FL, and have no means of getting there.

I have never seen anything that said the final test would be under Randi's supervision. I guess it might be - I'm sure he would be interested. However, if the claimant said that they were unable to perform with him present, he certainly would not be present.

[Lanzy]

In the second test they weigh you down with 400 lbs of bricks and see if you can really levitate.:D

See?, No one passes the second test.

[jojonete]

It also allows Randi to guard against the incompetence of the South Hudson Institute of Technology Skeptics' Club; if they missed some obvious way to cheat, he has a chance to review the footage and what-not and figure out if there are any new countermeasures he needs to put in.
(bolding mine)

No way. The final test has to follow the protocol, no "new countermeasures" can be added in the middle.
Of course, the final test may pay more attention to any countermeasures mentioned in the protocol (thay might have been overlooked in the preliminary test), but those wouldn't be new countermeasures as they were in the protocol all the time.

As rjh01 quoted (http://forums.randi.org/showthread.php?p=4819326) from the FAQ (http://www.randi.org/site/index.php/component/content/article/37-static/254-jref-challenge-faq.html):5.2 What happens between the preliminary test and the official test?

The protocol itself will not be changed, and neither will any of the documents you and the JREF have agreed upon. The final test may be longer, or require more conclusive results through more sets of the test to ensure that the preliminary test was not a fluke.


In the example given in the OP, the final test would consist in checking if the applicant can levitate again for another 30 minutes, in the very same conditions used for the preliminary test. I'd expect all JREF observers to pay much more attention to all details involved, but the test would be exactly the same, with exactly the same restrictions.

The agreed-upon protocol, once signed by Randi, is a legally binding contract. The JREF can't add any new countermeasures for the final test (like 400 lbs. of bricks ;) ). The applicant would surely reject any such countermeasures, and they're legally entitled to do so.

[RemieV]

jojonete,

All protocols created say "No sleight-of-hand, trickery, or cheating". We can certainly devise new ways to check for sleight-of-hand, trickery, and cheating.

-- Remie.

[drkitten]

No way. The final test has to follow the protocol, no "new countermeasures" can be added in the middle.

Sure it can. Use a high-speed video camera instead of a normal camera, for example, to capture any potential sleight of hand tricks, or add additional cameras to capture potential cheating from new angles.

Photograph using IR to detect heat sources.

Put a layer of dust down on the floor to make air currents (as from a mechanically-generated air cushion) visible.

et cetera, et cetera.

There are any number of additional countermeasures that could be put in without violating the protocols.

[CynicalSkeptic]

That would not, in all likelihood, how the protocol would be carried out. It would probably be more like: There are five 12' high columns in the room, each with a different single digit, 1-5, written on a notecard at the top of each column, face up. Without touching the columns, get all five of the numbers assigned correctly to each column.

That's the start of a decent protocol for Astral Projection, not Levitation.

[jojonete]

All protocols created say "No sleight-of-hand, trickery, or cheating". We can certainly devise new ways to check for sleight-of-hand, trickery, and cheating.
Sure it can. Use a high-speed video camera instead of a normal camera, for example, to capture any potential sleight of hand tricks, or add additional cameras to capture potential cheating from new angles.

Photograph using IR to detect heat sources.

Put a layer of dust down on the floor to make air currents (as from a mechanically-generated air cushion) visible.

et cetera, et cetera.

There are any number of additional countermeasures that could be put in without violating the protocols.
Yes, protocols say "no cheating". I don't think this should give JREF the right to add new anti-cheating measures not explicitly mentioned in the protocol.

If the JREF can add dust on the floor (and the claimant isn't entitled to complain it inhibits his/her levitating powers), can the JREF also add 400 lbs. of bricks (http://forums.randi.org/showthread.php?p=4821530) (thanks Lanzy for the example) and say it's an anti-cheating measure? Who draws the line that separates anti-cheating measures from cheating on JREF's part?

[RemieV]

Yes, protocols say "no cheating". I don't think this should give JREF the right to add new anti-cheating measures not explicitly mentioned in the protocol.

If the JREF can add dust on the floor (and the claimant isn't entitled to complain it inhibits his/her levitating powers), can the JREF also add 400 lbs. of bricks (http://forums.randi.org/showthread.php?p=4821530) (thanks Lanzy for the example) and say it's an anti-cheating measure? Who draws the line that separates anti-cheating measures from cheating on JREF's part?

Um... the claimant. Protocols are mutually agreed upon.

[Pup]

Um... the claimant. Protocols are mutually agreed upon.

Are protocol negotiations frozen before the preliminary challenge, or can they start up again before the final test? If they can start up again, the JREF could simply demand the 400 pounds of bricks after the first test but before the final test or else the testing can't proceed, and we're right back to the same question.

Yes, I realize that the JREF could make that demand before the preliminary and the applicant could just say, sorry, won't work, bye. But here's the reason it's more suspicious after the first test.

Here's the potential woo spin:

Before the first test, the JREF figures they have all avenues of cheating covered. Applicant passes the test. After seeing the results, the JREF realizes the applicant can actually do something paranormal, and in a desparate attempt to prevent paying the million, demands a new condition that they never would have demanded otherwise, for the sole purpose of making the performance impossible.

The contestant can walk away, but it's not as "clean" as if the protocol were negotiated in full and locked before the first test.

[MattC]

Yes, protocols say "no cheating". I don't think this should give JREF the right to add new anti-cheating measures not explicitly mentioned in the protocol.

If the JREF can add dust on the floor (and the claimant isn't entitled to complain it inhibits his/her levitating powers), can the JREF also add 400 lbs. of bricks (http://forums.randi.org/showthread.php?p=4821530) (thanks Lanzy for the example) and say it's an anti-cheating measure? Who draws the line that separates anti-cheating measures from cheating on JREF's part?

There's a sharp difference between throwing sand (a comparatively more realistic example than dust) onto the floor beneath someone who claims to be levitating and putting several hundred pounds of bricks onto them. One should ideally not affect the levitation process at all (sand), whereas one completely defeats the purpose (bricks). Instead of asking the claimant to simply levitate, you're now asking them to lift four hundred pounds plus their own body weight - a complete change in protocol most certainly not agreeable to either party.

Sand, on the other hand, if placed below the person levitating (I'm not entirely sure why you'd want to do this at the second test if not the first, it seems like a very natural precaution against cheating via manipulation of air currents) should have no effect whatsoever on the levitation process if it is indeed paranormal. This method of detecting cheating is nonintrusive to the claimed ability and therefore legitimate.


Here's the potential woo spin.

Before the first test, the JREF figures they have all avenues of cheating covered. Applicant passes the test. After seeing the results, the JREF realizes the applicant can actually do something paranormal, and in a desparate attempt to prevent paying the million, demands a new condition that they never would have demanded otherwise, for the sole purpose of making the performance impossible.

The contestant can walk away, but it's not as "clean" as if the protocol were negotiated in full and locked before the first test.

I'm not entirely sure that makes any sense. If you can do something paranormal, then by definition unobtrusive means of detecting normal cheating are (or should not be) relevant. While it's certainly possible to claim that the sand has interfered with your levitation powers, it's possible to claim that <insert idiotic thing here> interfered with your powers also, so I'm not entirely sure such a claim would hold water.

~ Matt

[Uncayimmy]

Look at the protocol itself and think of it as the contract as it is. In Pavel's protocol it says:
No sleight-of-hand, trickery, or cheating.

No manipulation of the envelopes besides as is outlined in the testing protocol.

I think you would be hard pressed to find any judge who would rule that the JREF would be violating the agreement by taking steps to ensure that these two requirements are not being violated by the claimant.

Suppose, for example, that Pavel somehow manages to perform an incredible feat of sleight of hand to swap envelopes hidden on his person (humor me). The JREF can't figure this out until they look at the film in slow motion. For the next test they watermark the envelopes unbeknownst to Pavel. After the first correct reading, they prove that the envelope he "read" was not actually the same envelope he was handed. They call off the test right there.

No court would rule against the JREF because taking steps to detect sleight of hand is implicit to enforcement of the contract. There are probably better ways of detecting it (inspecting his person before the test, for example). If the JREF can show cheating, then they are in the clear.

[jojonete]

Yes, protocols say "no cheating". I don't think this should give JREF the right to add new anti-cheating measures not explicitly mentioned in the protocol.

If the JREF can add dust on the floor (and the claimant isn't entitled to complain it inhibits his/her levitating powers), can the JREF also add 400 lbs. of bricks (http://forums.randi.org/showthread.php?p=4821530) (thanks Lanzy for the example) and say it's an anti-cheating measure? Who draws the line that separates anti-cheating measures from cheating on JREF's part?
Um... the claimant. Protocols are mutually agreed upon.
I don't see how your comment referst to my question. I'll try to put my point in other words:
a) Would it be fair for the JREF to add sand on the floor as a requirement between the preliminary and final test? (I'm assuming YES).
b) Would it be fair for the JREF to add 400 lbs of bricks as a requirement between the preliminary and final test? (I'm assuming NO).

Now, how does "Protocols are mutually agreed upon" refer to the difference between a) and b) ?

I'm not entirely sure that makes any sense. If you can do something paranormal, then by definition unobtrusive means of detecting normal cheating are (or should not be) relevant. While it's certainly possible to claim that the sand has interfered with your levitation powers, it's possible to claim that <insert idiotic thing here> interfered with your powers also, so I'm not entirely sure such a claim would hold water.
You say sand is unobtrusive. I mostly agree.
You say bricks are obtrusive. I certainly agree.
The point is, someone (e.g. the claimant) could disagree with us. Also, it's easy to imagine scenarios where reasonable people would disagree with each other about how obtrusive some anti-cheating measure is. An example: in the preliminary test the applicant looks like he's using shoes with air jets to lift himself up. The JREF asks to examine the shoes after the test, the applicant agrees and the JREF sees nothing unusual in the shoes. However, they suspect the shoes have something too small to be seen, so they just require the applicant to wear shorts and no socks or shoes in the final test.

In this scenario, I think some people would say the clothes of the claimant are unobtrusive and don't affect the levitating powers, while other people would say the claimant is entitled to be as comfortable as possible during the test, so they're entitled to wear anything they like. Both points of view are, at the very least, defendible.

You see the point: different people have different ideas about what's "obtrusive". Whose criterion should we use?

For the next test they watermark the envelopes unbeknownst to Pavel.
I don't think the JREF is entitled to lie (or hide information) to the applicant about the terms of the test. If the watermarks are an important part of the test, they should be mentioned in the protocol.

[Uncayimmy]

I don't think the JREF is entitled to lie (or hide information) to the applicant about the terms of the test. If the watermarks are an important part of the test, they should be mentioned in the protocol.
You missed the point of my post. The JREF is implicitly permitted to take steps to detect cheating. If they detect cheating, then the contract terms have been violated first by the claimant who did the cheating. It's case closed at that point.

Now, suppose the JREF tries to detect cheating and fails to do so. Suppose further that the claimant fails the test. If the claimant can demonstrate that the steps taken by the JREF inhibited the claimant from honoring his end of the contract, that would be solid grounds for a lawsuit.

[drkitten]

I don't see how your comment referst to my question. I'll try to put my point in other words:
a) Would it be fair for the JREF to add sand on the floor as a requirement between the preliminary and final test? (I'm assuming YES).
b) Would it be fair for the JREF to add 400 lbs of bricks as a requirement between the preliminary and final test? (I'm assuming NO).

I agree with both, for what it's worth.


Now, how does "Protocols are mutually agreed upon" refer to the difference between a) and b) ?

Presumably the protocol included the phrase "claimant will levitate himself" or the functional equivalent. Requiring the claimant to levitate himself and an additional object, even a feather, would therefore violate the protocol.

On the other hand, unless the procotol explicitly mentioned the surface over which the claimant would levitate, the JREF can do anything "reasonable" it likes without violating protocol. After all, if the first test were in a gym with a hardwood floor, and the second test were in a carpeted TV studio, that wouldn't violate protocol unless the claimant had specified otherwise.


The point is, someone (e.g. the claimant) could disagree with us. Also, it's easy to imagine scenarios where reasonable people would disagree with each other about how obtrusive some anti-cheating measure is.

Ultimately, that's what the court system would be for, if needed. You've already admitted that sand on the floor is a reasonable and nonobtrusive anti-cheating measure.



An example: in the preliminary test the applicant looks like he's using shoes with air jets to lift himself up. The JREF asks to examine the shoes after the test, the applicant agrees and the JREF sees nothing unusual in the shoes. However, they suspect the shoes have something too small to be seen, so they just require the applicant to wear shorts and no socks or shoes in the final test.

Bad decision on JREF's part. There are a lot less obtrusive ways to get the same effect.

[drkitten]

Now, suppose the JREF tries to detect cheating and fails to do so. Suppose further that the claimant fails the test. If the claimant can demonstrate that the steps taken by the JREF inhibited the claimant from honoring his end of the contract, that would be solid grounds for a lawsuit.

They wouldn't even need to "demonstrate" that (in the strict sense). But they would need a pretty convincing argument and explanation about why they didn't bother to put the necessary condition into the freely-negotiated protocol.

I agree that "I didn't think anyone would be asinine to expect me to levitate while wearing a suit of armor" is a good explanation. I don't think "the sand prevented me from rooting my chi" would be a good explanation. I feel confident that a judge and jury would agree, if needed.

[MattC]

I don't see how your comment referst to my question. I'll try to put my point in other words:
a) Would it be fair for the JREF to add sand on the floor as a requirement between the preliminary and final test? (I'm assuming YES).
b) Would it be fair for the JREF to add 400 lbs of bricks as a requirement between the preliminary and final test? (I'm assuming NO).

Now, how does "Protocols are mutually agreed upon" refer to the difference between a) and b) ?

By specifically agreeing on it beforehand. If a point of contention is or would be significant enough to halt the test outright (so if sand is really that offensive or deal-breaking), I'd like to think that the applicant (who ideally knows the most about their powers and the limitations thereof) would be aware of this and would have written something like it into the protocol statement. Further, I don't think that the JREF would or could agree to such a proposed test, one that effectively removes the possibility of detecting air currents.

You say sand is unobtrusive. I mostly agree.
You say bricks are obtrusive. I certainly agree.
The point is, someone (e.g. the claimant) could disagree with us. Also, it's easy to imagine scenarios where reasonable people would disagree with each other about how obtrusive some anti-cheating measure is. An example: in the preliminary test the applicant looks like he's using shoes with air jets to lift himself up. The JREF asks to examine the shoes after the test, the applicant agrees and the JREF sees nothing unusual in the shoes. However, they suspect the shoes have something too small to be seen, so they just require the applicant to wear shorts and no socks or shoes in the final test.

In this scenario, I think some people would say the clothes of the claimant are unobtrusive and don't affect the levitating powers, while other people would say the claimant is entitled to be as comfortable as possible during the test, so they're entitled to wear anything they like. Both points of view are, at the very least, defendible.

You see the point: different people have different ideas about what's "obtrusive". Whose criterion should we use?

I would point out that the protocol for the test is determined beforehand by agreement from both parties, or in short that there is precise agreement between the JREF and an applicant over what will constitute the "test" and the parameters thereof. Directly worded into the agreement is the passage Remie pointed out - this is a de facto authorization of the JREF by the applicant to use or devise any measures or methods to detect sleight of hand, trickery, or cheating so long as it does not conflict with the agreed-upon claim. It is for this reason that I support the use of sand as an anti-cheating method, and abhor the use of bricks for the same purpose - a claim that I can levitate myself is unaffected, or should be, by the presence of sand beneath me; it is certainly affected by the presence of four hundred pounds of bricks on me because now I have to lift those bricks in addition to myself.

I don't think the JREF is entitled to lie (or hide information) to the applicant about the terms of the test. If the watermarks are an important part of the test, they should be mentioned in the protocol.

"terms of the test" incorporates a good deal more than the protocol would, presuming you mean the phrase literally. Not revealing, for example, my intention to wear a lucky red shirt to the test would hypothetically be included under "hiding information from the applicant," though I'm sure we'd both agree that its effect on a levitation test would be minimal.

Ultimately I would say that information germane to the test should not be hidden from the applicant, and ideally written into the protocol itself - Remie has, so far, done an excellent job of writing these in. My intention, on the other hand, to wear a fedora to the test is probably not worth mentioning (very much like the fedora itself).

~ Matt

[Cuddles]

I don't see how your comment referst to my question. I'll try to put my point in other words:
a) Would it be fair for the JREF to add sand on the floor as a requirement between the preliminary and final test? (I'm assuming YES).
b) Would it be fair for the JREF to add 400 lbs of bricks as a requirement between the preliminary and final test? (I'm assuming NO).

Now, how does "Protocols are mutually agreed upon" refer to the difference between a) and b) ?

The problem is that you're trying to make it look as though two entirely different scenarios are somehow equivalent. Look at the two options you've given here. In one, you specifically say that the new thing will be added to the floor. In the other, you leave that part out, and then jump to assuming it will be used to directly effect the applicant. But in doing so, you've changed things completely.

In fact, adding either sand or bricks would be completely acceptable, as long as they're only added to the surrounding environment in an effort to detect cheating. Neither would be acceptable if used to weight the applicant down and prevent their ability from working. It's not the difference between the sand and bricks that makes the two cases different, it's the difference in what you think will be done with them.

[wackyvorlon]

It appears to me that some people are operating under a misapprehension. My understanding is that the purpose of the challenge is to prove that a person can do what they claim, not disprove it. Shenannigans like adding 400 pounds of bricks and such are not rigorous science. If the JREF pulled stunts like that, they would be no better than the promoters of the paranormal.

When you perform a rigorous, scientific test, you have to eliminate potential other causes for the effect. This is simply to remove error. Consider one age-old favourite of woo, the orgone accumulator. The inventor of this actually sent the device to Albert Einstein, and explained his claim that it was producing heat as a result of this "orgone". At first, it appeared to Einstein that it was working - the temperature at the top of the device was warmer than below. Further testing, however, revealed that there was a natural temperature gradient in the room - the accumulator was doing nothing.

You can see in the example how Einstein and his assistant took steps to rule out sources of error. The goal being to make certain that what they thought was happening, was actually happening. It is the same with the challenge. I, for one, would like nothing more than for some of these claims to be true. If one could levitate using only one's mind, that would be astounding! It could open up a whole new field of research. But, finding such a thing is of no use if it's the result of a lie, or an error. The universe does not conform itself to our wishes, and we must accept reality as imposed upon us by it. To fight against it is folly.

[Pup]

It appears to me that some people are operating under a misapprehension. My understanding is that the purpose of the challenge is to prove that a person can do what they claim, not disprove it. Shenannigans like adding 400 pounds of bricks and such are not rigorous science. If the JREF pulled stunts like that, they would be no better than the promoters of the paranormal.

It's my understanding that the purpose of the challenger is to prove that he or she can do what they claim, while the purpose of the JREF is to set the controls to prevent non-paranormal explanations.

But the JREF is not out to discover paranormal phenomenon. The challenge is, well, a challenge, not an experiment to test a hypothesis, because from the viewpoint of the JREF, there are no paranormal phenomona and thus no hypothesis to test (like "dousing is real"). So it's not really a scientific experiment, though it has similarities to one.

While the 400 pounds of bricks would indeed make the JREF look desperate and silly, I think that's only because it's a blatant example. Suppose, instead, somebody showed up with an X-ray machine in 1800 claiming to see inside solid boxes.

First test of a steel box went fine (really strong X-rays for 1800, bear with me here :)), but the JREF decided that they wanted the final test done through a stronger box so they used lead. Fair, or not? Suppose our inventor hadn't really tested his machine for every substance, but knew it went through steel, wood, paper, leather, etc., but had never tested it on lead and didn't know to specify "no lead" in the negotiations. Too bad, so sad, no prize.

However, in retrospect, we'd see how the apparently minor change from steel to lead was exactly the change that blew the challenger's chance, even though at the time everyone may have thought one metal was as good as another.

I think that would be the kind of thing that's on the borderline of drkitten's examples:


I agree that "I didn't think anyone would be asinine to expect me to levitate while wearing a suit of armor" is a good explanation. I don't think "the sand prevented me from rooting my chi" would be a good explanation. I feel confident that a judge and jury would agree, if needed.


I guess maybe the only lesson is for a challenger to pre-test and think of any reasonable variations in the protocol that the JREF might want, and specify whether they're acceptable before the preliminary.

[wackyvorlon]

First test of a steel box went fine (really strong X-rays for 1800, bear with me here :)), but the JREF decided that they wanted the final test done through a stronger box so they used lead.


But testing it on a lead box isn't the same as testing it on a steel box. They're different boxes, and may present the confounding factors you mention. The protocol would not be well designed if they changed a variable for no reason.

The claim is that a steel box can be seen through, not that it can peer through all materials. Clearly, for the claim that it can peer through all materials, it cannot. Changing the material of the box between the two experiments is bad design, and would invalidate the experiment.

EDIT: Again, I think you are demonstrating a misapprehension. If JREF were acting in bad faith and had no interest in finding real paranormal effects, then it would do as you describe. However, why would the JREF not want to find such a thing? Finding something like that could very easily win the investigator a nobel prize. It's extremely valuable, and I think everybody would be ecstatic to have it happen. As I said, however, it's not worth anything if the effect is fraudulent. It has to be real. Even if it's very small, or seemingly insignificant(the test of paranormal urination comes to mind), it's valuable. A falsehood is of no use to anyone. That is the goal of the design of the test. It is to produce neither false positive nor false negative - but to cleave the marrow of truth from the bone of contention.

[Gr8wight]

But testing it on a lead box isn't the same as testing it on a steel box. They're different boxes, and may present the confounding factors you mention. The protocol would not be well designed if they changed a variable for no reason.

The claim is that a steel box can be seen through, not that it can peer through all materials. Clearly, for the claim that it can peer through all materials, it cannot. Changing the material of the box between the two experiments is bad design, and would invalidate the experiment.

EDIT: Again, I think you are demonstrating a misapprehension. If JREF were acting in bad faith and had no interest in finding real paranormal effects, then it would do as you describe. However, why would the JREF not want to find such a thing? Finding something like that could very easily win the investigator a nobel prize. It's extremely valuable, and I think everybody would be ecstatic to have it happen. As I said, however, it's not worth anything if the effect is fraudulent. It has to be real. Even if it's very small, or seemingly insignificant(the test of paranormal urination comes to mind), it's valuable. A falsehood is of no use to anyone. That is the goal of the design of the test. It is to produce neither false positive nor false negative - but to cleave the marrow of truth from the bone of contention.

Read before replying. The claim in the example given was that the claimant could see through solid boxes. Last I checked, both steel and lead were solids for the purposes of the claim example.

[wackyvorlon]

I did read. And apparently, given the claim of seeing through solid boxes without any further qualification, the claim is false. It can only see through some boxes. This would be spelled out in the test, however. If you read the protocols that are published on this forum, a complete and thorough list of materials is included. If the claimant said he had not tried his device with lead boxes, but only steel, the protocol would specify steel. Otherwise, it would invalidate the experiment.

[Gr8wight]

I did read. And apparently, given the claim of seeing through solid boxes without any further qualification, the claim is false. It can only see through some boxes. This would be spelled out in the test, however. If you read the protocols that are published on this forum, a complete and thorough list of materials is included. If the claimant said he had not tried his device with lead boxes, but only steel, the protocol would specify steel. Otherwise, it would invalidate the experiment.

It seems to me this is Pup's hypothetical situation. What makes you think you have license to dictate the terms of his imaginary claim? In his example, he clearly states that his imaginary claimant has hypothetically claimed to be able to see through hypothetical "solid" objects. Barring further eloboration on the part of Pup's imagination, we cannot really speculate on more hypothetically detailed hypothetical protocol negotiations. Given the hypothetical situation exactly as Pup has described it to us, your previous statement was, hypothetically, wrong.

[Pup]

To clarify, I should say that in my hypothetical example, both the JREF and the X-ray inventor would be innocently unaware of the exact limits of the X-ray's ability. Therefore, it wouldn't be a deliberate attempt by the JREF to prevent a paranormal claim from being proven.

That mimics the claims of a lot of people about their paranormal powers: they work predictably except for unexpectedly when they don't, and the claimant may not be able to explain the exact limits of their powers. Of course, I'd argue that that's because they really don't work and they only seem to because of luck, confirmation bias, etc.

But in this case, both sides would think that a solid box is a solid box, so what's the problem? And yet there really would be a problem.

As I said, I guess the only practical solution is for both sides to consider, test and negotiate all reasonable requirements that their opponents might expect before the first test, and then it's settled, without any "unreasonable" demands (however one might define that) popping up from either side between the preliminary and final test.

[MattC]

To clarify, I should say that in my hypothetical example, both the JREF and the X-ray inventor would be innocently unaware of the exact limits of the X-ray's ability. Therefore, it wouldn't be a deliberate attempt by the JREF to prevent a paranormal claim from being proven.

That mimics the claims of a lot of people about their paranormal powers: they work predictably except for unexpectedly when they don't, and the claimant may not be able to explain the exact limits of their powers. Of course, I'd argue that that's because they really don't work and they only seem to because of luck, confirmation bias, etc.

But in this case, both sides would think that a solid box is a solid box, so what's the problem? And yet there really would be a problem.

As I said, I guess the only practical solution is for both sides to consider, test and negotiate all reasonable requirements that their opponents might expect before the first test, and then it's settled, without any "unreasonable" demands (however one might define that) popping up from either side between the preliminary and final test.

"unreasonable demands" can be defined very simply as any demand by either party contrary to the protocol statement as expressed in the signed & notarized contract between the JREF and the applicant. Being unreasonable, such a demand would not be followed and the decision to not follow it would be legally enforceable.


I did read. And apparently, given the claim of seeing through solid boxes without any further qualification, the claim is false. It can only see through some boxes. This would be spelled out in the test, however. If you read the protocols that are published on this forum, a complete and thorough list of materials is included. If the claimant said he had not tried his device with lead boxes, but only steel, the protocol would specify steel. Otherwise, it would invalidate the experiment.

The descriptiveness of the materials found in the protocol statement is directly correlated with its relevance to the experiment. To give you an example of what I mean by this, in the situation you suggest we would be very careful to specify what type of box we would be using while the type of chair we sat in during this test would not be mentioned. Materials germane to the applicant's stated abilities are elaborated upon in the test, other circumstances are rightfully considered irrelevant. The JREF is under no obligation to know more about the applicant's powers than the applicant does, and in several cases throughout the application where the JREF staffer was unclear about the applicant's abilities numerous followup questions were asked.

~ Matt

[Uncayimmy]

Like what Matt said, it really doesn't matter what the claimant says he can do in his claim. What matters is the protocol. If he says he can see through all solid objects, the JREF is not going to run through the periodic table followed by every known combination of elements in order to validate the claim. Together the JREF and the claimant will pick a substance for which no human has ever demonstrated an ability to see through. The protocol, which is part of the contract, then becomes restricted to that one substance. Period.

The JREF cannot change that substance unless, as I have said before, by doing so they reveal cheating or some other violation of the contract on the part of the claimant. If cheating is proved, then the claimant will be in breach, and the test will be over. If they fail to expose a breach of contract, then the claimant will have a very solid case for citing breach of contract by the JREF.

I don't know why everyone keeps making this so complicated. Simply put, the JREF can do whatever they want to in order to expose cheating. If they fail to expose cheating and what they did violated protocol, the JREF will be in breach (cheating). If they expose cheating, the test is over.

[MattC]

The JREF cannot change that substance unless, as I have said before, by doing so they reveal cheating or some other violation of the contract on the part of the claimant. If cheating is proved, then the claimant will be in breach, and the test will be over. If they fail to expose a breach of contract, then the claimant will have a very solid case for citing breach of contract by the JREF.

I don't know why everyone keeps making this so complicated. Simply put, the JREF can do whatever they want to in order to expose cheating. If they fail to expose cheating and what they did violated protocol, the JREF will be in breach (cheating). If they expose cheating, the test is over.

Yes and no. Well, yes and a yes with additional qualifiers.

Yes, if the JREF can expose cheating the test would be automatically called off with some angry words exchanged (very heated ones, I can imagine).

Yes, the JREF can do whatever they want to detect cheating so long as it does not conflict with the established protocol. Being a foundation established by a prominent magician, and having connections within the scientific world, the JREF is admirably suited for its current mission - it can detect cheating either by numbers or by people with a skill many organizations would be hard-pressed to match. A great deal of time and attention during protocol negotiations is paid to nailing down the precise parameters of the applicant's claim for just this reason.

Several claims jump to mind as indicative of this. The preliminary negotiations of David Koenig's claim, for example, focused almost entirely upon figuring out what it was he claimed to be able to do (through what really must in fairness be called an admirable **** smokescreen, few people can ******** that well and they deserve some credit for it). The negotiation process for Pavel's claim is nearing the final stage of discussion now (though some credit must be given to Startz for his aid in streamlining this procedure) after some time spent working out the preliminary details of the claimed ability.

I can think of only one case where testing an applicant was delayed for facility concerns rather than in negotiation, that of Achau Nguyen (I wonder what he's up to these days). Reading over the case, however, this seems to me a very clear outlier where a very unlucky combination of factors caused unavoidable delays in arranging a testing location and supervisor. In fact, it seems that many of the comparatively commonplace claims (telepathy, for example) the JREF has a "form" test of what it would like to see that they offer to the applicant almost immediately upon opening a dialogue, where if accepted negotiations can swiftly move into the facility stage.

~ Matt

[William Smith]

...
I don't know why everyone keeps making this so complicated. Simply put, the JREF can do whatever they want to in order to expose cheating. If they fail to expose cheating and what they did violated protocol, the JREF will be in breach (cheating). If they expose cheating, the test is over.

Knowing Randi, he would keep this as simple as possible.

Remember James Hydrick? (http://www.youtube.com/watch?v=7CASghTzNhc)



A proposition to all hypothetical-scenarios-blasting-people: There are currently several protocols being negotiated. One of them is the well-discussed Pavel Ziborov.

How about making some useful suggestions for Pavel's protocol? Perhaps you will find this more enlightning than "What if.." games.

[Startz]

A proposition to all hypothetical-scenarios-blasting-people: There are currently several protocols being negotiated. One of them is the well-discussed Pavel Ziborov.

How about making some useful suggestions for Pavel's protocol? Perhaps you will find this more enlightning than "What if.." games.

I fully agree with GzuzKryzt in principle. Since the Pavel protocol is currently under review by JREF, perhaps specific suggestions might wait until RemieV tells us what she might find useful. My take is that at this stage JREF would rather not have its elbow joggled, although I could be wrong.

(To avoid any misunderstanding, previous discussion here--including certainly from GzuzKryzt--has been very productive. This is just a timing issue.)

[Uncayimmy]

If the JREF violates the protocol and catches the claimant cheating, no court would rule against them. At the very least the court would rule that both parties violated the contract. In my opinion it is more likely that the court would say that the "no cheating" clause gives the JREF implicit permission to do take whatever steps needed to detect cheating so long as they actually detect cheating.

It's extremely difficult to come up with an example because of the way the protocols are written. Take the protocol here:
http://forums.randi.org/showthread.php?t=118952

It reads in part, "If any of the above listed are violated, or if it is determined the Subject has attempted to tilt the results in her favor due to trickery of any kind, the test will be halted immediately and considered a failure." I say if she agrees to not use trickery of any kind, the JREF is implicitly permitted to take steps to enforce that and prove she's cheating.

Looking at the protocol, there is very little the JREF is told they must do or cannot do. They actually have a lot of leeway. They can't make the claimant do something different unless she agrees to it, so the only thing left is to change something they do (violate protocol) or do something new. Is doing something new a violation of the protocol if it doesn't prevent the claimant from doing what she is obligated to do? I don't think so.

For the life of me I can't figure out an example in this protocol (or any others I looked at) where the claimant could cheat and the only method of detection would be for the JREF to change what they are required to do. Instead, let's look at some other type of civil contract.

Suppose we draw up a contract for you to shampoo my carpet once per week using my carpet cleaning machine. I pay you by the hour, and it usually takes two hours. Part of the contract says that I will provide a properly working machine, running water, and shampoo. I suspect that you're not actually cleaning my carpet when you know I am out of town. So, I disable the machine when I leave. Technically, I have violated the contract. You send me a bill for two hours of carpet cleaning without any mention of the machine being disabled.

You sue, and we go to court. Who wins?

Now, suppose you send me an invoice for two hours with an explanation that the machine was disabled, so you couldn't actually perform the contracted work. You're billing me for what you reasonably could have expected to be paid. Who wins?

[MattC]

Instead, let's look at some other type of civil contract.

Suppose we draw up a contract for you to shampoo my carpet once per week using my carpet cleaning machine. I pay you by the hour, and it usually takes two hours. Part of the contract says that I will provide a properly working machine, running water, and shampoo. I suspect that you're not actually cleaning my carpet when you know I am out of town. So, I disable the machine when I leave. Technically, I have violated the contract. You send me a bill for two hours of carpet cleaning without any mention of the machine being disabled.

You sue, and we go to court. Who wins?

Now, suppose you send me an invoice for two hours with an explanation that the machine was disabled, so you couldn't actually perform the contracted work. You're billing me for what you reasonably could have expected to be paid. Who wins?

In each case I am billing you for "what I reasonably could have expected to be paid" had you not violated your end of the contract. According to the terms of the contract as you state them, I have done nothing wrong - my ability to perform my end of the contract has been directly compromised by you failing to uphold yours.

~ Matt

[Uncayimmy]

In each case I am billing you for "what I reasonably could have expected to be paid" had you not violated your end of the contract. According to the terms of the contract as you state them, I have done nothing wrong - my ability to perform my end of the contract has been directly compromised by you failing to uphold yours.

~ Matt

That is not correct. In the first case you send a bill for services rendered when no services we're rendered. You lied. You also had a duty to inform me that I breached the contract. You didn't do that, and I did do that when I informed you that it was impossible for you have to have completed the contract.

In the second case you do the right thing and demand payment not because you fulfilled the contract but because I breached the contract. That's how contracts work.

In the former I have made an adequate case (preponderance of the evidence) to prove that you never even attempted to clean my carpet. Therefore, I do not have to pay. It doesn't matter that you could not have possibly fulfilled the contract because 1) you said you did when you didn't and 2) you did not inform me of my breach - I informed you after you lied about what you did.

It all hinges on whether you attempted to fulfill the contract rather than whether you could have fulfilled the contract. In the latter you prove that you attempted to fulfill it. In the former your actions prove that you never even attempted to do it. My actions are not a breach until you actually try to use the equipment.