A question about protocols & revision

[jmercer]

Kramer (et al.),

I was thinking about some of the posts here, and I was wondering about something. What if an applicant passed the preliminary test... and then somebody pointed out a possible flaw or exploit in the protocol after the fact.

Would JREF be required to proceed to the final test with the same protocol regardless of that flaw? Or would the protocol be subject to renegotiation under the circumstances?

(edited to add below)

My concern is that as time passes, someone may find a way to negotiate a protocol that allows them to "beat the system". I guess I'm just a bit paranoid, but 1 million bucks is a lot of incentive to cheat. :)

[Metullus]

Originally posted by jmercer
Kramer (et al.),

I was thinking about some of the posts here, and I was wondering about something. What if an applicant passed the preliminary test... and then somebody pointed out a possible flaw or exploit in the protocol after the fact.

Would JREF be required to proceed to the final test with the same protocol regardless of that flaw? Or would the protocol be subject to renegotiation under the circumstances?

(edited to add below)

My concern is that as time passes, someone may find a way to negotiate a protocol that allows them to "beat the system". I guess I'm just a bit paranoid, but 1 million bucks is a lot of incentive to cheat. :)

I see nothing in the rules that preclude changing the protocol after a preliminary test. The rules state that the preliminary test is undertaken only "to determine if the applicant is likely to perform as promised during a formal test. "

Nothing I read in the application requires that the preliminary test be identical to the formal test.

The application at least considers the possibility of changing the protocol: "No part of the testing procedure may be changed in any way without the agreement of all parties concerned."

[Patricio Elicer]

Originally posted by jmercer
My concern is that as time passes, someone may find a way to negotiate a protocol that allows them to "beat the system". I guess I'm just a bit paranoid, but 1 million bucks is a lot of incentive to cheat. :) I share your concern.

I just checked the "Paranormal Challenge" section on main page and found no clear statement of the kind "The JREF reserves the right to re-negotiate the protocol after the preliminary test has been conducted, etc, etc.......".

Instead, there's the following paragraph:Upon properly completing this document [the Application for Status of Claimant] and agreeing upon the test protocol, you will receive your application back, signed on the reverse by JR. The applicant then becomes eligible for the preliminary test, which, if successful, will result in the formal test.This strongly suggests, though is not stated explicitely, that the protocol for the final test will be the same as the one for the prelim. The applicant will get the agreed protocol document signed by James Randi prior to the preliminary test, so I don't think the JREF has a choice of re-negotiation after that.

I think it's worth adding this issue to the FAQs

[Patricio Elicer]

Also, on Rule 4 we read:No part of the testing procedure may be changed in any way without the agreement of all parties concernedPretty clear.

[jmercer]

True. It's sort of a mutual protection clause. Let's hope that JREF always gets the protocol right. And it's one more good reason for people to analyze the protocols here for Randi and Kramer.

[Vikram]

Originally posted by Patricio Elicer
I share your concern.

I just checked the "Paranormal Challenge" section on main page and found no clear statement of the kind "The JREF reserves the right to re-negotiate the protocol after the preliminary test has been conducted, etc, etc.......".

Instead, there's the following paragraph:This strongly suggests, though is not stated explicitely, that the protocol for the final test will be the same as the one for the prelim. The applicant will get the agreed protocol document signed by James Randi prior to the preliminary test, so I don't think the JREF has a choice of re-negotiation after that.

I think it's worth adding this issue to the FAQs
The FAQ already addresses this issue here (http://www.randi.org/research/faq.html#5.2)

5.2. What happens between the preliminary test and the official test?

What you should expect is for none of the agreed-upon documents to change. There will not be any renegotiation of your abilities or how they will be tested.

[Patricio Elicer]

Originally posted by Vikram
The FAQ already addresses this issue here (http://www.randi.org/research/faq.html#5.2) Ah, thanks!. I had missed that.

BTW, thanks Beleth! :)

[Timothy]

The question really hasn't been answered yet.

It would be insightful to get a comment from JREF itself about how it protects its interests from awarding the prize for a mistake or flaw in the protocol. (But perhaps for legal reasons it would be better to not have anything in posted casually in a forum that hadn't been reviewed by JREF's lawyers.)

While the FAQ states "What you should expect is for none of the agreed-upon documents to change. There will not be any renegotiation of your abilities or how they will be tested," that's simply the FAQ. It's not the application, nor could be considered binding.

If it is the intention of JREF to follow this to the letter, however, it does mean that the agreed upon protocol must be formulated with the $1 million formal testing in mind.

(I had originally thought that the aim of the preliminary was a controlled quick-and-dirty to assure that the applicant wasn't deceiving himself and to weed out the frauds, and that the formal would be a more rigorous version. Apparently that is not the case.)

Seeing how a test protocol that *might* have given the applicant a 1 in 2 chance of guessing (pointed out by myself and others) was given a casual OK in the forum by both James Randi and the Challenge Facilitator, there's a potential problem with the following conflicting interests:

- JREF wants to perform preliminary tests as expeditiously as possible (so as to not be accused of dragging things out).
- Ideally JREF wants to keep the protocols identical (so as to not be accused of changing the rules on an applicant).
- Hurried acceptance of preliminary protocols may allow a mistake to slip through unnoticed. (I don't know how close it was to actually happening in this case.)
- Drawn out negotiations of detailed, proper protocols wastes time, since few actually get tested.
- Posting an almost-final negotiated protocol for forum members to comment on makes it look like JREF doesn't know what it's doing.

I see no way to reconcile these. My only suggestion would be for JREF to take a little more time when the rare serious application comes along. (This one has been around for less than three weeks.)

- Timothy

[roger]

Yet Randi states (http://www.randi.org/jr/07-30-2000.html) Then, as stated in the rules, we have to (b) conduct a preliminary test - with much less stringent conditions and much more attainable percentages of success

And here (http://www.randi.org/jr/05-04-2001.html), in an letter written to the University of Arizona that offers to test Schwartz for the prize, Randi writes: Should that data and the responses to any questions be satisfactory to the Qualified Panel, the JREF will agree that the "preliminary examination" as designated in our official offer (see www.randi.org/research/challenge.html) has been accomplished. The next step would be to refine a protocol that would be satisfactory to the Qualified Panel and to the needs of Dr. Schwartz, to proceed with the formal test of the "human energy systems" phenomena.

So here the formal test would have been negotiated only after the prelim was passed.

Here's a commentary (http://www.randi.org/jr/03-16-2001.html) where Randi accepts a letter stating that the writer will cause angels to cause a specific earthquake at a specific place at a specific time as sufficient to pass the preliminary. Clearly, the final test can be in no way the same as the preliminary, nor has there yet been any negoiated final test.

In another example, Randi specifically states (http://www.randi.org/jr/062802.html) that he will negotiate the final test only after the preliminary is successfully passed: If he succeeds in the preliminary test, I will remain in Brazil while we agree on the formal test for the million-dollar prize, which will be carried out as soon as possible.

There are, no doubt, many more examples of this kind sprinkled through the commentaries, but these quotes amply demonstrate that the JREF, at least in the past, construed the preliminary and formal tests as different in kind and scope, and that the formal test is not necessarily worked out before the preliminary is done.

[KRAMER]

All I can add to what has already been offered here is the following:

If Randi gets fooled, the money is awarded regardless, and the JREF One Million Dollar Challenge is terminated.

So, one doesn't need to actually have paranormal powers to win the big bucks.

One simply has to be smarter than James Randi.

[Patricio Elicer]

Originally posted by KRAMER
If Randi gets fooled, the money is awarded regardless, and the JREF One Million Dollar Challenge is terminated.Fair enough, but it doesn't look reasonable that after getting fooled in the prelim, and being aware of that, he will allow to be likewise fooled in the final.

[jmercer]

On the other hand, I can see where permitting JREF to change the protocols could be presented in a negative light by a then-unsuccessful candidate.

"I passed the preliminary, and so they changed the protocols to make it impossible for me to win the million!" sort of thing. So - while I'm still concerned - I understand the logic and accept it.