Challenge Applications, past, present, future...

[Jeff Wagg]

Hello, I need your help.

As many of you have commented on, challenge applications are no longer being posted on the forum.

Due to limited resources both human and financial at the JREF, the idea has been proposed that we stop positing all the challenges.

Here's why:
1) We're not doing a good job of it since Kramer departed. (Translation: I'm not getting them up there.)

2) With me living in Vermont, and the applications coming into the Florida, the logistics are difficult.

3) It takes a LOT of time to enter all the applications. We received one last week that was well over 100,000 words that would all need to be typed in.

4) It complicates communication with claimants. Some are communicating with me on the phone, via e-mail, PMs, and USPS.

5) In many cases, posting the applications of people who have the appearance of mental illness makes the JREF look bad.

6) Some people apply just to "get their name in lights," so to speak.

However, I know that many people like reading the applications and consider it to be core to the JREF mission.

What I'm asking from you is ideas towards a solution to this problem. How can the applilcations be made public in a way that doesn't drain resources from the JREF beyond any meaningful benefit? Or is it best that they NOT be made public, and we create a sort of "Challenge News" area or something like that?

We could always invite claimants to test their ideas on the forum.

Thoughts? Ideas?

[TobiasTheViking]

"Challenge News" is better than nothing. And since i haven't got any good novell ideas myself i'll have to live with what smarter minds figure out. :)

[Darat]

I quite like the idea of a monthly "challenge news", a summary of numbers received, any interesting tit-bits from them, up date of any ongoing applications?

Perhaps also create forum entries for the applications that are moving forward? (Presumably that shouldn't be too onerous since you've got all that info to hand because you are currently dealing with them?)

I would also suggest that if at all possible the JREF scans in all new documents and just posts the scans as PDFs - there is no need to transcribe documents.

[Wowbagger]

You need to get more people handling the challenges.
If you formed a network of professional counselors or something, you can distribute the workload among them. They don't even have to live in Florida, of course, with the technology we got, nowadays.

[TheBoyPaj]

Numbers five and six can be solved by anonymising the claims.

Other than that, you need more volunteer typists, or scan the claims as previously suggested.

[Skiltch]

I agree with the already suggested ideas:
The applications could be scanned in and e-mailed to you, where you could just post them on the board as an image. This would solve the logistics and effort problems in 2 and 3.
The applications could have names blacked out, which would solve problems 5 and 6.
Applications on the phone would have to be transcribed, but e-mails and PMs could either be captured on screenshots and uploaded to the forums or copied into the message board directly using copy/paste.

Alternately, in some of the cases (such as telephone conversations) you could just summarize ("We have an applicant who called me the other day and said he could dowse for gold.")

I hope that the applications don't vanish entirely; they're a very interesting part of the forum.

[Jackalgirl]

Hello, I need your help. < snip >

Jeff,

I think that everyone else has already posted some pretty good ideas. I like the idea of just scanning the applications as images and posting them, or at least putting them into some kind of reference folder. Then, perhaps, kindly forum members could take a stab at reading & summarizing them. You, or someone else so designated, could double-check the summarization to make sure that it's accurate, and then post the summary in the Challenge section (with a link to the source material). I agree that blanking out the identifying information is a good idea.

You might want to consider soliciting people on a regional basis; i.e., people volunteer to summarize information from a particular area. That might make things too complicated, but it's an idea.

I'm moving to Japan next month (so I have to use a web cafe right now since my computer's being shipped), but I will be in a hopefully-cushy shore-duty-type job when I'm there. I would be more than happy to help summarize (or even transcribe) information once I'm settled. Of course, by then, I imagine you'll have a ton of much smarter people to help you than me. : )

-- Kat

[Gr8wight]

I agree with Darat. Perhaps a thread can be created when the application has officially been accepted and reaches the protocol negotiation stage. I think that is what most interests forum members. That would weed out a great many of the applications that are frivolous, untestable, or rejected for whatever reason.

I also like the idea of a periodical challenge update newsletter.

[roger]

What about a 2-4 sentence summary of each application? Now, I understand many applications are so obscurely worded, to put it politely, that no summary is possible. Yet, that in itself is a summary. Consider:


We received an application from Mr. A B on 3/4/06. He claims he can douse for gold. JREF is currently seeking qualified testers in his area.


We received an application from Ms. C D on 3/8/06. Her claims were unclear, and appeared to be written in ketchup. Jeff tried to clarify her claim, she became belligerent, at which point we closed her file permanently.

Any interesting claims, or claims that receive particular interest from the forum, could receive their own threads and more detailed documentation. I would argue that any claim that makes it to the preliminary test should eventually receive it's own thread with representative, if not exhaustive, documentation. If we are extremely curious we can always fly to FL and examine the records ourself.

[NobbyNobbs]

If there is a need for the application to be transcribed for posting, just find a few dedicated forum members to do the typing. I'm sure I wouldn't mind receiving a fax of a few applications a month and typing them up for the forum.

Although I don't see why a scan wouldn't work either.

[thomps1d]

I'll chime in with another vote for either scanning everything or sending material to volunteers for digitization...I know that I'd happily volunteer to shoulder some of the work free of charge. The challenge applications have always been one of my favourite parts of the forum.

[Unnamed]

I used to come for Randi's weekly commentary. Later, I started following the challenge application threads. Only much later I started looking around the other areas of the forum and created my account. I imagine that the same happened to others.

The Challenge is probably the most well-known aspect of the JREF. Please don't let it go in the dark.

[teck49]

As much as I don't want to post here, I feel must add my voice to the growing throng. The JREF is not just a trivial website with some nerds that do nothing but post in the forum. It is becoming more of an actual institution, and is quickly becoming woven into the fabric of western civilization. At the core of this process is the challenge, and the challenge thread that is propagating the awareness of the need for skepticism in our culture, and the absolute necessity of the teaching of scientific method.

If the challenge thread dies, it's just one more victory for the Sylvia Brown's of the world, and one step closer to the return of the dark ages. In my opinion it is essential that whatever steps need to be taken to keep it going, be taken now.

I'd gladly volunteer, but as a former, and probably future applicant, that would create a conflict of interest neither I, nor the JREF need.
I will state from a technical perspective that with the legal acceptance nation wide of electronically transferred signatures, (faxed, or emailed in jpegs), the entire application process can be done electronically, thereby eliminating the need for any transcription. If a volunteer would be willing to take care of the copying, pasting, and any editing that may be necessary, then Jeff’s duties could be trimmed down to simply answering the applicants emails, negotiating protocols, and the not-so-occasional reminding some poor soul to take their meds.

[rjh01]

Get some more resources. This means someone in Florida. It is important that the applications are put there to prove that they exist.

You could encourage e-mail applications.

[Cuddles]

I would say it is important to get applicant's names in the public domain, even if they aren't accepted, otherwise the JREF opens itself to claims of "I applied but they just ignored me". To point 5, I would say this actually supports JREF's position. The claims I have read seem fairly representative of believers in the paranormal. If they seem mentally ill when confronted with a test this can only help to convince observers just how silly such claims are.

I would also say that without the challenge the JREF forum would be destined to be just another debating board with nothing to distinguish itself from thousands of others.

I think sending applications to volunteers for typing up is a good idea, but could cause problems if applicants claim they alter or misrepresent applications.

[steenkh]

I think that the publication of the challenge correspondence is a publicity scoop for the JREF that should not be allowed to slip away. It is possible to see exactly why and how challenge application are rejected, and you can see the exact results of the few tests that are performed. This is invaluable, because many woo-woos will claim that they are not given a fair treatment, and this can only be countered by actually showing the correspondence.

The JREF challenge is what makes JREF famous all over the world. It is the reason why I started here. It is also the only part of JREF that is visible to woo-woos. It should get center attention.

If it is possible to have an official working with claims in Vermont, it should be just as possible to have a number of officials in other geographic locations that are responsible for publishing the correspondence. If money is the object, it is obvious from this thread that it would be easy to find volunteers who would do the job for nothing but the honour.

Alternatively, somebody would have to take on the job in Florida. I have no idea of the finances of the JREF, so I am not in a position to give any advice, but I would recommend to give priority to claims handling.

[Robaato]

(off topic): Jackalgirl, where in Japan are you going to be (if you care to say...)?

On topic: I have copious amounts of free time at the moment, and would be willing to help out in any way possible.

[Thomas]

I thought I had something of substance to add, but there are so many excellent proposals already. Thanks for making this thread Jeff.

[jmercer]

Like others here, I also think it's important to continue publishing the claims.

The Woo-community has repeatedly shown no reluctance in spreading lies concerning JREF's handling of various claims; taking the approach of publishing specific claims after the accusations would make JREF appear to be on the defensive to the undecided and/or gullible.

Publishing claims verbatim as they arrive does the reverse - it demonstrates to the world that JREF has nothing to hide.

Several posters have made points about how to address this issue resource-wise, but I'd like to suggest an alternative to typing in the claims. Low-resolution scanned images of the claims could be made and links to the images could be placed in the claim threads instead... this would have the added benefit of showing the original documents instead of typed copies. Even low-resolution documents are readable, and they shouldn't take up large amounts of disk space. (And I doubt you'll see a big bandwidth utilization from their retrieval, either.)

Regarding information that you don't want displayed - such as explicit language, private information, etc., such things can be photoshopped out. Alternatively, simply make a copy and white-out those items before scanning them for publishing.

Just a thought.

[lister]

It is possible to see exactly why and how challenge application are rejected, and you can see the exact results of the few tests that are performed. This is invaluable, because many woo-woos will claim that they are not given a fair treatment, and this can only be countered by actually showing the correspondence.

Yes! I totally agree. The verbatim transcripts of ALL correspondance are invaluable for supporting the notion that the challenge is fair and transparent. I have pointed believers many times to the threads as an example of how fairly people are treated. It would be a real step backwards if they were lost. I agree that scanned letters etc would be just as good if they cannot be transcribed.

[Startz]

As a frequent reader, albeit only occasional poster, I'd like to second (third? fourth?) the idea of posting scanned correspondence. It's not very resource intensive. And it's a real public service.

[Jackalgirl]

(off topic): Jackalgirl, where in Japan are you going to be (if you care to say...)?

On topic: I have copious amounts of free time at the moment, and would be willing to help out in any way possible.

(off topic) Raja, I and my husband are going to Misawa (I'm in the Navy). Are you, hopefully hopefully, nearby? It would be cool to meet a JREFite.

(on topic) You know, although I thought the idea of summarizing (with links to scans of the full topic) might be a good idea, I'm beginning to agree with jmercer & others. It is really important to make sure that /full/ transparency is maintained, and you can't do that if you filter the content in any way. As long as it's made clear in the Application that this information is going to be made public, I think it should be. Now, rule 3 says: "Applicant agrees that all data (photographic, recorded, written, etc.) gathered as a result of the testing may be used freely by the JREF". You might want to consider making it more explicit (such as "Applicant acknowledges that all provided documentation will be scanned and made public as a part of JREF's ongoing applicant log") or something.

Question: was there ever a point where the application rules came with "initial blocks", so that the applicant had to initial each rule to demonstrate that he/she had actually read them and agreed to them? Not that someone couldn't claim they just breezed through it, but (being in the Navy) I run into that "individual line acknowledgement" all the time and think it's fairly useful.

-- Kat

[Beleth]

The Challenge is the cornerstone of the whole Foundation. To let applications not be publicly acessible over the Web is to imprison the Foundation in the dark ages. And unless you're the Catholic Church, it's hard to get people to donate money to foundations imprisoned in the dark ages.

If transcribing the applications is too time-consuming, then I agree - scan them in and redact the parts where the applicant's personal information is given. Or require that the applicants include a digital copy of their application along with their notarized paper copy.

There has been a tremendous amount of good work done on the forums in helping out the test protocols of some applications. The GSIC chip protocol from last year springs immediately to mind. It would be a shame to only publish applications whose protocols have already been negotiated.

When all else fails, hire an intern to do it. It's not an expense; it's an investment.

[Spindrift]

Have you ever considering creating an on-line application?

While you would still get some snail mail apps, I would bet the majority would do it on-line.

That would cut down drastically on the transcribing.

[vIQleS]

I think there needs to be an intern who just transcribes applications - even if he's not actually the challenge administrator. Perhaps there needs to be some more fund-raising specifically for the purpose of paying this position.

If i lived in America and i didn't already have a great job - I'd take this job in a second (and work for a pittance - perhaps the intern could be paid in magic lessons :) )

Alternatively - I'll put my name down to transcribe applications. Send a scanned copy and I'll type it up and send it back - when I've done one send me the next. Then Jeff only needs to quickly check them...

What sort of volume are we talking anyway? 2 or 3 a week? 200 - 300 per week?

[Gr8wight]

In regards to Jeff's statement about the 100,000 word application. Hasn't Randi's strategy with those kinds of things in the past been to simply inform the applicant that anything more than a few simple paragraphs outlining the alleged ability and descritption of demonstration will not not be read? 100,000 words of badly spelled, grammatically incorrect, hand written drivel goes directly into the trash can, and the applicant is informed that they will have to try again after rereading the rules.

[Kimpatsu]

(off topic): Jackalgirl, where in Japan are you going to be (if you care to say...)?
As she's in the Navy, I'll posit Yokosuka.
Where in Nippon are you, Raja?

[Jackalgirl]

In regards to Jeff's statement about the 100,000 word application. Hasn't Randi's strategy with those kinds of things in the past been to simply inform the applicant that anything more than a few simple paragraphs outlining the alleged ability and descritption of demonstration will not not be read? 100,000 words of badly spelled, grammatically incorrect, hand written drivel goes directly into the trash can, and the applicant is informed that they will have to try again after rereading the rules.

That's a REALLY good idea -- I like it! If JREF decided to go with the "initial blocks" I suggested, one of them could read, "I understand that if my application is more than 500 words, not including the actual text of the application, that it will be returned, unaccepted, to me." Or something like that.

[saizai]

Scanning (w/ the contact details covered up) + OCR + forum-run summaries.

Maybe make mod powers to edit the post to add those summaries.

I find 'em quite interesting.

[saizai]

Also: make it all web-based except if absolutely necessary, and have the applicant submit a certified letter saying that everything sent under their email / whatever is theirs and has the same standing as physical mail.

[Flange Desire]

The Challenge Applications are integral to JREF's profile on the web,
and should not be retired.
I have no better suggestions than those good ones already posted.
Transparancy is the best policy.
$0.02

[SezMe]

I agree with teck49 (post #13). I think JREF should do all it can to get the application process into an all-electronic state, which completely removes the need for scanning/typing/etc. For example, requiring a notarized application forces the JREF into handling paper. Would a digitally signed app not serve the same purpose?

Similarly, the idea of a web-page application is a good one but I know it would take a lot of work.

My posting is intended to support the notion that having the apps be visible is central to the JREF mission.

[politas]

I agree with the idea that overly long applications are best rejected out of hand. Aside from that, Groklaw (http://www.groklaw.net/) has shown that scanning documents and getting forum members to transcribe them is an effective method, as long as there is an ability to correct transcribing errors.

[Robaato]

(off topic) Raja, I and my husband are going to Misawa (I'm in the Navy). Are you, hopefully hopefully, nearby? It would be cool to meet a JREFite.

-- KatAlas, I am in Tottori, which is pretty much the middle of nowhere, and half a country away from there....

However, maybe someday there could be a Japan JREF get together somewhere.

[dogjones]

Were it not for the challenge apps I would have glanced at the JREF site and forgotten about it. I suspect others would have done the same. Keep them, keep them, keep them.

I'll now throw my full weight (approx 155 pounds) behind the idea of scanning them. That's a reasonable amount of pressure.

[H3LL]

Just to add my support for scanned images of applications. A fine idea.

There are enough regular members of the forums to find resources to transcribe the information.

May I suggest that you post the image of the application and then call for one member to transcribe the document and one member to proofread it. I'm sure someone would be able to help. Those willing can let you know and I guess that you could mass-PM those on the list for volunteers.

Put me on the list.

We all have variable amounts of time to do the work. Right now I would not have time, but last week I could have transcribed one or two.

BTW, are you sure it's 100,000 words? That's half the size of The Republic by Plato.

ETA: This may work for other JREF projects that are not time critical but short of resources. I'm sure that lots of us would like to contribute. Failing that, make it a punishment for those misbehaving in the P, CE & SI forum. :D
.

[teck49]

It seems to be a unanimous return on this opinion poll. What say you Jeff? The sentiment is there. The technical/ logistical expertise is there. The offers to help are there. It looks like a done deal to me.
Now just pick someone to head up a team, toss the 100,000 word app in the fireplace where it belongs and "let's do this!"

[William Smith]

The Challenge Applications are integral to JREF's profile on the web,
and should not be retired.
I have no better suggestions than those good ones already posted.
Transparancy is the best policy.
$0.02

Second.
€0.02

[Gr8wight]

Second.
€0.02

Ooh, a raise!

Since my two pennies are Canadian, I'll just slide them back into my jeans and offer us my services as a volunteer to help the JREF with this situation in any way they might deem possible.

[Flange Desire]

It seems to be a unanimous return on this opinion poll. What say you Jeff? The sentiment is there. The technical/ logistical expertise is there. The offers to help are there. It looks like a done deal to me.
Now just pick someone to head up a team, toss the 100,000 word app in the fireplace where it belongs and "let's do this!"

There may be good reasons for it not happening exactly the way we may want.
There may be other factors to consider, and it may not be as easy as it seems, is all I am saying.

JW: If you have some other stumbling blocks (other than the obvious ones about resources), you might feel free to discuss those ...

[T'ai Chi]

Have more focus on the experimental design, and the statistics of the test (the outcome), rather than all the communications back and forth.

That is, focus more on the science than the personalities.

[steenkh]

Have more focus on the experimental design, and the statistics of the test (the outcome), rather than all the communications back and forth.

That is, focus more on the science than the personalities.
That would have worked if there was something to test, but in real life, most applicants fall by the roadside because they realise that they cannot do their thing with controls in place. The communication is very important because it enables outsiders to see that these applicants have not been met with unreasonable demands.

[Jeff Wagg]

Sorry, extremely busy of late..

Volunteers are a problem. Since I have started at the JREF, I have asked eager volunteers to undertake a number of projects. None of them have been completed, with one very large exception.

Using volunteers would also require materials to be sent via mail, and possibly lost. More likely, a volunteer would get bored and simply let the stuff pile up.

I like the result that scanning would provide. But its a very time consuming process, and I don't see a good source for resources.

I'm going to think about this some more. Possible, if we could have an online application form, where people just fill it out, that could save enough time that the paper applications could be managable. They could submit a notarized copy via e-mail to certify it, and then we'd be underway.

The cruise is in two days. I'll keep working on these idea, and please keep them coming. I'm very appreciative of the thought you've put into this.

[drkitten]

Have more focus on the experimental design, and the statistics of the test (the outcome), rather than all the communications back and forth.

Probably 95% of the applications do not get to a point where there is either a meaningful experimental design or quantifiable statistics. The vast majority do not even state a testable claim.

[ben m]

I don't see any reason to try to make the policy absolutely consistent. I think we'd all be happy if you posted as much as possible given your time constraints, even if it's in no way complete or thorough. A periodic News Digest would give room to mention everyone, so no applicant can accuse you of hushing up their abilities.

1) For email communications, post everything.
2) For paper mail, major events (initial contact, notarized apps or approximations thereof, significant protocol proposals) can be scanned; ongoing back-and-forths can be mentioned in the digest.
3) For phone/IM communications, just mention the challenger in the digest. "We received a call from John Doe, who claims that Nostradamus cured his hiccups. Advised him to read the FAQ, think about testability, and call back in a week."

The things I enjoy most about this forum are a) thinking about testability and b) seeing how people came to their weird beliefs. "How do you double-blind an EVP test?" is fun to think about; it's a puzzle and discussion fodder for forum residents, and an illustration of scientific inquiry for casual readers. On the other side, "I discovered that I could manipulate candles with my mind back in 1987. Here's what happened ... " gives us a some insight into the applicant's internal logic, which is interesting. So it'd be great if you could post such anecdotes when you get them.

Things to omit (or relegate to a line in the Digest), if you're short on time: a) discussions of affadavits, dates, or regional-testing-agent-contact-info, and b) unintelligible or claimless applications (e.g., Baltazar Lopez)---these are entertaining in a way, but there's plenty of similar stuff on the Web that doesn't need you to type it in.

[drkitten]

I don't see any reason to try to make the policy absolutely consistent. I think we'd all be happy if you posted as much as possible given your time constraints, even if it's in no way complete or thorough. A periodic News Digest would give room to mention everyone, so no applicant can accuse you of hushing up their abilities.

Unfortunately, a periodic News Digest wouldn't establish the needed transparency.

I think I agree with vIQleS and the others -- the Challenge is really one of the "core competencies" of the JREF and the primary reason that anyone has heard of the JREF or supports its mission. Without the Challenge, there's little to distinguish it from most of the other regional skeptic groups that no one has heard of or cares about.

The JREF Challenge is also one of the highest-profile arguments against the woo-of-the-week; "if you think you're for real, why waste time selling yourself to me for $50 -- why not go for an easy million instead?" And the transparency is key to being able to show that Randi doesn't cheat. Simply saying "we received an application from X and rejected it out of hand for failure to state a testable claim" doesn't establish any trustworthiness at all, especially when X then turns around and said "See, I applied and they rejected me because they knew I'd lose."

So I have to come down, strongly, on the "pay an intern if necessary because this is a major part of your business" side of the fence. Obviously, it's your foundation and your money -- but it's also your core credibility and brand identity at stake.

[H3LL]

Volunteers are a problem. Since I have started at the JREF, I have asked eager volunteers to undertake a number of projects. None of them have been completed, with one very large exception.

I've been a regular here for some time and don't remember seeing any requests for projects by JREF. Being blunt, could this be management of the volunteers as part of the issue?

Using volunteers would also require materials to be sent via mail, and possibly lost. More likely, a volunteer would get bored and simply let the stuff pile up.

I don't see why. The copies could be posted on a designated forum and the results posted on the same forum. The volunteer and results would be there for all to see, monitor and critique. Where then is the email issue?

I like the result that scanning would provide. But its a very time consuming process, and I don't see a good source for resources.

More time consuming than doing nothing but surely less time consuming than other options if JREF is serious about transparency?

I'm going to think about this some more. Possible, if we could have an online application form, where people just fill it out, that could save enough time that the paper applications could be manageable. They could submit a notarized copy via e-mail to certify it, and then we'd be underway.

This would be a good idea but both systems need to be in place. Several applicants seem to have trouble getting to grips with a pen, IT might be too much.

.

[politas]

Volunteers are a problem. Since I have started at the JREF, I have asked eager volunteers to undertake a number of projects. None of them have been completed, with one very large exception.The successful strategy used by Groklaw is to make the materials available, so that volunteers can do things as they have time available.

[kalen]

Kramer! Please come back!!!

[vIQleS]

Volunteers are a problem. Since I have started at the JREF, I have asked eager volunteers to undertake a number of projects. None of them have been completed, with one very large exception.


Two lists - one of people who want to transcribe, and one of people who want to proof read. You can be on both lists but you only get one thing at a time, when your name's at the top of the list you get sent the next scan or transcription.

Send each item to the name at the top of the list. Each volunteer gets one thing at a time - when it's returned, they go back in the list. When they get to the top send them the next one.

This would have to be automated in some sort of simple database otherwise it could get even more complicated than it is now...

And you might want to set deadlines - couple of days to a week probably...

[politas]

Two lists - one of people who want to transcribe, and one of people who want to proof read. You can be on both lists but you only get one thing at a time, when your name's at the top of the list you get sent the next scan or transcription.

Send each item to the name at the top of the list. Each volunteer gets one thing at a time - when it's returned, they go back in the list. When they get to the top send them the next one.

This would have to be automated in some sort of simple database otherwise it could get even more complicated than it is now...

And you might want to set deadlines - couple of days to a week probably...

Scan each page, make it available on the web with a "transcribe this" button which puts you on a split page, able to scroll the scanned page above and type in the transcription below. When a transcription of a page has been made, change the "transcribe this" link to a "proof-read transcription" button, allowing you to compare the scan with the transcription and either confirm or reject it. Do not allow the transcription and confirmation to be done by the same user, or from the same IP address. Require multiple confirmations to "lock" a transcript You could define JREF staff as privileged users, able to make a immediately locked transcript, or to over-ride the confirmation/rejection/locking process.

[rjh01]

Some people want to work more hours per month than others. You need to provide for this. Say minimum commitment 10 hours per month. Then the volunteer can work as many times that as they wish (ie 20, 30, 40 hours).

[ben m]

On time saving: most modern photocopiers are also scanners. Many also have multi-page document feeders and can even do both sides of the fed pages. I don't think Kinko's does, but many copy shops should be able to do this for you. You just plop a pile of material into the feeder, and the machine sucks it up at about two seconds per page. I've had good luck feeding formerly-folded sheets, spiral-notebook tearouts with the fuzz scissored off, etc., without getting bad jams. Any shop that has this service will presumably have various options for giving you the files afterwards. If there's not a storefront copy shop that will do it, there are big business-services companies that will do it for you for a few cents a page.

On transcribing: if I understand correctly, modern OCR (optical character recognition) for printed/typed text is a) nearly flawless and b) takes a few seconds per page. If you're getting typewritten (as opposed to handwritten) correspondence, post OCR-ed text in the Forum, and provide a link to the raw scans. No need for proofreading; OCR is good enough, and the linked scan can resolve any disputes about accuracy as they arise. This also saves bandwidth compared to making everyone read from scans. (It's probably a good idea to black out any signatures from such scans, though.) Some of the document scanning companies will also do the OCR for you.

For handwritten text, there's no good OCR solution, but why not post the raw scans in these cases?

[rjh01]

I have put printed documents on my scanner and that turns it into a text file. Very fast, accurate and easy. I am sure commercial ones can do it very fast. Look it up in the phone book.

[hcmom]

Scan each page, make it available on the web with a "transcribe this" button which puts you on a split page, able to scroll the scanned page above and type in the transcription below. When a transcription of a page has been made, change the "transcribe this" link to a "proof-read transcription" button, allowing you to compare the scan with the transcription and either confirm or reject it. Do not allow the transcription and confirmation to be done by the same user, or from the same IP address. Require multiple confirmations to "lock" a transcript You could define JREF staff as privileged users, able to make a immediately locked transcript, or to over-ride the confirmation/rejection/locking process.
Yeah, what politas said.

[CFLarsen]

I would also suggest that if at all possible the JREF scans in all new documents and just posts the scans as PDFs - there is no need to transcribe documents.

I got software that can OCR-scan PDFs.

[teck49]

Okay, so no word in two weeks. Is the challenge thread dead? Is it going to pull a "Lazarus" and come forth? Should we stay tuned to the same bat-time, same bat-channel?

[hcmom]

Sure, it's not like we were doing anything more exciting anyway...

[T'ai Chi]

Probably 95% of the applications do not get to a point where there is either a meaningful experimental design or quantifiable statistics.

One would have to put their job on hold, fly down, incur those expenses, and expenses to stay in a motel/hotel, food, etc., then sift through hundreds of paper files, that might not even be summarized...and this is just to check your claim of ~95%.

You made my point. :)

[CFLarsen]

One would have to put their job on hold, fly down, incur those expenses, and expenses to stay in a motel/hotel, food, etc., then sift through hundreds of paper files, that might not even be summarized...and this is just to check your claim of ~95%.

You made my point. :)

Why do you think JREF is at your beck and call?

[William Smith]

Have more focus on the experimental design, and the statistics of the test (the outcome), rather than all the communications back and forth.

That is, focus more on the science than the personalities.

How would you suggest JREF should realize "more focus on the experimental design", Mr. Chi?

What would be gained by more focus "the statistics of the test (the outcome)"?

Don't you think the communications with applicants allow a valuable insight into their minds, thought processes and behaviour?

[drkitten]

One would have to put their job on hold, fly down, incur those expenses, and expenses to stay in a motel/hotel, food, etc., then sift through hundreds of paper files, that might not even be summarized...and this is just to check your claim of ~95%.

Er, no. We've got a fairly representative sample in the "applications" that came in during KRAMER's tenure.

You could -- excuse me, any competent statistician could -- confirm the 95% figure within a day without leaving your Web browser.

[teck49]

Jeff, may we get an update as to the progress or lack thereof, of any discussion on the fate of the challenge forum? Please?

[scotth]

Jeff,

I think that everyone else has already posted some pretty good ideas. I like the idea of just scanning the applications as images and posting them, or at least putting them into some kind of reference folder. Then, perhaps, kindly forum members could take a stab at reading & summarizing them.

Actually, the way to go is to scan them directly into PDF documents and be done with it. Then post the PDFs.

[teck49]

The logistical details have been hashed & re-hashed. What is missing here is any participation from Jeff, or anyone else on the JREF staff. Is it just me, or do I smell apathy? What's up? Inquiring minds want to know.

[politas]

Actually, the way to go is to scan them directly into PDF documents and be done with it. Then post the PDFs.

Is there something about PDFs that make them a better medium than simple image files? Certainly for people on dial-up or other slow internet connections, large image PDFs can be very annoying to load.

[nathan]

Is there something about PDFs that make them a better medium than simple image files? Certainly for people on dial-up or other slow internet connections, large image PDFs can be very annoying to load.
A PDF of an image will be just as big as the image file[1] (well, bigger really). Sometimes PDFs can be smaller, but that's because they've resampled the image at a lower resolution (often making them illegible).

[1] for a suitable compression of the image file, png, gif, jpeg, depending on source.

[scotth]

Is there something about PDFs that make them a better medium than simple image files? Certainly for people on dial-up or other slow internet connections, large image PDFs can be very annoying to load.

Well, When I scan to PDF, I get 2 advantages.

1) If there is typewritten text, it is OCR'd and stored as text in the PDF making the file size smaller.
2) If it stays as all image (or any combo of text and image), the pages linked back together so that it is a "document" rather than a collection of files.

[politas]

Well, When I scan to PDF, I get 2 advantages.

1) If there is typewritten text, it is OCR'd and stored as text in the PDF making the file size smaller.
2) If it stays as all image (or any combo of text and image), the pages linked back together so that it is a "document" rather than a collection of files.

Ah, so you are talking about running the pages through an OCR process, not just saving the images in a PDF.

There are many different software tools which handle scannign and creating PDF files. Saying "just make them PDFs" means different things to different people.

As far as OCR goes, what is the accuracy level? I haven't looked into OCR in a long time, since the days when it had around 85% accuracy, which was almost completely useless for most tasks.

I consider the required acuracy for OCR to be about the same as the required accuracy for voice recognition - 100%. Anything less requires extensive proofreading, at which point it's almost as easy to simply type.

[scotth]

Ah, so you are talking about running the pages through an OCR process, not just saving the images in a PDF.

There are many different software tools which handle scannign and creating PDF files. Saying "just make them PDFs" means different things to different people.

As far as OCR goes, what is the accuracy level? I haven't looked into OCR in a long time, since the days when it had around 85% accuracy, which was almost completely useless for most tasks.

I consider the required acuracy for OCR to be about the same as the required accuracy for voice recognition - 100%. Anything less requires extensive proofreading, at which point it's almost as easy to simply type.

Well, I can scan directly into a PDF as a one step process. There is a check box for doing OCR or not.

The accuracy is amazing. It even gets the font perfect nearly every time. Generally, I can't tell without trying to 'select the text' whether it inserted the text as text or as an image.

So, with the OCR turned on, the look is the page is perfectly preserved. I hadn't done it in years, either. I used it the first time a few weeks ago and was incredibly impressed.

[steenkh]

The accuracy is amazing. It even gets the font perfect nearly every time. Generally, I can't tell without trying to 'select the text' whether it inserted the text as text or as an image.
That must be amazing. Considering the gibberish that the JREF has to put up with, what do you reckon is the accuracy of the OCR program to fully capture the original spelling and meaning?

[Jeff Wagg]

We haven't made a decision yet.

There are lots of factors to consider, not all of which are public. Please consider that while many people enjoy reading the challenge applications, people are also critical of the JREF for posting them. "Why are you abusing people who are obviously mentally ill" etc. It's a PR problem.

I frequently get mail that says basically, "When will you be posting the challenge applications again? I need a good laugh at these bozos!"

Well, if you've come to make fun of delusional people.. I'm not interested in supporting that.

Still, there are valid reasons to post the challenge correspondence. So this issue is not closed.

I'm testing a scan to pdf arrangement now. I think that might work, though OCR is out.

[teck49]

Jeff,
Glad to hear from you. Thanx for your reply.
If you're eliminating OCR from the process, then just scan them in as jpegs. No muss, no fuss, and only the guys from the conspiracy theory thread will be saying they've been altered.

[timokay]

I frequently get mail that says basically, "When will you be posting the challenge applications again? I need a good laugh at these bozos!"

Well, if you've come to make fun of delusional people.. I'm not interested in supporting that.

This is great to hear. It is my hope that any applicant who has a sincere challenge will be welcomed, and that the "bozo" ones, Like Sally for example, who just want a forum to spout nonsense are not goaded on for page after page.

As an aside, I wish I lived in Florida, and was a little closer to retirement. My years of experience as an Art Director (dealing with tempermental artists), and my ability to keep them on track and on topic, might make me qualified to shepard the applicants to actually being tested. The Skeptic group I am involved with on the west coast certainly thinks that I channel Kramer.

[Paul2]

Jeff, thanks for the update.

I would urge JREF to make a tentative decision about this, and then post the tentative decision, laying out the relevant factors, and then give the forum a chance to weigh in one last time on the factors that JREF saw as relevant. JREF could then make a final decision based on the forum's feedback.

This would give a minimal level of back-and-forth, of dialog, about this question, and I think would be essential for substantive as well as PR concerns for the forum members.

[T'ai Chi]

"Well, if you've come to make fun of delusional people.. I'm not interested in supporting that. "

Then you don't read the weekly commentary?

;)

[Skiltch]

Why not just have the claims be anonymous? There seem to be valid arguments against it (applicants do not want their names to be known, giving publicity to publicity hounds, making the site look like its in the business of making others look like morons), and the only reason for it I have seen is that people might apply and then say that they didn't (but people say that now, just replacing "I applied and they ignored me" with "I applied and they won't let me use my own protocol that is the only one that can demonstrate my superpower.")
Granted, safety concerns may sometimes trump this (For instance, if an applicant claims to be able to kill people and then resurrect them and plans to test it on random bystanders in a few days.) But generally I don't see the harm in anonymous entries, and this would solve the PR problem.

[petre]

Why not just have the claims be anonymous? There seem to be valid arguments against it (applicants do not want their names to be known, giving publicity to publicity hounds, making the site look like its in the business of making others look like morons), and the only reason for it I have seen is that people might apply and then say that they didn't (but people say that now, just replacing "I applied and they ignored me" with "I applied and they won't let me use my own protocol that is the only one that can demonstrate my superpower.")
Granted, safety concerns may sometimes trump this (For instance, if an applicant claims to be able to kill people and then resurrect them and plans to test it on random bystanders in a few days.) But generally I don't see the harm in anonymous entries, and this would solve the PR problem.

Then what happens if a Sylvia Brown or other such well-known wishes to submit anonymously? Or more likely, such a person would CLAIM to have submitted anonymously (because they didn't want to be bothered with all that extra fame) and was "rejected" for whatever reason they want to make up.

The purpose of the JREF is not to abuse the mentally ill anymore than it is the purpose of emergency services to do so. Those in need of mental help contact both, and both do their best to see that such people get the help they need.

[Skiltch]

But they claim that now, don't they? There are people saying "I wanted to apply but Randi was scared" and things like that. Making the claims anonymous wouldn't make it worse, I don't think.