JREF Challenge Statistics

[catbasket]

I feel it's time to quote something jj said, which I felt would be a good aide-mémoire:

Um, let's look at this T'ai Chi chap's performance. First, he's had at least 4 nics, those being T'ai Chi, Whodini, JZS, and Statisticool. Second, he's reknowned for taking things out of context, malicious misunderstanding, outright misstatement of others positions, and the like. Third, he has nothing positive to say, and appears to emerge mostly to indirectly attack or incite other posters.
T'ai - if you think you're impressing the lurkers ... well you've failed abysmally in my case.

[T'ai Chi]

Being that the responses I've received are 'self-selected', I can safely discard them then, according to some. ;)

Mercutio wrote


It tells nothing about the beliefs about dowsing in the general population,


So? Where was it said that that was a goal? It is setting out to answer what the statistics are of those who took the test, you know, that whole descriptive vs. inferential thing that I've repeatedly said.


It is an absolutely useless question.


Frequently it is said that the plural of "anecdote" and is not "data". I'm starting to think that some around here believe that the plural of "complaint" is "rebuttal".


This is why combining them is such a bad idea.


Check out the bold in what I wrote


Test information and results listed singlely is OK, but putting all of these single results in a table and not even combining them in a meta-analysis is taboo, according to some, for some odd reason.


That is, the possibility of a meta analysis was raised. The main issue is getting all the statistics for the tests from skeptical organizations made readily available to any interested party. If an organization has a test as its main draw, the least it should do IMO is make the data from the tests more easily available.

[T'ai Chi]

T'ai - if you think you're impressing the lurkers ... well you've failed abysmally in my case.

Do try and stay on topic.

[catbasket]

Do try and stay on topic.
Ok -
Does anyone believe that after 1,000+ tests that are statistical in nature are carried out, that anyone will win by chance?
Has the JREF carried out 1,000+ tests?
Were 1,000+ statistical in nature?

No. And no. (http://www.randi.org/research/faq.html#1.3)

Will anyone win by chance?

I think it's exceptionally unikely anyone will win by chance; someone could. But -


What exactly is the point of your "what if?" scenario?

[Mercutio]

Being that the responses I've received are 'self-selected', I can safely discard them then, according to some. ;)
You can, and do, so quickly here...why not when it is important?
So? Where was it said that that was a goal? It is setting out to answer what the statistics are of those who took the test, you know, that whole descriptive vs. inferential thing that I've repeatedly said.
Which does not answer my question--what is the purpose of asking the question? You get an answer--ok, what does it tell you? I can't see anything useful, but I was wrong once before, about 5 years back, so I suppose it could happen...
Check out the bold in what I wrote
Ok, so your contention is that putting them all in one table is not "combining them". I think it is, and I advised against it, even if no meta-analysis is performed.

That is, the possibility of a meta analysis was raised. The main issue is getting all the statistics for the tests from skeptical organizations made readily available to any interested party. If an organization has a test as its main draw, the least it should do IMO is make the data from the tests more easily available. because...

You have repeated your request many times. What you have not done is justify it. Suppose you get the table of data. What purpose does it serve?

[T'ai Chi]

Which does not answer my question--what is the purpose of asking the question? You get an answer--ok, what does it tell you?


You're asking, for example, what the number of tests done per year tells you? It tells you the number of tests done per year. You're asking what does how well the dowsers performed tell you? It tells you how well the dowsers who were tested performed.

Anything I can do to clarify.


Ok, so your contention is that putting them all in one table is not "combining them". I think it is, and I advised against it, even if no meta-analysis is performed.


So this is where your confusion lies Mercutio. A list of data from many single experiments is not the same as combining data from experiments. The assumptions behind your "advising" are therefore flawed, and are safely dismissed.


Suppose you get the table of data. What purpose does it serve?

A question one should be asking instead, is what purpose do tests serve if one has such a hard time seeing the statistics from them.

[Mercutio]

You're asking, for example, what the number of tests done per year tells you? It tells you the number of tests done per year. You're asking what does how well the dowsers performed tell you? It tells you how well the dowsers who were tested performed.
Thank you.

So it tells you nothing. It has no use outside describing a self-selected sample. It is useless.

Anything I can do to clarify.
Except actually clarify. But you have answered sufficiently; if you actually did have a use for it, you'd have told it.
So this is where your confusion lies Mercutio. A list of data from many single experiments is not the same as combining data from experiments. The assumptions behind your "advising" are therefore flawed, and are safely dismissed.
Nice try, but you got it backward. My advising was based on the mere fact of listing the data. That you do not call that "combining" is utterly irrelevant, although it does speak to your understanding. You do not call this listing "combining", and therefore do not see the inherent problem. You are quite simply wrong.
A question one should be asking instead, is what purpose do tests serve if one has such a hard time seeing the statistics from them.The challenge tests have already served their purpose. They have tested the claims of the claimants. They have no obligation to suit your purpose.

Seriously, TC, I am finding it harder and harder to believe that you are not understanding. Do you have an expert statistician that you trust, to ask to explain it to you? Failing that, could you elaborate on why you do not think that the data are not appropriate for further use? Perhaps if you do, I will be able to identify where you are mistaken. At this point, I am convinced that you are indeed mistaken--I am willing, even eager, to be proven wrong, but it seems excruciatingly clear to me that you are quite simply wrong, yet for some reason unable or unwilling to see this.

[T'ai Chi]

So it tells you nothing. It has no use outside describing a self-selected sample. It is useless.


What other descriptive statistics do you find "useless"?

If one is interested in skepticism, and especially the tests done by skeptical organizations, I'd think the data from their tests (http://www.statisticool.com/jrefchallengestats.htm) would be useful for similar descriptive reasons, so one can better understand claimant characteristics, experimental design, statistics in skepticism, and interpreting data, among others.


They have tested the claims of the claimants.


That is trivially true. What is also true, is that from some tests there are statistics. Can these be made more easily available?


They have no obligation to suit your purpose.


Strawman alert. No one is saying anyone has any obligation.


Seriously, TC, I am finding it harder and harder to believe that you are not understanding.


You are apparently finding a lot of things hard on this thread: arguing bias from a hypothetical non-realistic situation where optional stopping and testing the claimaint based on their expectation of performance are done, thinking the listing of single experiments is combining, and so on.


At this point, I am convinced that you are indeed mistaken--I am willing, even eager, to be proven wrong, but it seems excruciatingly clear to me that you are quite simply wrong, yet for some reason unable or unwilling to see this.

You want me to prove your opinon that my opinion is wrong, wrong?

The argument being presented is that it would be useful for skeptical organizations to make data from tests more easily available. I'm curious to understand why you believe one can be "wrong" about that opinion.

How many tests were done per year? What % of tests were on dowsing? What is the closest that someone has gotten to passing a test? How does this vary for different skeptical organizations? Broadly classified, what is the most common experimental design of these tests? Etc. I'd like to understand skepticism better, especially the results from tests done by skeptical organizations. You know what they say, if it cannot be expressed in number, it is knowledge of a meager type, or something like that.

It might work for you to know that dowsing is a claim that has been tested a lot. Others would like an actual number to really understand the issue.

[Mercutio]

What other descriptive statistics do you find "useless"?
Oooooh, you got me! They are not "useless"; they describe a small, self-selected sample. A sample which has already been examined, in terms of the question the sample was obtained to examine. They are only useless if we attempt to do anything else with them, like answer some of the questions your website suggests we will answer in looking at this sample.

If one is interested in skepticism, and especially the tests done by skeptical organizations, I'd think the data from their tests (http://www.statisticool.com/jrefchallengestats.htm) would be useful for similar descriptive reasons, so one can better understand claimant characteristics, experimental design, statistics in skepticism, and interpreting data, among others.
"useful for similar descriptive reasons"...are you admitting that they have no inferential use at all? And of course, you are wrong about better understanding claimant characteristics, too, at least in any practical sense. Suppose you find that there is a particular gender composition; you can make no statement about whether that composition is typical for such samples, whether it is typical for the population, about the reasons for the gender composition in this case, whether the gender composition might be an artifact of the challenge process or not...you are in possession of a trivial fact with absolutely no practical utility. You call this "useful", but other than repeating your assertion that it is, you do not explain how.
That is trivially true. What is also true, is that from some tests there are statistics. Can these be made more easily available?
It is not trivial; it was the purpose of the data. What is trivially true is that these descriptive data describe the sample.
Strawman alert. No one is saying anyone has any obligation.
Oh, good. We can safely ignore your requests.
You are apparently finding a lot of things hard on this thread: arguing bias from a hypothetical non-realistic situation where optional stopping and testing the claimaint based on their expectation of performance are done, thinking the listing of single experiments is combining, and so on.I have explained my reasoning at every point, or at least I think I have. Please feel free to show where I am mistaken. Please feel free to show your own reasoning, and what use the descriptive statistics would serve. I have no problems admitting I am baffled by that.
You want me to prove your opinon that my opinion is wrong, wrong?
Again, I believe that I have supported my opinion. If I have not, please feel free to show where I have failed.

The argument being presented is that it would be useful for skeptical organizations to make data from tests more easily available. I'm curious to understand why you believe one can be "wrong" about that opinion.
Did you not read my earlier posts today? At 12:37 (EST), I explained this. Do you contend that making misleading data available is a good thing? Perhaps you have more trust in the innate statistical capabilities of untrained individuals to take into account the inherent problems of self-selection (among others) into account.

How many tests were done per year? What % of tests were on dowsing? What is the closest that someone has gotten to passing a test? How does this vary for different skeptical organizations? Broadly classified, what is the most common experimental design of these tests? Etc. I'd like to understand skepticism better, especially the results from tests done by skeptical organizations. You know what they say, if it cannot be expressed in number, it is knowledge of a meager type, or something like that.
Suppose you had answers to each of these. What would each tell you? Would it tell you why X many tests were done? Whether dowsers were preponderant (for argument's sake) because they had most applications, or because the tests were easier, or because they were the ones who came to a mutual understanding with the testers, or some other reason? Without experimenter manipulation, what does the correlative data about which organization tests what tell you?

If you would like to understand skepticism better, this is a terrible way to start! If you wish to answer a given question, first find out what sort of data you would need in order to do that! These data simply do not tell you anything about the questions you claim to be asking! Please....find somebody you trust who knows about methodology, to explain it to you, if you do not want to listen to me.

It might work for you to know that dowsing is a claim that has been tested a lot. Others would like an actual number to really understand the issue.Tell me what the actual number would tell you. What would that number mean? (this is not a rhetorical question. What is it that you really think you could glean from this number?)

[T'ai Chi]

A sample which has already been examined, in terms of the question the sample was obtained to examine. They are only useless if we attempt to do anything else with them, like answer some of the questions your website suggests we will answer in looking at this sample.


Why do you believe they are useless? I don't find them useless. For example, the tests are done for the year 2005, but I'm interested in seeing a pie chart showing the % of type of tests for 2005. Maybe the trend in being tested for dowsing is down, up, or constant, from the years past, for example. Who knows.

You seem to want to pretend that once the test is over, one cannot get anything from the data other than 'test passed' or 'test failed', which is rather a limiting view.


"useful for similar descriptive reasons"...are you admitting that they have no inferential use at all?


Limited use for inferential statistics I'd say since the same is not randomly chosen, we don't know if they are representative. It is hard to say without seeing actual data and understanding the specific details of the tests. At this point I'm just interested in descriptive statistics of the tests done by skeptical organizations that are statistical in nature.


Suppose you find that there is a particular gender composition; you can make no statement about whether that composition is typical for such samples,


Understanding characteristics of the sample. I thought that was made clear when I talked about descriptive statistics for the sample.


Oh, good. We can safely ignore your requests.


You need to work harder at ignoring. ;)


I have explained my reasoning at every point, or at least I think I have.


Instead of asking me to show where you are mistaken, you need to show where you are correct. Making a hypothetical example that is non-realistic (relies on option stopping and pretending that the observed data is compared to what the claimant expects) is not a good way to attempt to show you are correct.


Do you contend that making misleading data available is a good thing?


Why do you claim it is not misleading when listed singly, but is misleading when all listed together?


Suppose you had answers to each of these. What would each tell you? Would it tell you why X many tests were done?


You questiosn are akin to asking me what a person's age would tell me. Well, it would tell me how old they are. You could keep asking what that really would tell me, but it isn't a very rational way to argue.


These data simply do not tell you anything about the questions you claim to be asking!


I claim to be asking questions? Nope. I list the actual questions out, for example. But sure they do. They tell us about the sample, which is what is being asked about. How many times will that need to be repeated?


Tell me what the actual number would tell you.


Try telling me some actual numbers first. I understand you are having some trouble with that. My point is that you shouldn't be having that trouble.

[CFLarsen]

Do try and stay on topic.

Ok -

Has the JREF carried out 1,000+ tests?
Were 1,000+ statistical in nature?

No. And no. (http://www.randi.org/research/faq.html#1.3)

Will anyone win by chance?

I think it's exceptionally unikely anyone will win by chance; someone could. But -


What exactly is the point of your "what if?" scenario?

Note that T'ai Chi never answered the question.

[Mercutio]

Why do you believe they are useless? I don't find them useless. For example, the tests are done for the year 2005, but I'm interested in seeing a pie chart showing the % of type of tests for 2005. Maybe the trend in being tested for dowsing is down, up, or constant, from the years past, for example. Who knows.
Ok, so you are looking to infer. In a self-selected sample, any trend you find is utterly meaningless, and you should know this.

The rest has been responded to at length.

[Mercutio]

[snip]
Understanding characteristics of the sample. I thought that was made clear when I talked about descriptive statistics for the sample.
[snip]
You questiosn are akin to asking me what a person's age would tell me. Well, it would tell me how old they are. You could keep asking what that really would tell me, but it isn't a very rational way to argue.


For those who are wondering...

Any numbers we are after are chosen for a reason. Suppose we want to measure somebody's size, or their speed. How big is this person? Well, it rather depends--are you planning on buying him pants? Fitting him in a car seat? Firing him through the air in a trebuchet? The use we are planning on putting the numbers to will determine what sort of numbers we gather in the first place. Numbers which are approprate for one use may be completely useless for another. How fast is this person? Again, it depends--in a sprint? A mile? A marathon? Running backward? Numbers taken to answer one question do not answer the other questions.

"Why do you want to know how big that person is?" "So I can know how big he is." This is TC's answer, and it is easily seen to be meaningless. When we do get an inkling of the sorts of questions he wants to be able to answer (at least according to his website), we see they cannot be answered using the dataset he wishes to use.

[T'ai Chi]

Ok, so you are looking to infer. In a self-selected sample, any trend you find is utterly meaningless, and you should know this.


Nope. It is not inferring to see the patterns in the data as was stated in


For example, the tests are done for the year 2005, but I'm interested in seeing a pie chart showing the % of type of tests for 2005. Maybe the trend in being tested for dowsing is down, up, or constant, from the years past, for example. Who knows.


If you understand inferring, it is making a statement about the population based on data from the sample. The above is just looking at information from the sample.

Trends in the sample are not "utterly meaningless" as you erroneously claim. For example, the number of fatalities at a particular intersection from month to month are quite not "utterly meaningless".

[CFLarsen]

Nope. It is not inferring to see the patterns in the data as was stated in

...

If you understand inferring, it is making a statement about the population based on data from the sample. The above is just looking at information from the sample.

Trends in the sample are not "utterly meaningless" as you erroneously claim. For example, the number of fatalities at a particular intersection from month to month are quite not "utterly meaningless".

Seeing development in gender patterns in the data is not making a statement about the gender of those attending the seances??

Am I missing something here?

[T'ai Chi]

Any numbers we are after are chosen for a reason.


That is trivially true.


Numbers which are approprate for one use may be completely useless for another. How fast is this person? Again, it depends--in a sprint? A mile? A marathon? Running backward? Numbers taken to answer one question do not answer the other questions.


It depends on what use, sure. The use in this case would be to get an understanding of the tests done by skeptical organizations that are statistical in nature. It may work for you to know that dowsing is the most tested claim, for example. Others need an actual number to increase their knowledge.


"Why do you want to know how big that person is?" "So I can know how big he is." This is TC's answer,


If a person wants to know how many tests per year are done, they need to see the number of tests per year that were done. You claim it is "meaningless" (which really means "meaningless to Mercutio"), but I wonder how you'd go about answering the question of how many tests per year were done without that number.

[CFLarsen]

If a person wants to know how many tests per year are done, they need to see the number of tests per year that were done. You claim it is "meaningless" (which really means "meaningless to Mercutio"), but I wonder how you'd go about answering the question of how many tests per year were done without that number.

But why do you need to know how many tests per year that are done?

To see the number? That's it? You just want to look at a number.

And you want JREF to pay for that?

This is insane.

[Mercutio]

It depends on what use, sure. The use in this case would be to get an understanding of the tests done by skeptical organizations that are statistical in nature. It may work for you to know that dowsing is the most tested claim, for example. Others need an actual number to increase their knowledge.
What sort of "understanding"? There are too many possible reasons that X would be tested more or less frequently in a self-selected sample. Knowing the raw numbers on this adds nothing.
If a person wants to know how many tests per year are done, they need to see the number of tests per year that were done. You claim it is "meaningless" (which really means "meaningless to Mercutio"), but I wonder how you'd go about answering the question of how many tests per year were done without that number.
You didn't understand. That's ok.

[H'ethetheth]

T'ai - if you think you're impressing the lurkers ... well you've failed abysmally in my case.
Second.

ETA: Though you did make me think about statistics, which is a good thing.

[Mercutio]

If you understand inferring, it is making a statement about the population based on data from the sample. The above is just looking at information from the sample.

Trends in the sample are not "utterly meaningless" as you erroneously claim. For example, the number of fatalities at a particular intersection from month to month are quite not "utterly meaningless".
Sorry, I missed this post before.

It depends, of course, on how (and "how" will depend on "why") the data were taken in the first place. These data are not the equivalent of "number of fatalities at a particular intersection"; as a self-selected sample, this is closer to "number of accidents you and your friends happened to see while passing a particular intersection over the course of a few months." If you noticed more, does that mean that there were more? No; it could easily be an artifact of the sampling process. You have no controls in place to assure random sampling, no controls in place to assure systematic operationalization of what constitutes "an accident". If you do find a "trend", it is meaningless, in that you have no way of knowing if it says something about the intersection or the observers!

If only some of the fatalities at that intersection are reported each month, and you have no control over what percentage are or are not reported, of what use is the "number of fatalities at a particular intersection"?

[Gr8wight]

Originally Posted by jj, not necessarily in this thread :
Um, let's look at this T'ai Chi chap's performance. First, he's had at least 4 nics, those being T'ai Chi, Whodini, JZS, and Statisticool. Second, he's reknowned for taking things out of context, malicious misunderstanding, outright misstatement of others positions, and the like. Third, he has nothing positive to say, and appears to emerge mostly to indirectly attack or incite other posters.

Wait! T'ai Chi is Justin? Why didn't you tell me earlier? And here I've gone and completely wasted all that time reading these threads. I started ignoring Justin months ago. I swear, people, isn't there some kind of rule about posting with multiple or serial identities? Well, there should be!

[CFLarsen]

Wait! T'ai Chi is Justin? Why didn't you tell me earlier? And here I've gone and completely wasted all that time reading these threads. I started ignoring Justin months ago. I swear, people, isn't there some kind of rule about posting with multiple or serial identities? Well, there should be!

There is.

When Justin tried to sign up under yet another userID, he was suspended for a week, and his posts merged together with his present userID.

[nathan]

Tai Chi,
you have not answered my question in response #238, namely why have you not discovered the %of female applicants yourself from the Challenge Applications forum?

Also, I gather you want to know the number of challenge tests per year. Why have you not determined this yourself, from the same forum?

[CFLarsen]

Tai Chi,
you have not answered my question in response #238, namely why have you not discovered the %of female applicants yourself from the Challenge Applications forum?

Also, I gather you want to know the number of challenge tests per year. Why have you not determined this yourself, from the same forum?
Because T'ai is not interested in the answer.

[T'ai Chi]

Mercy seems to be under the illusion that data cannot be used for purposes other than under which they were collected.

[drkitten]

Mercy seems to be under the illusion that data cannot be used for purposes other than under which they were collected.


No, he doesn't. Unless you totally lack any sort of understanding of statistics, and reading comprehension, and common sense.... oh, right. Carry on, then.

[Steven Howard]

Mercy seems to be under the illusion that data cannot be used for purposes other than under which they were collected.

It took you seventeen days to think of that response?

[Mercutio]

Mercy seems to be under the illusion that data cannot be used for purposes other than under which they were collected.Please show where I say that. (or is drkitten right?)

Do you suggest that biased data suddenly become unbiased when they are combined?

Remember your "fatalities at an intersection" example? I asked If only some of the fatalities at that intersection are reported each month, and you have no control over what percentage are or are not reported, of what use is the "number of fatalities at a particular intersection"?

Could you answer that? Take another 17 days if you need to.

[T'ai Chi]

Do you suggest that biased data suddenly become unbiased when they are combined?


You keep forgetting I am suggesting the possibility (http://www.statisticool.com/jrefchallengestats.htm) of looking into a meta-analysis ("One could also consider doing some type of meta-analysis on the data if appropriate."), among more important things like just seeing the actual data. I'm not stating that meta-analysis is definitely the way to go and it must be done no matter what. You have some odd need to pretend I am doing the latter.

Re: your undemonstrated claim that they are "biased". If they are, I'd suggest you talk to JREF since they're the ones designing biased tests according to you.


I asked [i]If only some of the fatalities at that intersection are reported each month, and you have no control over what percentage are or are not reported, of what use is the "number of fatalities at a particular intersection"?

I guess you'd have to ask the people who look at such data in real life for specifics, since that data is collected in real life (http://safety.fhwa.dot.gov/facts/stats2002/index.htm, http://www.ntf.se/english/pdf/0209Killed%20in%20Sweden.pdf, and many others), and they cannot control many things either.

But how about a simple example, for you- a number might hint that perhaps a change in the road, signals, or signs is in order.

[T'ai Chi]

Please show where I say that.


This


Numbers which are approprate for one use may be completely useless for another.


hints at it, but

(bold mine)


Because that is all the data are good for.


actually says it.

[Mercutio]

Re: your undemonstrated claim that they are "biased". If they are, I'd suggest you talk to JREF since they're the ones designing biased tests according to you.
Strawman. I did not ever say that the tests were inappropriate for their intended use. It is the further use of these tests which is where a bias emerges. I have repeatedly said that; it is dishonest of you to characterise my statement as you do.


I guess you'd have to ask the people who look at such data in real life for specifics, since that data is collected in real life, and many others), and they cannot control many things either.
I asked a hypothetical question. It is intended to make a point. Your link shows totals by state for a number of categories. Of far more interest are some of the links from your linked page. The statistics there are gathered for the purpose of looking at driver safety. As such, the categories and reporting techniques are exactly what you would expect in order to achieve that aim. They do not present a situation analogous to my hypothetical.
But how about a simple example, for you- a number might hint that perhaps a change in the road, signals, or signs is in order. "Might"? "Hint"? Yeah, that is pretty much the best you can do. The data on an unsafe intersection, if improperly collected, "might" seriously under-report the number of accidents. In comparison, then, another intersection "might" "hint" that it is worse than this first one, and get the sign or signal.

Your answer shows either that you do not understand sampling and experimental design, or that you are ignoring it for the sake of argument.

[Mercutio]

This

hints at it, but

(bold mine)

actually says it.
Gee...you don't link to my original post. Let's look at the words in context.

The data can answer these questions only about the small, self-selected sample itself. The data cannot tell us anything about the greater population, about which the questions are far more interesting.
You seem to be interested in individual results, but not a list of all the individual results. Who knows why.
Because that is all the data are good for. If I am interested in the questions you think are important, I would look for a sample which can answer them. This list cannot answer those questions.

You seem to be interested in asking questions of a database that is not designed to answer them, but not looking for a database that can. Who knows why.
What you "suspect" doesn't really matter.
Hey, I was only agreeing that my previous comment was a "No True Scotsman". Which would you prefer, that I assume you are interested, or that I assume you are not? It doesn't matter to me. If you are interested, there are better ways of answering the questions. If you are not, there is no reason to be asking them in the first place.

Ah...I see. You are taking my comments (or rather, you clip one sentence out of a larger comment) about this particular data set, and trying to make it look like I intended that comment to apply to all data sets. This goes back to your strawman from the previous post--I have said, and supported, that the tests are perfectly adequate for evaluating the claimants' claims, but are subject to a systematic bias when they are combined. This is what you have turned into a global statement of "Mercy seems to be under the illusion that data cannot be used for purposes other than under which they were collected." I am certain that now that your error has been pointed out, you will retract that statement.

And...in case you think, wrongly, that the combinational bias is the only problem, and wish to present the data for unspecified reasons in some large file, remember that the self-selection of subjects severely limits the utility of this sample. If you have genuine questions, there are better data sets to use. This one has done its job.

[T'ai Chi]

I did not ever say that the tests were inappropriate for their intended use. It is the further use of these tests which is where a bias emerges.


You have repeatedly said that, but you have not demonstrated that.

How could you? How could you possibly show that any other use of the data is biased as you are claiming?

And, again, you continue to ignore that I am suggesting the possibility (http://www.statisticool.com/jrefchallengestats.htm) of looking into a meta-analysis ("One could also consider doing some type of meta-analysis on the data if appropriate.") (underline "could", underline "consider", underline "possibility", and under "if appropriate" if that might help you understand) , among more important things like just seeing the actual data.

Why do you believe just seeing the actual data make bias magically emerge?


The statistics there are gathered for the purpose of looking at driver safety. As such, the categories and reporting techniques are exactly what you would expect in order to achieve that aim.


Um, so? So I showed a use of such data that you couldn't imagine any use for.

[T'ai Chi]

Let's look at the words in context.


It still doesn't change your erroneous belief that you repeat in your last post, that "..the further use of these tests which is where a bias emerges." That if the data is looked at other than intended, then it magically introduces bias. This is what you are desperately trying to argue.


This goes back to your strawman from the previous post--I have said, and supported, that the tests are perfectly adequate for evaluating the claimants' claims, but are subject to a systematic bias when they are combined.


What is your measure of this bias? Yes, numbers please.

And, yet again, you continue to ignore that I am suggesting the possibility of looking into a meta-analysis ("One could also consider doing some type of meta-analysis on the data if appropriate.") (underline "could", underline "consider", underline "possibility", and under "if appropriate" if that might help you understand) , among more important things like just seeing the actual data.

In other words, you keep droning on about combining, when I'm not even saying combining is the way to go. That is impossible to say without seeing actual data and getting a handle on specifics of the tests. Something which you are convinced you know everything about, sans data, apparently.


If you have genuine questions, there are better data sets to use. This one has done its job.

You've mentioned that already, but you still haven't shown that. If you are interested in other hypothetical data sets, then you are warmly encouraged to do some work and examine those. For those that want to examine data from tests from skeptical organizations, then examining data from tests from skeptical organizations is the way to go, Mercy.

[Mercutio]

You have repeatedly said that, but you have not demonstrated that.
Well, not to you, it seems. Others have understood it.

How could you? How could you possibly show that any other use of the data is biased as you are claiming?
I have not claimed that. I also have not claimed that this is the only problem. In my last post above, last paragraph, I mention the self-selection problem.

And, again, you continue to ignore that I am suggesting the possibility (http://www.statisticool.com/jrefchallengestats.htm) of looking into a meta-analysis ("One could also consider doing some type of meta-analysis on the data if appropriate.") (underline "could", underline "consider", underline "possibility", and under "if appropriate" if that might help you understand) , among more important things like just seeing the actual data.
And you are getting feedback on your suggestion. As for seeing the actual data, you have not given a reason to prefer a self-selected sample over a properly randomized one...if you can suggest a legitimate reason that this particular self-selected sample can answer a question you have, I would support it. So far, nothing.

Why do you believe just seeing the actual data make bias magically emerge?
The meta-analysis makes the first type of bias emerge. The self-selection of subjects results in the second type of bias. It does not have to emerge, it is there by the nature of the challenge. This is not a scientifically selected, random or representative sample. There is no requirement for it to be; it is the collection of people who accept the challenge. It is, necessarily, a bad sample to do descriptive statistics on.
Um, so? So I showed a use of such data that you couldn't imagine any use for.No, you just think you did. Did you click on the links from the page you linked? The actual data were not collected by self-report; the people collecting the data had much more control over what was collected. These are not at all analogous samplings.

[Mercutio]

It still doesn't change your erroneous belief that you repeat in your last post, that "..the further use of these tests which is where a bias emerges." That if the data is looked at other than intended, then it magically introduces bias. This is what you are desperately trying to argue.
No desperation here; the only one who does not see it, it seems, is you.

What is your measure of this bias? Yes, numbers please.
Is this finally something you are interested in? Might I suggest that you do some research on it and get back to us?

I have presented the bias; one other here even suggested an analysis that might account for it. Everyone here but you, it seems, understands it.

And, yet again, you continue to ignore that I am suggesting the possibility of looking into a meta-analysis ("One could also consider doing some type of meta-analysis on the data if appropriate.") (underline "could", underline "consider", underline "possibility", and under "if appropriate" if that might help you understand) , among more important things like just seeing the actual data.
I believe you said this above, and I responded there.

In other words, you keep droning on about combining, when I'm not even saying combining is the way to go. That is impossible to say without seeing actual data and getting a handle on specifics of the tests. Something which you are convinced you know everything about, sans data, apparently.
No. I have mentioned two distinct reasons for not using the data. One addresses combining, but the other (the self-selection bias) does not.
You've mentioned that already, but you still haven't shown that. If you are interested in other hypothetical data sets, then you are warmly encouraged to do some work and examine those. For those that want to examine data from tests from skeptical organizations, then examining data from tests from skeptical organizations is the way to go, Mercy.You have not mentioned any tests, any questions, for which this is the appropriate data base. As I said above, if you have a question for which this is the appropriate database, I would support it. I can't think of why you would want a self-selected sample to answer anything worthwhile, but if you can suggest something, go ahead. You have not done so yet. Oh, and "Mercy" really doesn't work, because of the different "c" sounds. I am sure if you use your imagination you can come up with something much better. Good luck.

[T'ai Chi]

Well, not to you, it seems. Others have understood it.


I'm only interested in your evidence (that you fail to show).


In my last post above, last paragraph, I mention the self-selection problem.


Again, that may be a problem if inference is in mind. As you've been told many times, inference is not in mind.


As for seeing the actual data, you have not given a reason to prefer a self-selected sample over a properly randomized one...if you can suggest a legitimate reason that this particular self-selected sample can answer a question you have, I would support it. So far, nothing.


In one of your early posts you ask the question of 'how many tests are statistical in nature?' That is one example of a question that could be answered. Are you now not interested in this question?


It is, necessarily, a bad sample to do descriptive statistics on.


Now that is being pretty dogmatic.

If inference were done, yes, that would be wrong. If descriptive statistics are done, that is not necessarily bad. In fact, descriptive statistics computed from such samples are done quite often.

I refer you to a book that you can check out and read (but which you probably won't and you'll ask me to read it to you): Sampling: Design and Analysis, Lohr, p. 8, second paragraph, titled What Good Are Samples with Selection Bias. She talks about the sample being valid unless the investigators wanted to generalize the estimates to a larger population - ie, they are doing descriptive and not inferential statistics (as I've been saying all along, eh?) She gives a specific example. Please, go read it.


The actual data were not collected by self-report;


Um, no kidding!, since in a fatality the people die, Mercutio. But all drivers are not assigned a number, and some selected to die and some not. That is what you are complaining about, the non-randomization, with your comment


As for seeing the actual data, you have not given a reason to prefer a self-selected sample over a properly randomized one...


Here is some more traffic fatality data:

http://www-fars.nhtsa.dot.gov/main.cfm
http://www.tsc.berkeley.edu/html/data.html
http://www.zzapp.org/rileygea/itsa/bkfat.htm
http://www.dot.wisconsin.gov/drivers/drivers/traffic/crash/fatality.htm

A lot of them look at trends even.

[T'ai Chi]

No desperation here; the only one who does not see it, it seems, is you.


You keep repeating your belief, but you haven't demonstrated it.

You have demonstrated you are incorrect when you say a such samples are "necessarily" bad. That is very, very, very, very, wrong.


I have presented the bias;


What is a numerical measurement of this bias?


I can't think of why you would want a self-selected sample to answer anything worthwhile,


Mercy, your lack of imagination is not a good argument.

[CFLarsen]

Good work, Merc.

[nathan]

Ah, you're back. Please answer my question. 'why have you not discovered the %of female applicants yourself from the Challenge Applications forum?' I asked you this in response #238 and response #273, yet you have not responded.

In response #218 you claimed
What % of the applicants have been female? Interesting question. Seems unnecessarily difficult to get a numeric answer.
I've told you where to get this information. Why are you incapable of determining this ratio for yourself from the challenge applications forum?

As I commented in #273, I gather you want to know the number of challenge tests per year. Why have you not determined this yourself, from the same forum?

[T'ai Chi]

Ah, you're back. Please answer my question. 'why have you not discovered the %of female applicants yourself from the Challenge Applications forum?' I asked you this in response #238 and response #273, yet you have not responded.

In response #218 you claimed

I've told you where to get this information. Why are you incapable of determining this ratio for yourself from the challenge applications forum?

As I commented in #273, I gather you want to know the number of challenge tests per year. Why have you not determined this yourself, from the same forum?

I'm sorry, how do you know what numbers I have and don't have?

As you've also been told, these numbers are not all the numbers we need to answer the question fully.

An obvious reason is that they don't cover multiple years, just one year.

Seeing trends from year to year could be desirable.

[Mercutio]

I'm only interested in your evidence (that you fail to show).

Again, that may be a problem if inference is in mind. As you've been told many times, inference is not in mind.
And as you have been told many times, that is not the only problem.
In one of your early posts you ask the question of 'how many tests are statistical in nature?' That is one example of a question that could be answered. Are you now not interested in this question?
That question applied to whether or not the data could be used for meta-analysis. If you are backing down from meta-analysis, what is your purpose for this information"
Now that is being pretty dogmatic.

If inference were done, yes, that would be wrong. If descriptive statistics are done, that is not necessarily bad. In fact, descriptive statistics computed from such samples are done quite often.
Yes, when people are interested in exploring that sample. These data have been collected with answering very specific questions in mind. We do not need to further describe these data; they have served their purpose. Unless and until you can come up with a legitimate question for descriptive analysis of these data, you are merely fishing.

I refer you to a book that you can check out and read (but which you probably won't and you'll ask me to read it to you): Sampling: Design and Analysis, Lohr, p. 8, second paragraph, titled What Good Are Samples with Selection Bias. She talks about the sample being valid unless the investigators wanted to generalize the estimates to a larger population - ie, they are doing descriptive and not inferential statistics (as I've been saying all along, eh?) She gives a specific example. Please, go read it.
I'll check if it is in our library. Meanwhile, it sounds like she is making exactly the point I have been. If you do not want to generalize the results to the larger population, you must have a question that is limited to that sample. So far, you have none. If you get numbers, you have no way at all of determining why those numbers are what they are, given that this is a self-selected sample.
Um, no kidding!, since in a fatality the people die, Mercutio. But all drivers are not assigned a number, and some selected to die and some not. That is what you are complaining about, the non-randomization, with your comment

A lot of them look at trends even.Do people volunteer to be in car accidents?

[T'ai Chi]

That question applied to whether or not the data could be used for meta-analysis.


Which I said might be appropriate, might not. Without looking more at data, who knows. Just dismissing it outright is not very open-minded.


These data have been collected with answering very specific questions in mind.


How is a question of 'How many tests were done per year?' not specific for you?


Unless and until you can come up with a legitimate question for descriptive analysis of these data,


Again, your lack of imagination is of no relevance.

[CFLarsen]

I'm sorry, how do you know what numbers I have and don't have?

As you've also been told, these numbers are not all the numbers we need to answer the question fully.

An obvious reason is that they don't cover multiple years, just one year.

Seeing trends from year to year could be desirable.

Perhaps. But you have been told where to find the available data. You were interested in this, and you know now where to find it.

Yet, you slink away again.

[Mercutio]

You keep repeating your belief, but you haven't demonstrated it.
The things I keep repeating have worked, every time but one. Again, my lack of imagination may be a poor excuse, but so far I can't seem to come up with something that this last person can comprehend. I am sure it must be my fault.

You have demonstrated you are incorrect when you say a such samples are "necessarily" bad. That is very, very, very, very, wrong.
Context, dearie. You are arguing both sides of this one. I have not said they are necessarily bad as a global statement, and have already told you that this is your strawman. They are perfectly adequate for asking the questions that they were designed to ask. (remember, you told me to take it up with the skeptic's organizations? Remember, that was your strawman?) They are necessarily bad when we use them for inference. Your own source agrees with that. Until you have a research question that is limited to the sample, I must conclude you are using them for inference. Certainly, the way you talk about using the data on your website implies that you are using them for inference.


What is a numerical measurement of this bias?
Don't know. No real need to answer this question, when better samples exist to answer questions about the population. The test tube is dirty. Do we spend time looking at how dirty it is, or do we run our tests with a clean test tube?


Mercy, your lack of imagination is not a good argument.

It is not an argument. I have repeatedly asked you for examples. My ignorance of proper questions to answer with this dataset is simply that. You could cure it, and I have (as we often do when we are ignorant of something) asked someone whose actions suggest he might have some answers. So far it seems like you are as unable as I am to come up with legitimate reasons to look at those data.

[nathan]

I'm sorry, how do you know what numbers I have and don't have?
I'm making the assumption that you're capable of reading the challenge application forum and extracting the numbers from it. Perhaps that is beyond you.

As you've also been told, these numbers are not all the numbers we need to answer the question fully. An obvious reason is that they don't cover multiple years, just one year.
Pray tell me how finding the % of female applicants from the data available is not a desirable thing, if you want to know the % of female applicants? Obviously it will be from a subsampling of the entire cohort of applicants, but surely with your knowledge of statistics you could extrapolate that :)

Seeing trends from year to year could be desirable.
This is an additional question you did not pose to begin with. Methinks you just added this now to avoid doing the work of finding the answer yourself.

[Mercutio]

I'm sorry, how do you know what numbers I have and don't have?

As you've also been told, these numbers are not all the numbers we need to answer the question fully.

An obvious reason is that they don't cover multiple years, just one year.

Seeing trends from year to year could be desirable.With a self-selected sample, what would the trend from year to year be able to tell you?

What would it be able to tell you that isn't better answered with a different, systematically collected, data set?

[Ripley Twenty-Nine]

T'ai Chi,

Would it be possible for you to post for us the following information concerning your post history? :

Date/Time of Post, Username used to Post with, JREF Board Posted in, Subject Category of Post, JREF User Replied To, and length (In characters) of post?

I think this would be very, very interesting information to have.

Thanks!

[CFLarsen]

I'm making the assumption that you're capable of reading the challenge application forum and extracting the numbers from it. Perhaps that is beyond you.

...

This is an additional question you did not pose to begin with. Methinks you just added this now to avoid doing the work of finding the answer yourself.

You answered it yourself: T'ai Chi wants other people to do the hard work for him.

He has done it before, when he wanted other people to find transcripts of psychic readings, only to make all sorts of tallying. Some of that tallying included gender - but no reference to year-by-year. Then, it was OK to merely count gender. Now, it isn't.

He is simply inventing a new excuse not to do any work himself. He would rather have other people do it for him, and then scold them, if they won't dance to his pipe.

Strangely enough, he has removed that "study" from his website, and has refused to explain why. Fortunately, I have a copy. Available on request.

[Gr8wight]

Please allow me to ask my question from several weeks ago one more time.

Because we don't have 20 dowsing experiments done simlarly. We have one dowsing experiment finding gold under a cup, one dowsing experiment finding addresses with a pendulum, one telepathy experiment sending thoughts to another person, one martial arts experiment attempting to stop an attacker without touching him... How do you combine those results?

By the way, can you tell me what element all of those tests had in common?

Justin (I won't play games by using your sock puppet name),

I don't mean to harp on this point, but it lies at the very basis of this discussion. Mercutio, CFLarsen, and others have tried to explain to you why data from many different tests cannot be combined into any meaningful meta analysis, but you ignore their knowledgeable attempts to help. My question is important because it involves the only piece of data that encompasses all of the preliminary tests. It can be stated as a simple percentage, which is easy for everyone - even you - to understand.

Have a go, won't you? If you can't come up with the answer just say so, and I'll explain it all using real easy words. I promise.

[T'ai Chi]

This is an additional question you did not pose to begin with. Methinks you just added this now to avoid doing the work of finding the answer yourself.

From my page (http://www.statisticool.com/jrefchallengestats.htm)
(bold added)


The JREF has been making strides in making information on "the claims received, the correspondences exchanged between the JREF and the applicant, and subsequent protocol negotiations and test results" electronic, but not, as far as I can tell, the numerical results of past preliminary tests, which in my opinion are just as, if not more, interesting and relate more to the science.


I've always been interested in data from the past as well.

So, again, why do you assume I did not calculate what you are demanding? And why do you think what you think I calculated or not matters to proposing the idea of seeing interesting data?

[T'ai Chi]

They are necessarily bad when we use them for inference.


Who is doing inference?

You argue against doing that, and I agree.


Until you have a research question that is limited to the sample, I must conclude you are using them for inference.


That is incredibly flawed logic. I have not talked about koala bears, therefore you must conclude I am talking about koala bears.

Your demonstrated (ill)ogic is about as pertinant as your lack of imagination.

[T'ai Chi]

T'ai Chi,

Would it be possible for you to post for us the following information concerning your post history? :

Date/Time of Post, Username used to Post with, JREF Board Posted in, Subject Category of Post, JREF User Replied To, and length (In characters) of post?

I think this would be very, very interesting information to have.

Thanks!

Thanks for sharin'.

[Mercutio]

from your page:the numerical results of past preliminary tests, which in my opinion are just as, if not more, interesting and relate more to the science. Relate how? What can this self-selected sample tell you about "the science"? Please, I lack imagination; tell me.

[T'ai Chi]

I don't mean to harp on this point, but it lies at the very basis of this discussion. Mercutio, CFLarsen, and others have tried to explain to you why data from many different tests cannot be combined into any meaningful meta analysis, but you ignore their knowledgeable attempts to help.


It was already explained, ad nausem, that
(bold added)


One could also consider doing some type of meta-analysis on the data if appropriate.


That is, as explained, many times, I'm not saying meta-analysis is the way to go no questions asked.

The main point, as also pointed out several times, is to see some interesting data that skeptics should be interested in seeing.

[T'ai Chi]

from your page: Relate how? What can this self-selected sample tell you about "the science"? Please, I lack imagination; tell me.

I'm not on trial here. Your demands are ignored.

If you aren't willing to think how test results relate more to science than Kramer's emails, I cannot help you.

[T'ai Chi]

With a self-selected sample, what would the trend from year to year be able to tell you?


It all depends on what the specific data is. Obviosuly, something like the number of tests done of dowsing per year would tell us the number of dowsing tests that were done per year, and if that number is increasing or not.

You like to argue in hypotheticals, which are easily picked apart (no matter if the hypotheticals are yours or mine). I don't see the point of such avoiding-data-at-all-costs inquiries.


What would it be able to tell you that isn't better answered with a different, systematically collected, data set?

If you're interested in hypothetical as yet to be collected datasets, you're welcome to investigate those.

[Mercutio]

I'm not on trial here. Your demands are ignored.
It's a request, not a demand. I even said please. I have said before, and I mean it, if you can show a legitimate use for these data I would gladly support your request. You may (and will, likely) ignore my request, but frankly you act against your own interests in doing so. I honestly think that the data are inappropriate for any use beyond their challenge purpose (for reasons of self-selection, and of the bias that may emerge in combination, as previously explained); if you honestly think they are of use, I have no clue why you would not say how. Claus dismisses you out of hand; I am more patient. I don't care what I might think of you, if you have good ideas, I am for them. If you do not share them, I cannot support them. Your loss.

If you aren't willing to think how test results relate more to science than Kramer's emails, I cannot help you.Willing to think? Certainly. Willing to suspend what I know about biased samples? Sorry, no. Thinking about how test results relate means critically examining the possibilities. If the data don't work for a particular purpose, it is better to obtain good data than to argue about bad data.

[Mercutio]

It all depends on what the specific data is. Obviosuly, something like the number of tests done of dowsing per year would tell us the number of dowsing tests that were done per year, and if that number is increasing or not.
But of course, that number is meaningless. It does not tell us if there are more dowsers, because we simply cannot know if the reasons for self-selection were constant, or whether there was some unrelated reason that there were more dowsing challenges. In fact, if we picked up an apparent trend that the dowsing challenges were increasing, it would tell us nothing; is it a representative sample? We simply cannot know--not only why there were more dowsers--but even whether there were more dowsers, in anything but a self-selected sample. An ad in a dowsing website could easily throw off any claims of "trend", and no one has control over that.

You like to argue in hypotheticals, which are easily picked apart (no matter if the hypotheticals are yours or mine). I don't see the point of such avoiding-data-at-all-costs inquiries.

If you're interested in hypothetical as yet to be collected datasets, you're welcome to investigate those. Well, if I had a question to answer, I would have a much better idea of how to collect the data. That is pretty much the whole point of the "hypothetical as yet to be collected datasets"; they are collected to answer a particular question. Obviously, you have some reason to keep looking at the data which have already been collected, and I cannot fathom why. What question can be answered? You will not help with that, and I ...apparently...lack imagination.

[Ripley Twenty-Nine]

Thanks for sharin'.
So you're not going to do this for me? Why not?

[T'ai Chi]

It's a request, not a demand. I even said please.


But I'm not on trial here. I find your repeated questioning without a point tiresome and I will not tolerate it.


I have said before, and I mean it, if you can show a legitimate use for these data I would gladly support your request.


Your support or lack of is of no relevance to me.


Willing to suspend what I know about biased samples? Sorry, no.

Just like you're not willing to suspend what you know, I'm not willing to suspend what I know (Lohr and other resources and examples).

We're at a 'we both disagree with each other' place. Yup.

[T'ai Chi]

So you're not going to do this for me? Why not?

I'm not on trial to answer your questions.

I suggest you try a more productive approach.

[T'ai Chi]

It does not tell us if there are more dowsers, because we simply cannot know if the reasons for self-selection were constant, or whether there was some unrelated reason that there were more dowsing challenges.


Of course it can tell you the basic fact if there were more dowsers tested in a certain year.


is it a representative sample?


This has been addressed plenty of times.


An ad in a dowsing website could easily throw off any claims of "trend", and no one has control over that.


And? No one can control the stock market, weather, and many other things that people get descriptive statistics on.


Obviously, you have some reason to keep looking at the data which have already been collected, and I cannot fathom why. What question can be answered?


This has been addressed plenty of times.

[Gr8wight]

It was already explained, ad nausem, that
(bold added)



That is, as explained, many times, I'm not saying meta-analysis is the way to go no questions asked.

The main point, as also pointed out several times, is to see some interesting data that skeptics should be interested in seeing.

And, as is your M.O., you have completely ignored the question that was put to you. What one thing do every single preliminary test ever conducted by the JREF have in common?

[Mercutio]

But I'm not on trial here. I find your repeated questioning without a point tiresome and I will not tolerate it.

Your support or lack of is of no relevance to me.

Just like you're not willing to suspend what you know, I'm not willing to suspend what I know (Lohr and other resources and examples).

We're at a 'we both disagree with each other' place. Yup.I am sorry that you see scientific inquiry as spanish inquisition. You see no point? Funny, I thought I was the one with no imagination.

Fine, agree to disagree. The offer is open. If you can produce questions for this dataset, do so. If you do not wish to contribute to a scientific community, then "agreeing to disagree" is as good an excuse as any.

[Mercutio]

Of course it can tell you the basic fact if there were more dowsers tested in a certain year.
This has been addressed. A faulty answer is as bad, or worse, than no answer. What does the self-selected sample give you that a proper sample would not? There are, of course, many things that a proper sample would give you that a self-selected sample will not. Even your level of understanding of statistics and research methods will show that.
This has been addressed plenty of times.
Yes. It is not a representative sample.
And? No one can control the stock market, weather, and many other things that people get descriptive statistics on.
The data we collect on those are intended for different purposes. And are collected with those purposes in mind. Unless you wish to suggest that a self-selected "reportyourweather.com" will be a better source of data than systematically collected data, these examples are useless for your argument.
This has been addressed plenty of times.So far, the answer is "no, there is no reason to look at these data."

[T'ai Chi]

What one thing do every single preliminary test ever conducted by the JREF have in common?

I'm not interested in every single test, only the ones that are statistical in nature, and not just JREF, but any skeptical organization that does similar tests.

Surely, if there are tests that are statistical in nature, on dowsing, it seems reasonable to look into the possibility of combining their data. And if not appropriate, then no biggie, don't do it. But seeing the data, and calculating descriptive statistics are what is important.

[T'ai Chi]

The offer is open.


Your "offer" is not genuine and not of interest.

[T'ai Chi]

(snip)


All of this was previously addressed.

[SixSixSix]

Your "offer" is not genuine and not of interest.
I'm curious - does this sort of "screw you, answer my questions!" approach work for you normally? It seems a very strange way to elicit answers.

[Mercutio]

Your "offer" is not genuine and not of interest.
You can read minds? Will you be applying, yourself?

[T'ai Chi]

You can read minds? Will you be applying, yourself?

It is my opinion your offer is not genuine.

And it is a fact that it doesn't interest me.

[CFLarsen]

I'm not on trial to answer your questions.

I suggest you try a more productive approach.

It is my opinion your offer is not genuine.

And it is a fact that it doesn't interest me.

O....K.

Could you explain to us what a more productive approach would be?

How can we persuade you that our offers are genuine?

What, exactly, are you interested in discussing?

(Could someone re-post these questions? T'ai Chi pretends to have me on ignore. A couple of times should do it.)

[nathan]

So, again, why do you assume I did not calculate what you are demanding? And why do you think what you think I calculated or not matters to proposing the idea of seeing interesting data?
Are you now claiming that you do have the % of female applicants? Why could you not simply have said 'I have this number', rather than going round the houses trying to avoid answering my question about why you didn't work this out for yourself?

You originally said:
What % of the applicants have been female? Interesting question. Seems unnecessarily difficult to get a numeric answer.

*seems*, not *seemed*

Use of the present tense indicates you were trying and failing to determine this number. Have you worked this number out between then and now? Why didn't you say so?

So, TaiChi
1) Do you have the % of female applicants?
2) If not, why have you not worked it out from the information available?

You're currently playing a game of 'guess what numbers I have'. Why don't you just tell us?

Oh, yes, because you're just trying to be argumentative.

[Mercutio]

I'm not on trial to answer your questions.

I suggest you try a more productive approach.

It is my opinion your offer is not genuine.

And it is a fact that it doesn't interest me.

O....K.

Could you explain to us what a more productive approach would be?

How can we persuade you that our offers are genuine?

What, exactly, are you interested in discussing?

(Could someone re-post these questions? T'ai Chi pretends to have me on ignore. A couple of times should do it.) *sigh*

[Mercutio]

It is my opinion your offer is not genuine.
That's the nice thing about opinions. They don't have to have any relation to reality.

And it is a fact that it doesn't interest me.
Just curious: is there anything at all that you will go on record as actually being interested in? Enough, say, to actually engage in 2-way conversation about it, giving your own opinions as well as soliciting and reacting to those of others? Just curious...

[CFLarsen]

Thanks.

[Ripley Twenty-Nine]

I'm not on trial to answer your questions.

I suggest you try a more productive approach.
But I asked so nicely.

All I'm doing is asking you for the same type of information that you are asking the JREF to provide. You want the JREF to compile all of this data for you, and yet you won't do the same thing when asked. How is this any different?

[Gr8wight]

I'm not interested in every single test, only the ones that are statistical in nature, and not just JREF, but any skeptical organization that does similar tests.

Surely, if there are tests that are statistical in nature, on dowsing, it seems reasonable to look into the possibility of combining their data. And if not appropriate, then no biggie, don't do it. But seeing the data, and calculating descriptive statistics are what is important.

I'm prepared to quote a hard statistic to you, Justin. Are you prepared to receive it?

[CFLarsen]

I'm prepared to quote a hard statistic to you, Justin. Are you prepared to receive it?
I'd like to see it.

[Gr8wight]

I'm prepared to quote a hard statistic to you, Justin. Are you prepared to receive it?

Justin has posted in four different threads since I posted this message, but has not posted here again. I wonder why that is.

[T'ai Chi]

I'm prepared to quote a hard statistic to you, Justin. Are you prepared to receive it?

"Gr8wight",

You are free to post anything you'd like, as you've always been.

[T'ai Chi]

Justin has posted in four different threads since I posted this message, but has not posted here again. I wonder why that is.

"Gr8wight",

Perhaps because I'm not hanging on your every word like you apparently are on mine?

[EHocking]

<snip>
You've mentioned that already, but you still haven't shown that. If you are interested in other hypothetical data sets, then you are warmly encouraged to do some work and examine those. For those that want to examine data from tests from skeptical organizations, then examining data from tests from skeptical organizations is the way to go, Mercy.So have you begun to collate the 18 months worth of 141 applications to the JREF Challenge in order to get an idea of the type of data you are proposing would be interesting to analyse?

[Gr8wight]

"Gr8wight",

You are free to post anything you'd like, as you've always been.

You're the one who started this thread claiming an interest in statistics about the JREF and other paranormal challenges. Now you feign disinterest in the only hard statistic that can confidently be quoted for the JREF challenge. What, exactly is your position, Justin? Do you want to take a stab at what that statistic is?

[SixSixSix]

Now you feign disinterest in the only hard statistic that can confidently be quoted for the JREF challenge.
Engage Pedantic Mode:
Technically, you want "uninterest" rather than "disinterest" there.
Pedantic Mode disengaged.

;)

[Gr8wight]

Engage Pedantic Mode:
Technically, you want "uninterest" rather than "disinterest" there.
Pedantic Mode disengaged.

;)

Dear SixSixSix,

Thank you for your comment. I had to think about it for a minute, but I understand why the distinction is important. Justin is feigning uninterest. But then again, his beginning was nothing more than feigned interest. Justin is a big, big troll.

Of course, I already knew that. I'm not sure why I continued posting here after I discovered that T'ai Chi was really Justin employing a new, sock-puppet identity. Oh, I remember. I wanted to make sure that everyone else reading the thread was clear that T'ai Chi was really the well known troll Justin employing a new sock-puppet identity.

[SixSixSix]

If you get a chance, check out the movie "Wilt". It's a 80s or 90s British comedy starring Mel Smith and Griff Rhys Jones (of "Alas Smith and Jones" fame, if you guys get British comedy series over there); there's a line in the movie that makes clear the difference between disinterest and uninterest.

I was of course being needlessly pedantic, and it wasn't as if what you were saying was unclear. Kiless hates me for correcting her on things like that; possibly I should take that to heart. ;)

[Gr8wight]

It has now been five days since Justin last posted in this thread. A thread that he started, and in which he has averaged more than ten posts per page over the 9 pages. Justin, who seemed so interested in a statistical analysis of JREF challenge tests suddenly seems not to want to hear me quote the one piece of hard data that does cover all the tests ever conducted, regardless of their nature. What are you hiding from, Justin?

It's really too bad, isn't it, Justin, that with Kramer gone, and Randi away from the office, there aren't any new claims to discuss. Which means there aren't any new threads being posted to this forum. Which means this thread, which you seem to be so embarrassed about, just won't get pushed off the front page, darn it all!

See you tomorrow, Justin.

[EHocking]

I'm not on trial here. Your demands are ignored.

If you aren't willing to think how test results relate more to science than Kramer's emails, I cannot help you.So have you even attempted to review the Challenge entry emails to determine if you CAN derive any interesting statistics from them?

You've stated that you would find this interesting, yet you've not even posed how, or indeed if, you'd go about it.

Since you're the one who seems to think these statistics are "cool", why have you not investigated further and shared your investigation with the forum.

[EHocking]

Since the OP of this thread doesn't seem to have any interest in his own website content, here's my first pass on the applications that Kramer had worked on and posted at JREF.

The following is a first pass review of the 143 claims listed at the Challenge Applications Forum (http://forums.randi.org/forumdisplay.php?f=43) Some will disagree with my application categorisation and no doubt my interpretation of the result, but I have been as objective as possible in the data gathering process.

Category|No|Male|App'ns|App'n Corr'n|App'n Corr'n|Accepted|Protocols|Protocols|Examiner|Test Date|Tested|Failed
||||/Affidavit (N)|/Affidavit (Y)||Begun|Not Begun|Agreed|Not Agreed|Found|Agreed||
Telepath|24|19|20|||12|8|4|2|3|5|2|2|2
Dowser|17|14|16|||13|4|7|3||8|3|3|3
Telekinetic|16|14|13|2||9|4|3|1|1|3|||
Energy|11|9|11|2||7|2|5|1|4|4|1|1|1
Healer|11|9|10|5|2|6|3|4|1|3|3|1|1|1
Psychic|11|7|7||1|7|3|5|2|3|5|2||
Predictions|9|7|6|3||5|4||||1|||
Gambling|5|4|4|||2||1||1|1|||
Theoreticist|5|5|4|||||||||||
Device|4|4|4|3||2||2|1|1|1|1|1|1
UFO|4|4|2|1||2||3||3|1|1||
Homeopath|3|2|3|5||2||2||1|2|||
Medium|3|1|2||1|2||2|1|1|2|||
Remote Viewer|3|2|2|||||||||||
Clairvoyant|2|1|1|||1|1|||||||
Ghosts|2|2|1|1||1||1||1||||
Humour|2|2||||||||||||
Perpetual Motion|2|2|2|||2|1|||1||||
Astrologer|1|1|1|||1||1||1|1|||
Bible Code|1|1|1|4|1|||||||||
Dowser - Device|1|1||3|1|||||||||
EVP|1|1|1|3||1||||||||
Exorcist|1|1|1|||||||||||
Illusion|1|1|1|7|2|1||1||1||||
Numerology|1|1|1|10|3|||||||||
Painter|1|1||3||||||||||
Suicide|1|1|1||2|||||||||
Totals|143|117|115|52|13|76|30|41|12|25|37|11|8|8
% of Total Claims||82%|80%|36%|9%|53%|21%|29%|8%|17%|26%|8%|6 %|6%
% of Application||||45%|11%|66%|26%|36%|10%|22%|32%|10% |7%|7%| |

Some notes on the table above.

1. The App'n Corr'n / Affidavit columns indicate the response to requests to correctly submit an application or to provide affidavits when asked by JREF.2. Protocols discuss whether a discussion was started on test protocols and whether they agreed
3. Table content is listed by number of applications per categoryThe categories are as per the applicants claim, or my determination of it from the subsequent correspondence, so may not be exactly as some others may interpret them. Below is the logic on those categories that some may lump together:



Telepath : Quite literally a mind reader or thought projectionPsychic : Telepath that doesn't deal with mind pictures, more emotional responses, or guessing cards etc.
Medium : Talks to the dead
Clairvoyant : Divination of personal history or future events
Energy : Chi, TT where no healing is boasted

DISCUSSION

45% of the 115 applications did not respond to requests to a. submit the application correctly, b. to exactly define their claim, or c.supply affidavits.

Of the 66% (76/115) of claims that were accepted, 39% (30/76) actually got to the point of negotiating test protocols, but only 16% (12/76) finalised the protocol negotiation. This is despite the fact that JREF found external examiners that were willing to negotiate a test in 48% (37/76) of the claims.

Only 8 of the 12 who agreed a test protocol went on to be tested, even though 11 of the cases agreed a test date.

All 8 who were tested failed.

[petre]

Since the OP of this thread doesn't seem to have any interest in his own website content, here's my first pass on the applications that Kramer had worked on and posted at JREF.
[snip]


Wait, how on earth did you compile those statistics without creating a 9-page thread? You mean, you just looked at the data and compiled it without complaining that others wouldn't do that for you? Astounding!

[CFLarsen]

Wait, how on earth did you compile those statistics without creating a 9-page thread? You mean, you just looked at the data and compiled it without complaining that others wouldn't do that for you? Astounding!
I don't think someone will start bitching "the analysis is flawed, but I'm not going to do it better myself".

No, no.

[Ersby]

Good work, EHocking.

[EHocking]

Wait, how on earth did you compile those statistics without creating a 9-page thread? You mean, you just looked at the data and compiled it without complaining that others wouldn't do that for you? Astounding!I seem to recall a favourite response by a certain poster, "I'm not going to do your homework for you." Pity he won't even do his own.

[EHocking]

I don't think someone will start bitching "the analysis is flawed, but I'm not going to do it better myself".

No, no.Now, Now. Anyone would think you are getting cynical.

[EHocking]

Good work, EHocking.It was quite interesting to look back on the work that Kramer did for JREF and what he had to put up with.

Odd, there seems to be a poster missing from this last part of the discussion.

[William Smith]

Nice work, EHocking.

[T'ai Chi]

..., here's my first pass on the applications that Kramer had worked on and posted at JREF.


Nice!

..but what part of


The JREF has been making strides in making information on "the claims received, the correspondences exchanged between the JREF and the applicant, and subsequent protocol negotiations and test results" electronic, but not, as far as I can tell, the numerical results of past preliminary tests, which in my opinion are just as, if not more, interesting and relate more to the science.


are you not getting? :)

You reviewed the 143 claims listed (which I already acknowledged as being available). You did not look at any data besides that forum data, data which is not easily available in the same manner. A major point you're missing here, is that I'm only interested in tests that are statistical in nature, not all tests. This is so that probabilities of outcomes can be talked about.

But I think you have the right idea, and have shown that such statistics in general are useful to aid in understanding.

[Mercutio]

But I think you have the right idea, and have shown that such statistics in general are useful to aid in understanding.
What do they help you to understand? Anything in particular, TC?

[CFLarsen]

Nice!

..but what part of



are you not getting? :)

You reviewed the 143 claims listed (which I already acknowledged as being available). You did not look at any data besides that forum data, data which is not easily available in the same manner. A major point you're missing here, is that I'm only interested in tests that are statistical in nature, not all tests. This is so that probabilities of outcomes can be talked about.

But I think you have the right idea, and have shown that such statistics in general are useful to aid in understanding.


I don't think someone will start bitching "the analysis is flawed, but I'm not going to do it better myself".

No, no.

OK, I admit it: I was wrong.

[EHocking]

Nice!

..but what part of


The JREF has been making strides in making information on "the claims received, the correspondences exchanged between the JREF and the applicant, and subsequent protocol negotiations and test results" electronic, but not, as far as I can tell, the numerical results of past preliminary tests, which in my opinion are just as, if not more, interesting and relate more to the science.

are you not getting? :)

You reviewed the 143 claims listed (which I already acknowledged as being available). You did not look at any data besides that forum data, data which is not easily available in the same manner. A major point you're missing here, is that I'm only interested in tests that are statistical in nature, not all tests. This is so that probabilities of outcomes can be talked about.

But I think you have the right idea, and have shown that such statistics in general are useful to aid in understanding.There are 12 claims where protocols were agreed. 8 of these were tested.

You initiated this thread - which of these 12 preliminary tests are statistical in nature?

Until you actually make an effort to review the subset of the data that you propose would be interesting to analyse, to get an idea of the effort involved and to determine if there *are* any tests that are of a statistical nature, then your snide and condescending criticisms of my effort come across merely as childish and petty.

(ETA: grammar)

[Ersby]

your snide and condescending criticisms of my effort come across merely as childish and petty.

Ah, but he used a smiley face. People who are "open minded" are allowed to be patronising and offensive if they use smiley faces.

[T'ai Chi]

You initiated this thread - which of these 12 preliminary tests are statistical in nature?

Until you actually make an
(snip)


I did initiate the thread, that is a fact. You believing that therefore that means I must do certain things doesn't make much sense.


, then your snide and condescending criticisms of my effort come across merely as childish and petty.

I pointed out some obvious things about it. I suggest you don't take it so personal.

[CFLarsen]

You never change, do you?

[T'ai Chi]

People who are "open minded" are allowed to be patronising and offensive if they use smiley faces.

Sweet!

BTW


At the end of my research I find a hit rate of between 28.6% and 28.9% depending on certain choices concerning which scoring methods to use on particular experiments. This doesn't have quite the headline grabbing appeal of 1 in 3 instead of 1 in 4 but the hit rate is still highly significant for 6,700 sessions.


What is your open-minded conclusion from this?

[Ersby]

What is your open-minded conclusion from this?

That you took one paragraph out of a larger body of text.

If you can explain to me about funnel graphs and also address the four points I make at the end of the article, then I'm here to learn.

btw, what is your open-minded conclusion?

[EHocking]

I did initiate the thread, that is a fact. You believing that therefore that means I must do certain things doesn't make much sense.Incorrect - this is a discussion forum. We discuss things here. You initiated a discussion, but don't wish to participate. Says a lot more about you than me, my friend.
I pointed out some obvious things about it. I suggest you don't take it so personal.Rule 8 prevents me from suggesting what you can do.

[EHocking]

Ah, but he used a smiley face. People who are "open minded" are allowed to be patronising and offensive if they use smiley faces.A standard practice by usenet kooks, trolls and cowards. For me it merely indicates the poster is insincere in their post. They *really* want to give a person "the bird" but lack the courage.

[T'ai Chi]

If you can explain to me about funnel graphs..


I'd hope you already know about funnel graphs if you wrote about funnel graphs. :)

I'd suggest instead of visually assessing a funnel plot, to do a statistical test to assess the assymetry.

What the funnel plot shows from what I can see from the published studies, is that the effect fluctuates around 0 with a lot of variation and no consistency. But the psi proponents already say as much. :)

[T'ai Chi]

They *really* want to give a person "the bird" but lack the courage.

The person who wants the bird might not be worth the time and effort.

Much more rational to toss them in the Ignore list. :)

[EHocking]

The person who wants the bird might not be worth the time and effort.

Much more rational to toss them in the Ignore list. :)And here's me thinking that you thought statistics are cool and might even discuss the results of the tabled data.

Instead, I get the second threat, today, from you, to ignore me.

[Ersby]

What the funnel plot shows from what I can see from the published studies, is that the effect fluctuates around 0 with a lot of variation and no consistency. But the psi proponents already say as much.

The problem with the "capricious, actively evasive nature of psi" (I assume that's what you're referring to) is that it's indistinguishable from "no effect being measured".

[T'ai Chi]

The problem with the "capricious, actively evasive nature of psi" (I assume that's what you're referring to) is that it's indistinguishable from "no effect being measured".

Perhaps.

In same cases if there is a super tiny p-value (what do we have here, 1 x 10^-12 or something?) it is difficult to say without better designed studies.

[T'ai Chi]

And here's me thinking that you thought statistics are cool and might even discuss the results of the tabled data.


I already discussed what I thought of your results.

[Mercutio]

Perhaps.

In same cases if there is a super tiny p-value (what do we have here, 1 x 10^-12 or something?) it is difficult to say without better designed studies.
Given the data here, it is not difficult to say. It is impossible to say.

What is it that you find helpful about these data? You have not answered that. What do they help you to understand?

[Ersby]

Given the data here, it is not difficult to say. It is impossible to say.

What is it that you find helpful about these data? You have not answered that. What do they help you to understand?

Which data are you referring to? EHocking's or my ganzfeld stuff?

[T'ai Chi]

What is it that you find helpful about these data? You have not answered that. What do they help you to understand?

This has already been addressed when I talked about descriptive statistics.

[petre]

Perhaps.

In same cases if there is a super tiny p-value (what do we have here, 1 x 10^-12 or something?) it is difficult to say without better designed studies.

So from this, can we assume that you've finally accepted JREF tests are not suited to the purpose you desire, and as such you now look elsewhere?

[T'ai Chi]

So from this, can we assume that you've finally accepted JREF tests are not suited to the purpose you desire, and as such you now look elsewhere?

You, as before, are free to assume what you'd like.

I'm not sure how the above follows.

[Mercutio]

Which data are you referring to? EHocking's or my ganzfeld stuff?
The former. If the conversation was referring to yours, I retract my comment, as I am unfamiliar with your dataset.

[Mercutio]

This has already been addressed when I talked about descriptive statistics.
Well, yes and no. It still sounded like you wanted to draw inferences to a larger population. Now is when you can demonstrate what it is you really meant, and show I am wrong.

All I can see is a collection of data that would seriously tempt one to ask meaningless questions. For instance, "why are there more X than Y?", when of course the sample self-selection renders that a meaningless question. We have absolutely no control over the representativeness of the sample, no control over sampling variability--any descriptive statistics could not even be said to represent the population of JREF applicants, since we have no reason to believe that the next set of self-selected applicants need follow these percentages!

So, sorry if you think you already addressed the question, but I will ask it again (I hope you can see why): What is it that you find helpful about these data? You have not answered that. What do they help you to understand?

[CFLarsen]

btw, what is your open-minded conclusion?

Note that T'ai Chi didn't answer this question.

[T'ai Chi]

Well, yes and no.
(snip)


You've already been answered.

[Mercutio]

You've already been answered.
You did write something, yes.

This situation is different. Your "answer" was in the abstract. Now, we have a concrete example. Perhaps my dissatisfaction with your response was due to misunderstanding you in the abstract.

I still do not see any use for these data, nor any reason to compile them together. I would welcome your example showing I am wrong. Saying "you've already been answered" is not advancing anything.

[CFLarsen]

In a word: No.

[T'ai Chi]

(snip)


You've been answered.

[Gr8wight]

You've been answered.

Translation: "I'm not going to answer you."

[Mercutio]

You've been answered.
No, TC, I have not. I have been rudely dismissed.

I have explained, twice now, why your previous response is not an answer to the current question. I took the time to read what you have written, and I have asked legitimate questions about it. I have taken the time to give you answers to the questions you have asked, when you have asked them. I cannot interpret your refusal to answer as a reaction to my rudeness, because I have not been rude. I have simply persisted in asking legitimate questions for which you have not yet given answers. If persistence offends you, there is a cure: answer the question the first time.

If you truly do not understand why your response is not an answer to my question after both of my explanations, please ask me or someone else to explain it to you more simply. Do not merely dismiss it out of hand. And if you do not want to answer it, just say so, and don't pretend it has already been answered.

[T'ai Chi]

No, TC, I have not. I have been rudely dismissed.


You've been answered and dismissed.

[CFLarsen]

You've been answered and dismissed.
And you are a liar. As evidenced in this thread.

[Mercutio]

You've been answered and dismissed.
Well, this time you are at least half right.

Does anyone else reading this thread, who might have understood TC's "answer" better than I did, want to take a stab at applying it to the concrete example we now have to work with?

[petre]

Perhaps.

In same cases if there is a super tiny p-value (what do we have here, 1 x 10^-12 or something?) it is difficult to say without better designed studies.

So from this, can we assume that you've finally accepted JREF tests are not suited to the purpose you desire, and as such you now look elsewhere?

You, as before, are free to assume what you'd like.

I'm not sure how the above follows.

Well, when you said "here", perhaps I was assuming too much when I guessed that you were refering to the offered statistics on JREF tests. You appeared to be commenting that something was "difficult to say" without "better designed studies". Therefore, I was asking if you were now going to depart in search of those "better designed studies" that would be less "difficult to say" whatever it is you wish to say about whatever it is you are actually interested in.

[EHocking]

I got a PM today asking me if I kept any notes on what I reviewed for the table I produced. The answer is that I spreadsheeted all the claims in Excel (which retained the URL to the posts) and there are many more classifications and "scoring" that wasn't presented in the table.

I found that the results might serve as a small insight into the preliminary apllication proces - well certainly intrigued me.

I'm more than happy to supply the first pass file to anyone interested (PM me), but I think I'll do a second pass and apply more rigour to my scoring/classification process. Once the claim process is back up and running when James has recovered his health and his fire, I think it would be very interesting to keep tabs on the process, especially where it breaks down, and especially the *reasons* the process breaks down.

Give me some time to refine my review process - and I'll make it available to all. As I said, if anyone wants the first pass Excel for a reference - PM me, but the 2nd pass will probably be more useful.

[William Smith]

We look forward to your 2nd pass.

If you find the time and energy, could you keep an eye on the "tests that are statistical in nature", which T'ai Chi mentioned in his OP?

'Bout time we got back to constructive grounds here. After all, some people in this thread have a lot going for them. (My math skills suffice only to correctly count my change in stores.)

[EHocking]

We look forward to your 2nd pass.

If you find the time and energy, could you keep an eye on the "tests that are statistical in nature", which T'ai Chi mentioned in his OP?

'Bout time we got back to constructive grounds here. After all, some people in this thread have a lot going for them. (My math skills suffice only to correctly count my change in stores.)That's indeed what I want to fix in my second pass.

I got so disenchanted with the number of claims that didn't even get to the protocol stage, that I stopped bothering with noting the proposed targets and agreed targets.

One of the things that may prove interesting is to see what the original claimed success rate was and what the protocol process decided was a reasonable success rate.

Hopefully I'll get it done by the time the "claims department" is up and running again. A little welcome back present for James, perhaps.

[nathan]

From my page (http://www.statisticool.com/jrefchallengestats.htm)
I've always been interested in data from the past as well.

You did not state that *here*, in your original question about determining the % of female applicants. If you're going to discuss things here, don't expect us to randomly look at your other writings not posted here to fill in gaps of your conversation.


So, again, why do you assume I did not calculate what you are demanding? And why do you think what you think I calculated or not matters to proposing the idea of seeing interesting data?

I *inferred* that you'd not calculated this because (a) you claimed it seemed extraordinarily hard to do (b) you asked the JREF to provide more information (c) you've never said that you have calculated it and (d) you've never posted the number you (now appear to claim to have) calculated.

I notice that you've not actually answered my question -- do you now claim to have calculated the % of female applicants? yes or no? You're continuing in your now apparently usual method of evading the question. You could quite easily clarify the matter by either saying that you have not calculated this percentage (that *you* *yourself* claim to be an interesting data point), or by saying that you have calculated it and found it to be X.

Come on TC, answer the direct question.

[T'ai Chi]

Come on TC, answer the direct question.

I don't take orders.

Try again.

[CFLarsen]

"Questions to T'ai Chi" = "Orders".

Revealing.

[Ersby]

I love the way he posted just to tell us he wouldn't be posting.

[CFLarsen]

I love the way he posted just to tell us he wouldn't be posting.

"It's not about me, it's about the issue"....

[falcon2424]

Well, most of the tests with dousers have a preliminary alpha of what, .001? Let's assume that we set the alpha to about this level for the other test that involve effective randomness.

The chance of the next N tests all returning 'not true' is 1-(1-.001)^N.

The expectation value for the number of successes in the next N tests is simply .001N.

I'm assuming, of course, that there is no cheating, and that there aren't any real dousers.

This means that, we'd expect that in the next 1000 tests, there would be a greater than 50% chance that there would be at least one winner. (The 50% mark is reached after the 693rd test is done)

Obviously, we're not counting tests like, "I have a perpetual motion machine" which have no dealing with probability.

[nathan]

I don't take orders.

Try again.

You appear to be misspelling 'I don't answer questions'.

bye bye you waste of molecules.

[CFLarsen]

You appear to be misspelling 'I don't answer questions'.

bye bye you waste of molecules.
"Ad hom. Stick to the issue"....

[drkitten]

Well, most of the tests with dousers have a preliminary alpha of what, .001? Let's assume that we set the alpha to about this level for the other test that involve effective randomness.


Unfortunately -- as has been repeatedly pointed out -- this is not a justifiable assumption.

The nominal 0.001 cutoff is simply a typical maximum that the JREF will accept (to keep them from having to deal with paranormal claims like "I can predict the suit of the next card in a deck, as long as I don't have to repeat the trial." But it's not at all clear that all, or even "most" of the tests use this cutoff. A relatively recent challenger, for example, claimed that he could make it snow in Oakland, CA, on a specific July date. I can certainly establish that the probability of this happening is less than 0.001 (it hasn't happened yet, and there have been more than 1000 July days in the past hundred or so years) -- but it's sufficiently less than 0.001 that we can't talk meaningfully about the expected number of successes.....

In point of fact, most of the the JREF challenge communications don't get to the point where I can assess what the actual probability of success is. The applicants aren't able to define their protocols well enough. If we have a thousand applicants, and of those thousand, 998 never get to the point of taking the test, what can we say about the expected number of successes?

[T'ai Chi]

The nominal 0.001 cutoff is simply a typical maximum that the JREF will accept (to keep them from having to deal with paranormal claims like "I can predict the suit of the next card in a deck, as long as I don't have to repeat the trial." But it's not at all clear that all, or even "most" of the tests use this cutoff.


Ask Chip Denman, a statistician who consults with JREF, about what alpha JREF typically uses for tests that are statistical in nature.

[Ersby]

Ask Chip Denman, a statistician who consults with JREF, about what alpha JREF typically uses for tests that are statistical in nature.

"I don't take orders. I give them."

[drkitten]

Ask Chip Denman, a statistician who consults with JREF, about what alpha JREF typically uses for tests that are statistical in nature.

Ask him yourself. "I don't take orders."

[T'ai Chi]

"dr" said


But it's not at all clear that all, or even "most" of the tests use this cutoff.


I suggested how he could learn where to get that info. I'm not repeatedly demanding anyone answer a question.

[CFLarsen]

"dr" said



I suggested how he could learn where to get that info. I'm not repeatedly demanding anyone answer a question.

"I don't take orders. I give them."

[drkitten]

I suggested how he could learn where to get that info.

No, you didn't. You suggested where I could get that information "for tests that are statistical in nature." I pointed out that it's not clear even how many tests are statistical in nature, because the protocol is not hammered out to that extent.

[Lamuella]

At what point would the not winning by chance be 'odd' ?

It all depends on the structure and rigor of the tests. We don't know what the odds are against someone passing "by chance", and we really can't, because they change for every test.

In addition, if every person who has shown up to demonstrate their powers is a quack, the chances against one of them passing a test are very very low. The chances of someone saying something happened when it clearly didn't are also rather low.

As a rather odd example, let's say that you showed ten thousand people a black card and asked them what color it was. Would it be "odd" if none of them said "white"?