The "Process" of John Edward

[Lurker]

Originally posted by BillHoyt


Finally, you might begin to answer your own question about Houston if you turn this gaffe around.

When I look back at my college career I can only think of two teachers that I truly thought were bad teachers. Both of them had the same problem - they refused to answer questions.

Lurker

[BillHoyt]

Originally posted by Thanz

First, what insults? Where in my post did I insult anyone?
Gee, I wonder.
Do you disagree that small sample sizes can produce strange and unreliable results?
Never said that I disagree with that.
If my numbers are wrong, what are the right numbers for Lurker's office and the letter A?
You need to use the cumulative probability density, and to be sure you are accumulating over the tail of interest.
I have already admitted that my knowledge of stats is limited. But even with my limited understanding of stats, I can see that Lurker's office representation of the letter "A" is further away from the norm than JE's guesses of the letter "J". Or are you saying that this is incorrect as well?
I didn't say that. I said, in fact, that this can happen and not be significant. I chose the next letter in the alphabet and had the opposite result. Now what does that mean?

Cheers,

[BillHoyt]

Originally posted by Lurker


When I look back at my college career I can only think of two teachers that I truly thought were bad teachers. Both of them had the same problem - they refused to answer questions.

Lurker

The root of "educate" comes from the Latin, educere. Most skeptics I know decidedly do not want to be spoon-fed information. Now I know everybody's different, of course.

BTW, Randi does NOT write for Skeptical Inquirer. Just thought you'd like to know that.

Cheers,

[Thanz]

Originally posted by BillHoyt

Gee, I wonder.
I'm serious. I am not aware that I insulted anyone.

Never said that I disagree with that.
Good. Nice to know I am not completely off target.

You need to use the cumulative probability density, and to be sure you are accumulating over the tail of interest.
You bet. As should be painfully obvious, I don't know how to do that. Can you post the numbers please?

I didn't say that. I said, in fact, that this can happen and not be significant. I chose the next letter in the alphabet and had the opposite result. Now what does that mean?
Ah, but I am only concerned with the A-hole in Lurker's office. It is not a B-hole, so I don't need to look at B.

If it can happen in Lurker's office and not be significant, why would it be significant for JE?

[Lurker]

Bill:

>BTW, Randi does NOT write for Skeptical Inquirer. Just thought >you'd like to know that.

And this nonsequitor was included because...?

It is funny that other people use the same comaprisons I use. And even the same math I use. To wit:


http://www.talkleft.com/archives/003306.html

"A study of the city's murder rate shows an unlikely factor at the heart of the violence. Chicago's rate is three times that of New York not because of policing, but because of a lack of good, affordable housing."

I don't care about the argument presented here but the math. Clearly they used the same method of comparison that I used. Are they wrong too?

Lurker

[Thanz]

Just wanted to say that I found a different Poisson calculator, and I think that it will give me the numbers that BillHoyt uses.

Calculator here (http://home.clara.net/sisa/poisson.htm)

For Lurker's office, the probablity of 6 names with A in 231 is .007559, or less than 1%.

Must be a huge A-hole.

Or, I have screwed something up again.

[Lurker]

Oopsy, here is someone else using the same comparison methodology I am using.

"The third group, with a total of 55 cities, contains 8,316,455 people, with 1,741 murders, for an average rate of 20.93 per 100,000 -- roughly twice the California average."

What's that Bill? I can't compare subpopulations to a a census? Isn't that what they are doing here? And they are using what measure of comparison? Percentages? Oh no. they can't do that, can they?

Lurker

[Lurker]

Source for previous: http://members.aol.com/gunbancon/Frames/CramerMurder.html

Now Bill, if you want to argue that the confidence interval for a 5% level of significance would be too broad to make inferences on the specific frequencies for each letter I will agree with you. But if you had increased the sample size tremendously my guess is that confidence interval would tighten up considerably and eventually you would get confidence intervals that would not have the possiblity of all letters being practically the same.

And amazingly, I am back to where I first interrupeted this sorry thread. I am totally with Thanz on the opinion that a sample size of 78 would be far too small to have any idea on what the true means are for each letter. If Bill Hoyt would like to differ in opinion, he is welcome but I ouwld then ask him what he thinks a good size would be (and provide the significance too.)

Lurker

[BillHoyt]

Originally posted by Lurker
Oopsy, here is someone else using the same comparison methodology I am using.

"The third group, with a total of 55 cities, contains 8,316,455 people, with 1,741 murders, for an average rate of 20.93 per 100,000 -- roughly twice the California average."

What's that Bill? I can't compare subpopulations to a a census? Isn't that what they are doing here? And they are using what measure of comparison? Percentages? Oh no. they can't do that, can they?

Lurker

Don't strawman me. I never said that. I have made the point six ways from Sunday. Here, for example: "You simply can't compare percentages unless you know the denominators are truly the same."

[BillHoyt]

Originally posted by Thanz
Just wanted to say that I found a different Poisson calculator, and I think that it will give me the numbers that BillHoyt uses.

Calculator here (http://home.clara.net/sisa/poisson.htm)

For Lurker's office, the probablity of 6 names with A in 231 is .007559, or less than 1%.

Must be a huge A-hole.

Or, I have screwed something up again.

Yes, you have. I said we need the cumulative probability function here.

[BillHoyt]

Originally posted by Lurker
Bill:

>BTW, Randi does NOT write for Skeptical Inquirer. Just thought >you'd like to know that.

And this nonsequitor was included because...?

It is funny that other people use the same comaprisons I use. And even the same math I use. To wit:


http://www.talkleft.com/archives/003306.html

"A study of the city's murder rate shows an unlikely factor at the heart of the violence. Chicago's rate is three times that of New York not because of policing, but because of a lack of good, affordable housing."

I don't care about the argument presented here but the math. Clearly they used the same method of comparison that I used. Are they wrong too?

Lurker

No, they are not wrong.

[Lurker]

Originally posted by BillHoyt


Don't strawman me. I never said that. I have made the point six ways from Sunday. Here, for example: "You simply can't compare percentages unless you know the denominators are truly the same."

But Bill, the denominator is NOT the same in the examples provided. Why in the very one you responded to they are comparing the murder rate of a specific city (Population A) to the murder rate of California (Population B). And the devils, they even use percentages to make the comaparison ("roughly twice")!

You had better write a letter to them showing them the error of their ways. Yes, I know it will be a lot of work as it will take you 6-7 letters to try and get across why you are right and they are wrong but certainly they will rewrite their studies in light of your information.

Lurker

[BillHoyt]

Originally posted by Lurker


But Bill, the denominator is NOT the same in the examples provided. Why in the very one you responded to they are comparing the murder rate of a specific city (Population A) to the murder rate of California (Population B). And the devils, they even use percentages to make the comaparison ("roughly twice")!

You had better write a letter to them showing them the error of their ways. Yes, I know it will be a lot of work as it will take you 6-7 letters to try and get across why you are right and they are wrong but certainly they will rewrite their studies in light of your information.

Lurker

As I see it, you have three choices here, Lurker:

o Try not to understand
o Try to understand and agree
o Try to understand and disagree

But until you understand what I am saying, we can't discuss the issue. Now you can use the Aussie web page to give you more clues.

[Thanz]

Originally posted by BillHoyt


Yes, you have. I said we need the cumulative probability function here.
Well, when I did the calculation, it spat out columns for single and cumulative. Then it had probabilities listed. For your J numbers, it shows a probability of >=18 at 0.033393, with an expected number of 11.05.

For your B numbers, it shows a prob. of <=9 at 0.382435.

Using the same thing for A, with an expected average of 15.015, and a count of 6, it shows a probability of <=6 at 0.007559.

Where did I go wrong? Why are my results consistent for your posted J and B results, but my A results are wrong?

[BillHoyt]

Originally posted by Thanz

Well, when I did the calculation, it spat out columns for single and cumulative. Then it had probabilities listed. For your J numbers, it shows a probability of >=18 at 0.033393, with an expected number of 11.05.

For your B numbers, it shows a prob. of <=9 at 0.382435.

Using the same thing for A, with an expected average of 15.015, and a count of 6, it shows a probability of <=6 at 0.007559.

Where did I go wrong? Why are my results consistent for your posted J and B results, but my A results are wrong?

My mistake. I took your sentence here literally: "For Lurker's office, the probablity of 6 names with A in 231 is .007559, or less than 1%."

Yes, .007... is the tail area for <=6.

Cheers,

[Thanz]

Originally posted by BillHoyt


My mistake. I took your sentence here literally: "For Lurker's office, the probablity of 6 names with A in 231 is .007559, or less than 1%."

Yes, .007... is the tail area for <=6.

Cheers,
Now, for the million dollar question:

Isn't the statistical support for the A-hole just as strong as your statistical support for JE's cold reading? In fact, isn't the support for the A-hole stronger than support for cold reading?

[Lurker]

Originally posted by BillHoyt


As I see it, you have three choices here, Lurker:

o Try not to understand
o Try to understand and agree
o Try to understand and disagree

But until you understand what I am saying, we can't discuss the issue. Now you can use the Aussie web page to give you more clues.

I have read the Aussie site and I guess I will fall into category three above. Let me see what the Aussie site says again:

1. Non-demographic differences between the census and sample. Yep, I addressed that already. And since Bill claims my method is wrong mathematically, this does not apply.

2. Use percentages. Yep, I did use percentages, not absolute numbers. Can't see any conflict with this one. Clearly this is where a looming math error would exist but since I DID use % I don't see it.

3. Use specific geography. Again, not a problem with the math itself but how the sample was obtained. Also addressed by me previously.

and so on...

I am unaware of where my MATH is in error. Please be specific here, Bill. We have gone back nad forth quite a bit and you always refrain from being explicit.

thanks!

Lurker

[BillHoyt]

Originally posted by Thanz

Now, for the million dollar question:

Isn't the statistical support for the A-hole just as strong as your statistical support for JE's cold reading? In fact, isn't the support for the A-hole stronger than support for cold reading?

Fascinating display, Thanz.

[BillHoyt]

Originally posted by Lurker


I have read the Aussie site and I guess I will fall into category three above. Let me see what the Aussie site says again:...
Lurker,

Which part of "You can't simply look at 2.6% and 6.5% and say they are 60% different without knowing you are really comparing apples with apples" don't you understand? Do you think you can compare 2.6% of an inch an hour and 6.5% of a foot per hour and say they are 60% different? Do you honestly not see the relevance of that example?

[Lurker]

Originally posted by BillHoyt

Lurker,

Which part of "You can't simply look at 2.6% and 6.5% and say they are 60% different without knowing you are really comparing apples with apples" don't you understand? Do you think you can compare 2.6% of an inch an hour and 6.5% of a foot per hour and say they are 60% different? Do you honestly not see the relevance of that example?

Again you try to obfuscate with the same poor analogy. My units are PEOPLE, not inches and feet. Not black people and white people.

So NO, I do not see the relevence of your example. If anything, I am starting to think you are being purposely lazy in your generalizations.

Funny how when I quote a study that used the same methodolgy as mine you backed off and said they were correct. Yet you refuse to define the difference. They used POPULATION A as a subset of POPULATION B and used % to show the difference. Why is MY math in error and not theirs? It is a simple question, Bill. I am starting to think you have no answer and will insist on your inch/hour example or apples/oranges example til the cows come home.

Perhaps you are not as strong in math as you think you are...


Lurker

[T'ai Chi]

Originally posted by BillHoyt

We can define the test any way we want, given that the null hypothesis, the data set, the distribution and the level of significance all work together. I choose "J" because it is the highest frequency initial in the population.

Cheers,

I understand that you can define the test any way you want. Of course. :)

If you are only interested in the letter J, then OK, that works. If you are interested in testing more letters, say k number of letters, doing your test k times isn't wise statistically, so you'd need some other statistical tool.

[BillHoyt]

Originally posted by T'ai Chi


I understand that you can define the test any way you want. Of course. :)

If you are only interested in the letter J, then OK, that works. If you are interested in testing more letters, say k number of letters, doing your test k times isn't wise statistically, so you'd need some other statistical tool.

Quite right, T'ai. Especially if we're using a significance level cut-off of .05. I think I already addressed this in a post way back when.

Cheers,

[BillHoyt]

Originally posted by Lurker
Again you try to obfuscate with the same poor analogy. My units are PEOPLE, not inches and feet. Not black people and white people.


Interesting that you bring up "black" and "white" people. Are aborigines not people, Lurker?

Cheers,

[Lurker]

Originally posted by BillHoyt


Interesting that you bring up "black" and "white" people. Are aborigines not people, Lurker?

Cheers,

Ah, ye olde divert with strawman tactic. Sorry, Bill. I've seen it done much better by others.

As expected, you did not answer a single one of my questions. Where did you learn that non-skill from?

Sorry to say, it is people like you who give us skeptics a bad name.

Lurker

[neofight]

Originally posted by Lurker


Ah, ye olde divert with strawman tactic. Sorry, Bill. I've seen it done much better by others.

As expected, you did not answer a single one of my questions. Where did you learn that non-skill from?

Sorry to say, it is people like you who give us skeptics a bad name.

Lurker

Well, I know next to nothing about statistics, but I read this exchange anyhow, and I have to agree with Lurker here. Bill's responses are all very "lawyer-like", and I found them frustrating to read, and I wasn't even involved in the conversation.

Just my unsolicited two cents worth of observation. ;) .......neo

[Thanz]

Originally posted by BillHoyt


Fascinating display, Thanz.
That's it? "Fascinating display"? Not even an attempt at a substantive comment?

I see that, as you have with Lurker, you avoid any questions that you don't want to answer. I'll try asking them again:

Do you agree or disagree that there is just as much statistical support for the A-hole in Lurker's office as there is for John Edward cold reading, as presented in this thread? Why or why not?

Are you even able to answer a direct question? I notice that your buddy CFLarsen has pretty much dropped out here. Maybe he can do a LarsenList (tm) of the questions you are avoiding. Of course he won't, because you are not a "believer", but one can dream....

[BillHoyt]

Originally posted by Lurker


Ah, ye olde divert with strawman tactic. Sorry, Bill. I've seen it done much better by others.

As expected, you did not answer a single one of my questions. Where did you learn that non-skill from?

Sorry to say, it is people like you who give us skeptics a bad name.

Lurker

Strawman, Lurker? Diversion? Did you not read the Aussie web page? Are aborigines people, Lurker?

[BillHoyt]

Originally posted by Thanz

That's it? "Fascinating display"? Not even an attempt at a substantive comment?

I see that, as you have with Lurker, you avoid any questions that you don't want to answer. I'll try asking them again:

Do you agree or disagree that there is just as much statistical support for the A-hole in Lurker's office as there is for John Edward cold reading, as presented in this thread? Why or why not?

Are you even able to answer a direct question? I notice that your buddy CFLarsen has pretty much dropped out here. Maybe he can do a LarsenList (tm) of the questions you are avoiding. Of course he won't, because you are not a "believer", but one can dream....
Thanz,

No. Sigh. I just can't answer questions anymore! Post after post after post of non-answers. Yep, that's me.

This question is so outlandish that your insistence on an answer is a "fascinating display" of your ignorance of science and statistics. There is no evidentiary basis for the existence of your concocted and insulting construct. None. That means, that Occam's razor must be invoked here and your hypothesis is running uphill.

The cold reading hypothesis presents nothing outlandish. It presents an hypothesis that does not require the multiplication of entities. It has, therefore, more statistical support.

An experiment never stands on its own.

[Thanz]

Originally posted by BillHoyt

I just can't answer questions anymore! Post after post after post of non-answers. Yep, that's me.
Truer words have never been spoken.

This question is so outlandish that your insistence on an answer is a "fascinating display" of your ignorance of science and statistics. There is no evidentiary basis for the existence of your concocted and insulting construct. None. That means, that Occam's razor must be invoked here and your hypothesis is running uphill.
Actually, I thought it was a rather amusing construct, not insulting. But that is beside the point.

I was not asking about whether my theory has any support or validity outside of the statistics. Based on the statistics alone, the A-hole is just as supported as JE cold reading. There was a hypothesis, and the numbers supported it. We cannot rule out the A-hole based solely on the statistics.

The cold reading hypothesis presents nothing outlandish. It presents an hypothesis that does not require the multiplication of entities. It has, therefore, more statistical support.
No, it does not have more statistical support. It has more logical support, but not more statistical support. Statistics, like logic, are a tool to be used in testing a hypothesis. And in these two cases, the stats themselves lead at least equal support to both hypotheses.

You have admitted in this thread that if you were to look at more than one letter, the stats tool you have used is not appropriate. You have also chided me for not looking at other letters in Lurker's office. Yet, you have only looked at one letter and have seemed to conclude that you have done some sort of meaningful analysis. You haven't.

You know that 78 (or 85) is too small a sample to do the sort of meaningful analysis that this problem requires, but are stubbornly sticking to your pathetic J analysis as if it proves something. Well, it doesn't prove anything more than my A-hole analysis. Which is to say, it is pretty worthless in and of itself.

You have also spectacularly failed to show why luker's comparison of percentages is in any faulty, especially considering that you have accepted the same analysis done by others. You keep saying apples and oranges, but what Lurker has really done is compared one bushel of apples to the entire crop of apples (of which the bushel is a part) to see if the bushel is representative of the crop (which it wasn't). Do understand that, or are you going to avoid the issue some more?

[Lurker]

Originally posted by BillHoyt


Strawman, Lurker? Diversion? Did you not read the Aussie web page? Are aborigines people, Lurker?

Bill, when are you going to stop beating your wife?

FYI, I was not totally clear but my black/white was an example. A more true example than inches feet.

And you know I did read the website. When are you going to actually respond to my post where I go through it as it relates to my problem? When are you going to show me the differences between the studies I provided that used the same methodology as me?

Oh, that's right. I am posting to Bill Hoyt and he is the consumate avoider of answering questions.

Just my opinion.
Lurker

[BillHoyt]

Originally posted by Thanz

Truer words have never been spoken.

[B]
Actually, I thought it was a rather amusing construct, not insulting. But that is beside the point.

I was not asking about whether my theory has any support or validity outside of the statistics. Based on the statistics alone, the A-hole is just as supported as JE cold reading. There was a hypothesis, and the numbers supported it. We cannot rule out the A-hole based solely on the statistics.

[B]
No, it does not have more statistical support. It has more logical support, but not more statistical support. Statistics, like logic, are a tool to be used in testing a hypothesis. And in these two cases, the stats themselves lead at least equal support to both hypotheses.

You have admitted in this thread that if you were to look at more than one letter, the stats tool you have used is not appropriate. You have also chided me for not looking at other letters in Lurker's office. Yet, you have only looked at one letter and have seemed to conclude that you have done some sort of meaningful analysis. You haven't.

You know that 78 (or 85) is too small a sample to do the sort of meaningful analysis that this problem requires, but are stubbornly sticking to your pathetic J analysis as if it proves something. Well, it doesn't prove anything more than my A-hole analysis. Which is to say, it is pretty worthless in and of itself.

You have also spectacularly failed to show why luker's comparison of percentages is in any faulty, especially considering that you have accepted the same analysis done by others. You keep saying apples and oranges, but what Lurker has really done is compared one bushel of apples to the entire crop of apples (of which the bushel is a part) to see if the bushel is representative of the crop (which it wasn't). Do understand that, or are you going to avoid the issue some more?

I am answering the questions. You are simply not understanding. The J analysis is a valid analysis. You stubbornly insist on your sweeping assertion that 78 is inadequate. I answered early on that that statement is wrong. I then elaborated to inform you that the hypothesis, distribution model, significance level and the data themselves all determine whether or not 78i is adequate.

I then demonstrated this by constructing a valid hypothesis, null hypothesis and then proceeded to choose the most appropriate bin for my hypothesis, etc. "J" was not arbitrary. "J" is the most frequently seen initial. And it is spectacularly frequent for JE.

You then analyzed "A"s and showed them to be significant. I objected, whereupon you constructed your laughable hypothesis about "A"s to demonstrate, a posteriorisignificance. I then responded by choosing "B"s and showing them to be non-significant.

You don't get it. You just don't dive into data looking for your predetermined answer. This is what you did. No good grounds.

[Thanz]

Originally posted by BillHoyt
I am answering the questions. You are simply not understanding. The J analysis is a valid analysis. You stubbornly insist on your sweeping assertion that 78 is inadequate. I answered early on that that statement is wrong. I then elaborated to inform you that the hypothesis, distribution model, significance level and the data themselves all determine whether or not 78i is adequate.

I then demonstrated this by constructing a valid hypothesis, null hypothesis and then proceeded to choose the most appropriate bin for my hypothesis, etc. "J" was not arbitrary. "J" is the most frequently seen initial. And it is spectacularly frequent for JE.
Well, I guess here is where we disagree. I do not think that an analysis of the letter "J" in isolation tells us anything meaningful as to whether JE is cold reading. There are other factors which may help to explain why the distribution of "J" in this sample does not appear to be random. It also tells us nothing about the distribution of other common letters or any rare letters.

I looked at some other tests, in particular the one suggested by T'ai Chi. Here is a short description of that test:

Chi-square goodness-of-fit test. The goodness-of-fit test is simply a different use of Pearsonian chi-square. It is used to test if an observed distribution conforms to any other distribution, such as one based on theory (ex., if the observed distribution is not significantly different from a normal distribution) or one based on some other known distribution (ex., if the observed distribution is not significantly different from a known national distribution based on Census data).
This sounds like it does exactly what I would descibe as a meaningful test of this data - it can compare the entire distribution of letters in JE guesses to the entire population. Do you agree that such an analysis would be much more meaningful, in terms of demonstrating whether JE consistently uses more frequent letters at the expense of less frequent letters (as we would expect a cold reader to do)?

Here is a description of the adequate sample for such a test:
Random sample data are assumed. As with all significance tests, if you have population data, then any table differences are real and therefore significant. If you have non-random sample data, significance cannot be established, though significance tests are nonetheless sometimes utilized as crude "rules of thumb" anyway.

A sufficiently large sample size is assumed, as in all significance tests. Applying chi-square to small samples exposes the researcher to an unacceptable rate of Type II errors. There is no accepted cutoff. Some set the minimum sample size at 50, while others would allow as few as 20. Note chi-square must be calculated on actual count data, not substituting percentages, which would have the effect of pretending the sample size is 100.

Adequate cell sizes are also assumed. Some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count. When this assumption is not met, Yates' correction is applied.
According to this, each cell should have at least five. Or 80% of the cells should have at least five. Do you agree that 85 is inadequate for this analysis?

You then analyzed "A"s and showed them to be significant. I objected, whereupon you constructed your laughable hypothesis about "A"s to demonstrate, a posteriorisignificance. I then responded by choosing "B"s and showing them to be non-significant.
Hmm.... You looked at other letters to see if they were significant. Why did you not look at any other letters for YOUR hypothesis? J may be the most frequent letter, but that doesn't mean it is the only one of importance.

[BillHoyt]

Originally posted by Lurker
FYI, I was not totally clear but my black/white was an example. A more true example than inches feet.
And aborigines are...?
And you know I did read the website. When are you going to actually respond to my post where I go through it as it relates to my problem? When are you going to show me the differences between the studies I provided that used the same methodology as me?
And aborigines are...? And yet the web page says "Users should present their statistical estimates as percentages where both numerator and denominator are data from the same census" Now why do you suppose they say this. After all, the denominator in any census is "people" right? An aborigines are "people", right?

Yet somehow this mysterious caveat. Somehow, despite the census denominator being the same, they advise users of the data to be sure both the numerator and denominator come from the same census. Maybe it is a side effect of Aussies living upside down and all that?

Oops. Nope. It has nothing to do with living upside down. Here is the NIH giving a similar caveat for dental data!

"The user is cautioned about comparing percentages or means and concluding that differences exist without considering the confidence interval. For example, although two percentages may appear to be different, such as 43.6 % and 47.2%, if their confidence intervals overlap the difference between the two percentages are not actually statistically different. In this case, no statement can be made suggesting that one percentage is significantly different from the other."
NIH Dental, Oral and Craniofacial Data Resource Center (http://drc.nidcr.nih.gov/dqsfeatures.htm)

Oh, no! The confidence interval is yet another consideration when comparing percentages! (Or maybe that's what the Aussies were getting at? Gee, I wonder.) It must be a conspiracy from all these whacky skeptics who give skepticism a bad name and who aren't as strong in math as they think they are.

One last time: you cannot compare percentages the way you did unless the denominator units are the same. This "sameness" includes statistical considerations such as those hinted at by the Aussie site and spelled out more directly at the NIH site. You had a small sample whose representativeness of the population was unknown, and you attempted to compare it with the population at large.

Now re-read the NIH quote and think about it in terms of your n=231 sample, the population and the comparison you tried to make. Are you really willing to make the claim that the confidence intervals for the sample counts you cited do not overlap? If not, then "no statement can be made suggesting that one percentage is significantly different from the other."

[BillHoyt]

Originally posted by Thanz
Well, I guess here is where we disagree. I do not think that an analysis of the letter "J" in isolation tells us anything meaningful as to whether JE is cold reading. There are other factors which may help to explain why the distribution of "J" in this sample does not appear to be random. It also tells us nothing about the distribution of other common letters or any rare letters.
These other factors are other hypotheses that might also be tested. I didn't say it says anything about the rare letters. In order for the frequent ones to be used excessively, other letters must be used less frequently. We can focus on the upper tail of the frequent letters to look for the skew. We do not need to see the full histogram.
I looked at some other tests, in particular the one suggested by T'ai Chi. Here is a short description of that test:
I know the test. I commented on why it would not work for these data. Jeff Corey then suggested Fisher's exact test. Please refer to those posts.
Hmm.... You looked at other letters to see if they were significant. Why did you not look at any other letters for YOUR hypothesis? J may be the most frequent letter, but that doesn't mean it is the only one of importance.
I looked at another letter in a pedagogic response to your selection of "A". I didn't look at other letters for my test because you destroy the validity by dipping back into the same data over and over again. There are 26 letters and a 1 in 20 chance of getting a significant answer. We'd expect to stumble onto one. That is why my a prior selection of "J" is important. There was a rational basis for it, and it absolutely fit the hypothesis. If we speculate there may be skewing in favor of the most frequent initials, we'd expect to see that by looking at the absolutely most frequent initial.

[T'ai Chi]

BillHoyt wrote:

(from several pages back)

Now I'm assuming that J is the most frequent first initial. I would actually choose that from the control data; the names database. I am also assuming I would simply look at one such datum. There are, of course, several initials I could test in this way. The problem of data mining comes into play, though, and I really must set my significance level higher if I want to do this.


You'd actually want to set your significance level, 5%, lower.

[BillHoyt]

Originally posted by T'ai Chi
BillHoyt wrote:

(from several pages back)


You'd actually want to set your significance level, 5%, lower. [/B]

Sorry, I was speaking too loosely. By "higher" I meant "more discriminatory".

Cheers,

[Lurker]

Originally posted by BillHoyt

Oh, no! The confidence interval is yet another consideration when comparing percentages! (Or maybe that's what the Aussies were getting at? Gee, I wonder.) It must be a conspiracy from all these whacky skeptics who give skepticism a bad name and who aren't as strong in math as they think they are.


Bill, your sarcasm might be more effective had I not mentioned confidence intervals before as sources of error. Please see my quotes below. Now you have commenced selective gathering of evidence.

"And if you want to make the claim that the confidence intervals created by the standard deviations are too high that is your option."

"Now Bill, if you want to argue that the confidence interval for a 5% level of significance would be too broad to make inferences on the specific frequencies for each letter I will agree with you."


One last time: you cannot compare percentages the way you did unless the denominator units are the same. This "sameness" includes statistical considerations such as those hinted at by the Aussie site and spelled out more directly at the NIH site. You had a small sample whose representativeness of the population was unknown, and you attempted to compare it with the population at large.


Again with the selective gathering of evidence. Why show this one? Why do you STILL avoid explaining why the study I cited compared a CA city homicide rate to the CA state rate. Clearly they use POP A and POP B in the denominators. Why is all right for them but not for me? It looks like you gather the evidence that supports your theories and ignore contrary evidence. You must be a WOO-WOO!

Lurker

[Lucianarchy]

Originally posted by Lurker


Again with the selective gathering of evidence. Why show this one? Why do you STILL avoid explaining why the study I cited compared a CA city homicide rate to the CA state rate. Clearly they use POP A and POP B in the denominators. Why is all right for them but not for me? It looks like you gather the evidence that supports your theories and ignore contrary evidence. You must be a WOO-WOO!

Lurker

A case of 'Hoyt by his own petard', methinks.

:wink8:

[BillHoyt]

Originally posted by Lurker
Bill, your sarcasm might be more effective had I not mentioned confidence intervals before as sources of error. Please see my quotes below. Now you have commenced selective gathering of evidence.

"And if you want to make the claim that the confidence intervals created by the standard deviations are too high that is your option."

"Now Bill, if you want to argue that the confidence interval for a 5% level of significance would be too broad to make inferences on the specific frequencies for each letter I will agree with you."
I know full well you first raised confidence intervals. That makes the irony of that percentage comparison all the more dramatic. It also befuddles me that you would not be able to glean from either the Aussie site or the NIH site the full import of their caveats on percentage comparisons.
Again with the selective gathering of evidence. Why show this one? Why do you STILL avoid explaining why the study I cited compared a CA city homicide rate to the CA state rate. Clearly they use POP A and POP B in the denominators. Why is all right for them but not for me? It looks like you gather the evidence that supports your theories and ignore contrary evidence. You must be a WOO-WOO!
Lurker, I have addressed the issue so many times, I am tired. The denominators must be the same units. If they are populations, then populatiion descriptive paramters become an issue. That was made clear by the Aussie site when it advised against mixing and matching numerators and denominators from different census data. They went on to speak to the issue of sampling biases and a bit on how to avoid getting tripped up on them. The NIH site spelled it out a bit more when it said the confidence intervals were a huge consideration when one compares percentages and attempts to identify differences as significant.

You compared the raw percentages of a sample of 231 with the population as a whole. Those denominators are not the same. The confidence interval of the 231 sample is far larger than that of the census data. Do the calculations; you will see the 60% signifance wash away.

The homicide rate data does not suffer the same problem. Do you see percentage differences there declared significant even though the error bars overlap? Your harping on the fact I haven't directly addressed this means you have totally overlooked the fact that both the sites I gave you gave descriptions of the conditions under which you can make such percentage comparisons. I cannot say where the disconnect is. Only you can say that.

[Lurker]

Originally posted by BillHoyt

You compared the raw percentages of a sample of 231 with the population as a whole. Those denominators are not the same. The confidence interval of the 231 sample is far larger than that of the census data. Do the calculations; you will see the 60% signifance wash away.

The homicide rate data does not suffer the same problem. Do you see percentage differences there declared significant even though the error bars overlap?

First off, provide evidence for your assertion that the homicide rate data did not "suffer from the same problem". You made the claim, you provide the evidence.

Second, where did I mention significance of my data? I merely compared means.

Third, you said there was something wrong mathematically with my formula. I have yet to see you provide evidence of this.

Lurker

[BillHoyt]

Originally posted by Lurker
First off, provide evidence for your assertion that the homicide rate data did not "suffer from the same problem". You made the claim, you provide the evidence.
That was an assumption on my part based on the Uniform Crime Reporting program first established by the FBI in 1929. It gives uniform definitions to crimes and specifies how they are to be tallied and reported. All cities and states report to this system.

" To ensure these data are uniformly reported, the FBI provides contributing law enforcement agencies with a handbook that explains how to classify and score offenses and provides uniform crime offense definitions. Acknowledging that offense definitions may vary from state to state, the FBI cautions agencies to report offenses not according to local or state statutes but according to those guidelines provided in the handbook. Most agencies make a good faith effort to comply with established guidelines."
UCR (http://www.fbi.gov/ucr/ucr.htm)

Now it is possible that the data you referred to was not UCR data or it is possible that something went awry with these data such that the assumption that any random or systematic biases are not the same for all the data. If that is so, then they also erred in the comparisons.
Second, where did I mention significance of my data? I merely compared means.
That would be found in your post:

"While this analysis is anecdotal I think it clearly shows that using only a sample size of 78 people to define a population's first initial to their first name is woefully inadequate. At least if you want to have a reasonable precision to your numbers."

The bolded part translates to "significant".
Third, you said there was something wrong mathematically with my formula. I have yet to see you provide evidence of this.
I did not say something was wrong with the formula! I said you had an arithmetic error and a statistical error. The arithmetic error is using different denominator units, which I have explained ad nauseum. The statistical error overlaps that one in one sense, because you need to analyze the sample and the population statistically to understand whether or not the denominator units are truly the same. It then goes beyond that in not using statistical tools to compare populations. Ask yourself this: if you could really do what you did, why can't we simply set a "differences in percentages" criterion and use this simple test to compare populations. Why on earth do we go through all the fuss of statistics? Chi-square tests, one-tailed, two-tailed, binomial, fisher's exact, student's? Why all these clap-trap terms? Mean, variance, standard deviation, skew, kurtosis, moment-generating functions? Why not just subtract, divide and see if the percentage difference is greater than some pre-set criterion?

[Thanz]

Originally posted by BillHoyt

These other factors are other hypotheses that might also be tested. I didn't say it says anything about the rare letters. In order for the frequent ones to be used excessively, other letters must be used less frequently. We can focus on the upper tail of the frequent letters to look for the skew. We do not need to see the full histogram.
Wouldn't the full histogram be more accurate? Wouldn't it make more sense to at least look at more than one letter? I am shocked that you attach any meaningful significance to your simple analysis of one letter.

I know the test. I commented on why it would not work for these data. Jeff Corey then suggested Fisher's exact test. Please refer to those posts.
What you said was:
Chi square won't work with frequencies less than 5. That makes it useless for this case.
In other words, we don't have the data to do the test. The sample size is TOO SMALL. Just like I said.

You have not answered my question in the previous post, which was:Do you agree that such an analysis would be much more meaningful, in terms of demonstrating whether JE consistently uses more frequent letters at the expense of less frequent letters (as we would expect a cold reader to do)?

There are 26 letters and a 1 in 20 chance of getting a significant answer. We'd expect to stumble onto one. That is why my a prior selection of "J" is important. There was a rational basis for it, and it absolutely fit the hypothesis. If we speculate there may be skewing in favor of the most frequent initials, we'd expect to see that by looking at the absolutely most frequent initial. Yes, we would expect to see that. But how can we have any confidence, considering that we have not looked into any other letters, that out of the 26 letters "J" is not just a significant answer we "stumbled onto"? How do we know that "J" isn't just a fluke? Is it simply because it confirms your hypothesis? That is not what I would consider great critical thinking.

Given what you have just said, looking at "J" in isolation of the other letters is a fool's errand. We can not know with any certainty whether the significant result we find is significant due to the cold reading hypothesis or if it is just a fluke.

[Lurker]

Originally posted by BillHoyt

That was an assumption on my part based on the Uniform Crime Reporting program first established by the FBI...


That still does not resolve the POP A vs POP B question. How can someone compare a specific city's homicide rate to the states'? Clearly theyare two different populations and the denominators would be different. Do you agree or disagree that the denominators are different? Further, they then compare the percentages and say how different they are. They did not mention significance. They did not mention confidence. They only compared the means. Yet you accept their statements. It strikes me as a bit hypocritical, on your part.



"While this analysis is anecdotal I think it clearly shows that using only a sample size of 78 people to define a population's first initial to their first name is woefully inadequate. At least if you want to have a reasonable precision to your numbers."

The bolded part translates to "significant".



I am going to go over this very slowly, Bill, so you understand. If 78 people fail to adequately define the population due to lack of significance, it only proves my contention that 78 was far too small a sample size. You see, if each letter is +/- 10% then all letters seem pretty much the same. Thus, the sample size is far too small. It seems you are in full agreement with my contention. I am glad to see you are coming around.


I did not say something was wrong with the formula! I said you had an arithmetic error and a statistical error. The arithmetic error is using different denominator units, which I have explained ad nauseum. The statistical error overlaps that one in one sense, because you need to analyze the sample and the population statistically to understand whether or not the denominator units are truly the same. It then goes beyond that in not using statistical tools to compare populations. Ask yourself this: if you could really do what you did, why can't we simply set a "differences in percentages" criterion and use this simple test to compare populations. Why on earth do we go through all the fuss of statistics? Chi-square tests, one-tailed, two-tailed, binomial, fisher's exact, student's? Why all these clap-trap terms? Mean, variance, standard deviation, skew, kurtosis, moment-generating functions? Why not just subtract, divide and see if the percentage difference is greater than some pre-set criterion? [/QUOTE]

Clearly a quick comaprison of means tells can indicate if we have tested enough to arrive at the proper distribution. I never meant it as an absolute test. Once we think we are getting better numbers (subjectively) we can then perform teh exhaustive statistical analysis. At least, that is how I would approach it for a problem like this. Frankly, that is probably how YOU would approach it also since you have for many days now refused to comment on how big a sample is required to create a proper histogram of the US. The real reason you won't supply an answer is you don't know how to figure it out. Welcome to the club.

Lurker

[Thanz]

Bump for BillHoyt. You have some unanswered queries here, sir.

[BillHoyt]

Originally posted by Thanz
Bump for BillHoyt. You have some unanswered queries here, sir.

Thanz,

1. I am repeating myself and the points are still not getting across to either you or Lurker. I can't say whether this is deliberate or not, but it is quite tiresome. The specific points you last raised, for instance, I addressed in at least three different posts!

2. I stand by the ooodles of posts I have made on this subject.

3. Did you not notice there is nobody else on this thread anymore?

4. Bye.

[Thanz]

Mr. Hoyt -

Perhaps your points do not get through because you consistently fail to make them in clear and concise language. Instead, you just play some game of socratic method an oblique answers, and complain when we don't make your points for you.

I am simply asking for a clear, direct answer to my clear, direct questions. I don't think that is too much to ask.

Here they are again. They can be dealt with by you in one post, in the same time it took you to type your previous non-answer.

1. Do you agree that such an analysis would be much more meaningful, in terms of demonstrating whether JE consistently uses more frequent letters at the expense of less frequent letters (as we would expect a cold reader to do)?

2. How can we have any confidence, considering that we have not looked into any other letters, that out of the 26 letters "J" is not just a significant answer we "stumbled onto"?

3. How do we know that "J" isn't just a fluke? Is it simply because it confirms your hypothesis?

[Lurker]

Thanz:

It would appear that we can put Mr. Hoyt in with Claus as people that refuse to answer questions when the going gets tough. A shame, really.

Lurker

[BillHoyt]

Originally posted by Thanz
Mr. Hoyt -

Perhaps your points do not get through because you consistently fail to make them in clear and concise language. Instead, you just play some game of socratic method an oblique answers, and complain when we don't make your points for you.

I am simply asking for a clear, direct answer to my clear, direct questions. I don't think that is too much to ask.

Here they are again. They can be dealt with by you in one post, in the same time it took you to type your previous non-answer.

1. Do you agree that such an analysis would be much more meaningful, in terms of demonstrating whether JE consistently uses more frequent letters at the expense of less frequent letters (as we would expect a cold reader to do)?

2. How can we have any confidence, considering that we have not looked into any other letters, that out of the 26 letters "J" is not just a significant answer we "stumbled onto"?

3. How do we know that "J" isn't just a fluke? Is it simply because it confirms your hypothesis?

1. That is not the question. The question is: what can be learned from the available data. I have made this clear going back pages, in post after post.

2. We would have LESS confidence by dipping into the same data set. The significance level is .05. Twenty six dips in, we would expect to get a "significant" hit by chance alone. You cannot do that. I chose the most frequently seen letter based on the control population. I saw excessively reliance on that letter. I stopped dipping in NOT because I got what I was looking for, but because I had completed the test and because further such testing would be statistically invalid. I addressed this previously and I will not repeat this again.

3. We don't. Your response to this ought to be rich.

[Thanz]

Originally posted by Lurker
Thanz:

It would appear that we can put Mr. Hoyt in with Claus as people that refuse to answer questions when the going gets tough. A shame, really.

Lurker
I am afraid that you are correct. It bothers me when I see people who can "dish it" but can't "take it". They are behaving like bullies, nothing more.

BTW, do you ever check your PM's?

[Thanz]

Originally posted by BillHoyt
1. That is not the question. The question is: what can be learned from the available data. I have made this clear going back pages, in post after post.
It is MY question, and I note that you have evaded it again. This started when I complained that the data was insufficient to learn anything meaningful. A question about whether a test that would have meaning can be conducted with this data is certainly relevant. Do we have enough data to learn anything? If the only test we can do is on one letter, is it worth anything at all?

Isn't it better to design a test that will best answer our question, and then try to seek data for that test? You are coming at the problem backwards. Or, at least, from a different direction.

I am saying - the data is insufficient for the kind of test we need to do. You say, I can do X test with this data, without regard for whether that test actually tells us anything meaningful.

So, please answer my question.

2. We would have LESS confidence by dipping into the same data set. The significance level is .05. Twenty six dips in, we would expect to get a "significant" hit by chance alone. You cannot do that. I chose the most frequently seen letter based on the control population. I saw excessively reliance on that letter. I stopped dipping in NOT because I got what I was looking for, but because I had completed the test and because further such testing would be statistically invalid. I addressed this previously and I will not repeat this again.

3. We don't. Your response to this ought to be rich.
All that your test has shown is that the number of J guesses out of a sample of 85 are higher than random. We don't know if this is a fluke, if it is related to the way the callers are chosen, if it is related to the demographics of JE's audience, or if it is due to cold reading by JE. So, we really don't know anything. It provides very weak support, at best, to the cold reading hypothesis.

Do you not agree that at least some of these concerns would be addressed by the analysis proposed by Tai Chi? That comparing the entirety of his guesses (all letters) with an adequate sample size will reveal MUCH MUCH more than your simple J comparison?

I do not consider your J comparison to be meaningful or helpful in determining whether JE is a cold reader. If this is the best analysis that you can do with this data, then I stand by my assertion that the sample size is too small for a meaningful analysis and thank you for helping to demonstrate this fact.

[BillHoyt]

Originally posted by Thanz
It is MY question, and I note that you have evaded it again. This started when I complained that the data was insufficient to learn anything meaningful.
No, Thanz, it is your question NOW, and it is changed from your original claim. Let me refresh your memory. From 8/11 4:29:
Next, 78 guesses is a woefully small sample size. I don't know if we can really glean anything significant from such a small sample.
Now you've slipped in "meaningful", a weaseled substitution for "significant". I demonstrated a perfectly valid approach is significant. The null hypothesis failed.
A question about whether a test that would have meaning can be conducted with this data is certainly relevant. Do we have enough data to learn anything? If the only test we can do is on one letter, is it worth anything at all?
Yes, we have more than enough data to say JE appears to have over-selected the most frequent forename initial. The number of Js in his readings were significantly higher than would have been expected by chance.
Isn't it better to design a test that will best answer our question, and then try to seek data for that test? You are coming at the problem backwards. Or, at least, from a different direction.[quote]
You are trying to make the data irrelevant. It won't work. Would I like more data? Most certainly. That, however, is a different question. What we have can be analyzed and profitably. What I analyzed is very suggestive. Sorry it doesn't agree with you. But argue it on honest grounds, not this pap that is so insulting to the audience. Not with this weaseling.

[quote]I am saying - the data is insufficient for the kind of test we need to do. You say, I can do X test with this data, without regard for whether that test actually tells us anything meaningful.

So, please answer my question
Please shut up long enough to listen. This has gotten so tiresome, you nitwit, that everybody has disappeared from the thread. Congratulations on yet another attempt to thwart JREF!

THE DATA ARE SUFFICIENT TO DO A TEST. THEY DEMONSTRATE A SKEWING OF FORENAME INITIAL CHOICES. THEY SHOW THAT JE'S CHOICE OF THE MOST COMMON FIRST INITIAL WAS CHOSEN FAR TOO FREQUENTLY.

All that your test has shown is that the number of J guesses out of a sample of 85 are higher than random. We don't know if this is a fluke, if it is related to the way the callers are chosen, if it is related to the demographics of JE's audience, or if it is due to cold reading by JE. So, we really don't know anything. It provides very weak support, at best, to the cold reading hypothesis.
This demonstrates an astound lack of knowledge of science. Yes, it can be a fluke. It would be a fluke 1 time in 20. I have stated that before, several times.
Do you not agree that at least some of these concerns would be addressed by the analysis proposed by Tai Chi? That comparing the entirety of his guesses (all letters) with an adequate sample size will reveal MUCH MUCH more than your simple J comparison?
It might. I would welcome more testing. I would welcome more data. It is you who wish to sweep these data under the rug.

I do not consider your J comparison to be meaningful or helpful in determining whether JE is a cold reader. If this is the best analysis that you can do with this data, then I stand by my assertion that the sample size is too small for a meaningful analysis and thank you for helping to demonstrate this fact.
The test is significant. The hypothesis is valid. It addresses the essential question. And the test rejects the null hypothesis.

[Thanz]

Originally posted by BillHoyt
Now you've slipped in "meaningful", a weaseled substitution for "significant". I demonstrated a perfectly valid approach is significant. The null hypothesis failed.
I apologize for loose language. I did not mean "significant" in the solely statistical sense. I meant "Significant" as a synonym for "meaningful"

Yes, we have more than enough data to say JE appears to have over-selected the most frequent forename initial. The number of Js in his readings were significantly higher than would have been expected by chance.
There were a grand total of 7 extra J guesses. Again, we have no idea why. One letter does not cold reading make.

Please shut up long enough to listen. This has gotten so tiresome, you nitwit, that everybody has disappeared from the thread. Congratulations on yet another attempt to thwart JREF!
I am not trying to thwart the JREF at all, sir. If you are looking for the reason others have left the thread, I would suggest you look at yourself.

THE DATA ARE SUFFICIENT TO DO A TEST. THEY DEMONSTRATE A SKEWING OF FORENAME INITIAL CHOICES. THEY SHOW THAT JE'S CHOICE OF THE MOST COMMON FIRST INITIAL WAS CHOSEN FAR TOO FREQUENTLY.
Plese be quiet long enough to listen. There is no need to shout. The only test that the data is sufficient to do does not meaningfully illuminate the question as to whether JE is cold reading. The analysis of any one initial, whether it be the most common, least common, or whatever is in the middle, is not sufficient to provide meaningful evidence for the cold reading hypothesis.

This demonstrates an astound lack of knowledge of science. Yes, it can be a fluke. It would be a fluke 1 time in 20. I have stated that before, several times.
It could also be any of the other things I mention. There is nothing inherent in your test to isolate the cold reading effect. An analysis of just one letter is simply insufficient to do this. I don't think that anyone besides you has said this.

Let me turn this around. If the sample showed, for example, 12 J guesses and was therefore not statistically significantly different from random guesses, would you conclude that JE was likely NOT cold reading?

It might. I would welcome more testing. I would welcome more data. It is you who wish to sweep these data under the rug.
I do not wish to sweep the data under the rug. I applaud the idea behind the analysis - that is, a comparison of JE guesses to the general population. I even went and found a possible source for the population name data. I just don't think that we have ENOUGH of the data yet. I am not saying we ignore what we have - I am saying we need more to add to what we have.

The test is significant. The hypothesis is valid. It addresses the essential question. And the test rejects the null hypothesis.
Not to put too fine a point on it, the test is crap. It's results would neither confirm nor deny the essential question of whether JE cold reads. It simply does not provide enough information.

It is like someone walked up to us with a spoon and asked us to build a swimming pool for tomorrow. I say that the tools are insufficient. You say Nonsense, grab the spoon and dig a small hole. You then fill it with water, dip your leg in, and move it around. You then try and convince me you are swimming.

You are not swimming. You are just kicking your leg around in a puddle of muddy water.

[T'ai Chi]

[b]
2. We would have LESS confidence by dipping into the same data set. The significance level is .05. Twenty six dips in, we would expect to get a "significant" hit by chance alone. You cannot do that. I chose the most frequently seen letter based on the control population. I saw excessively reliance on that letter. I stopped dipping in NOT because I got what I was looking for, but because I had completed the test and because further such testing would be statistically invalid. [b]

This is why we need to test all the high frequency letters at the same time in one test, for example.

(I defined high frequency letter as a letter where 78*frequency of that letter > 5)

letter, frequency from Census Bureau combined name list (1990)
a, .064861
c, .072108
d, .074201
j, .133615
m, .100377
r, .08003

And if anyone has the actual observed counts of these letters from JE's readings, we could try another test.

[BillHoyt]

It could also be any of the other things I mention. There is nothing inherent in your test to isolate the cold reading effect.
The test rejects the null hypothesis. If you do not understand either the import or the limits of this statement I can elaborate. Or you can consult some good basic experimental design textbooks or websites.
An analysis of just one letter is simply insufficient to do this.
An analysis of this particular letter IS sufficient to reject the null hypothesis. This is all I have said. This is all I am claiming. You seem to not understand what an experimental result means.
Let me turn this around. If the sample showed, for example, 12 J guesses and was therefore not statistically significantly different from random guesses, would you conclude that JE was likely NOT cold reading?
No, I would conclude that I cannot reject the null hypothesis. If I were writing this as a scientific paper I would say "The results do not support refuting the null hypothesis."

Now, Thanz, let us turn the tables back and look at what I said way back when:

"I used the Poisson function to model the population. With an expected mean of 11.05, a count as high as (or higher than) 18 is expected to happen around 3% of the time. (Now remember we're rejecting the null hypothesis at 5%.) Based on that, I would reject the null hypothesis and say that this analysis refutes the hypothesis that JE's guesses are indistinguishable from purely random."

My language was circumspect in the ways that scientific papers are circumspect. One of the other woos whose understanding of science, like yours, rivals that of a tablespoon of peanut butter, castigated me for using "lawyer-like" language. She was, of course, ignored.

[Lurker]

Originally posted by BillHoyt
The results were a revised total guess count of 85. I then tallied the "J"s separately. I picked the "J"s because they are the most frequent initial. According to the US census data presented earlier, "J" surnames are 13.36% of the total population. In this analysis of 85 JE name guesses, I counted 18 "J" names. I calculated the expected number of "J"s (formally, the "expectation function") as 11.05.

I used the Poisson function to model the population. With an expected mean of 11.05, a count as high as (or higher than) 18 is expected to happen around 3% of the time. (Now remember we're rejecting the null hypothesis at 5%.) Based on that, I would reject the null hypothesis and say that this analysis refutes the hypothesis that JE's guesses are indistinguishable from purely random.

I'm double-checking my counts and calculations.


I have double checked them for you Bill and you were pretty close to the actual answer.

First off, your expected mean is off as it should be 11.357. Using Poisson, I found the probability of getting 18 or more to be equal to 1-P(17) = 1-0.958476 = 0.041524 or just over 4%. Still rejected by the null hypothesis of 5%.

Lurker

[Lurker]

Thanz:

Finally got around to my PM. I have looked at your "A" analysis and it is correct (I ran the numbers and came out with the same).

Bill objects to you looking for you predetermined answer. Yet didn't he notice that the "J" seemed overrepresented in the JE sample and then ran the stats on them? Isn't that data mining? Why doesn't Bill recognize that he did it himself?

Funny how he dismissed your "A" analysis even though the significance is the same as his analysis. It appears only Bill's stats are valid (and I caught some minor erros in THAT too!).

Lurker

[Thanz]

BillHoyt -

How can you read my post and not understand that the problem lies in your hypothesis? Your hypothesis, in that it focusses solely on the one letter, is insufficient to tell us anything meaningful about the broader question of JE and cold reading.

The data is insufficient to test a hypothesis that would be meaningful to the broader question of JE and cold reading. Which is what I have been saying.

[BillHoyt]

Originally posted by Lurker


I have double checked them for you Bill and you were pretty close to the actual answer.

First off, your expected mean is off as it should be 11.357. Using Poisson, I found the probability of getting 18 or more to be equal to 1-P(17) = 1-0.958476 = 0.041524 or just over 4%. Still rejected by the null hypothesis of 5%.

Lurker

Thank you for the corrections to my reported results.

[BillHoyt]

Originally posted by Lurker
Thanz:

Finally got around to my PM. I have looked at your "A" analysis and it is correct (I ran the numbers and came out with the same).

Bill objects to you looking for you predetermined answer. Yet didn't he notice that the "J" seemed overrepresented in the JE sample and then ran the stats on them? Isn't that data mining? Why doesn't Bill recognize that he did it himself?

Funny how he dismissed your "A" analysis even though the significance is the same as his analysis. It appears only Bill's stats are valid (and I caught some minor erros in THAT too!).

Lurker

Lurker,

That would be because I did not do what you said. I chose "J" because it was the most frequent letter in the Census data. I have said this several times! I then did the count and the analysis. Please read my posts and report my posts accurately.

Here is my post, once again, with a pertinent section in bold:

"I re-worked the transcripts and came up with different results and a different method. Here it is, in a nutshell:

1. I used the census data figures orginally presented, although these may need tweaking.
2. I excluded the CO show data, and concentrated solely on the available, unedited transcripts from LKL, etc.
3. I looked at JE's style and adjusted the counting procedure as follows:
o I counted all of his name guesses
o Whether he stated them as names or initials, I counted them
o I excluded impossible-to-deal-with things such as "a B softened by a vowel," and chalked that up to a "B" guess.
o I included even bizarre names such as "pepper", "salt", "brooklyn" and other nickname guesses, except that
o I only counted "Liz", "Elizabeth" type guesses as the full given name, and did not also count an "L". but
o When JE recited a littany of names, I counted each one, whether they had the same initial or differing initials (again excluding the "Liz/Elizabeth, Ronny/Ronald, and Bill/William" type guesses, where I only counted the intial of the full given name.

Sound a bit complicated? You should read the transcripts. I could not see another way to approach things fairly given that sometimes he was all over the board. My hypothesis was, that, if there is a JE mediumship process, i should honor as much of it as I could figure in making the counting rules.

The results were a revised total guess count of 85. I then tallied the "J"s separately. I picked the "J"s because they are the most frequent initial. According to the US census data presented earlier, "J" surnames are 13.36% of the total population. In this analysis of 85 JE name guesses, I counted 18 "J" names. I calculated the expected number of "J"s (formally, the "expectation function") as 11.05.

I used the Poisson function to model the population. With an expected mean of 11.05, a count as high as (or higher than) 18 is expected to happen around 3% of the time. (Now remember we're rejecting the null hypothesis at 5%.) Based on that, I would reject the null hypothesis and say that this analysis refutes the hypothesis that JE's guesses are indistinguishable from purely random."

[neofight]

Originally posted by BillHoyt
My language was circumspect in the ways that scientific papers are circumspect. One of the other woos whose understanding of science, like yours, rivals that of a tablespoon of peanut butter, castigated me for using "lawyer-like" language. She was, of course, ignored.

Ah, yes. Ignored at the time, but eventually acknowledged. ;) Oh, was I supposed to be insulted by that comment, Hoyt? LOL Not bloody likely, considering the arrogant boor that delivered the 'insult'. :D

And certainly, by "One of the other woos", you were not implying that Thanz is a *woo*, are you? :rolleyes: Because that's how your statement reads..........neo

P.S. And Bill, Thanz is right. If anyone killed this thread, it was you, and not the two honest and likeable skeptics known as Lurker and Thanz. :p

[T'ai Chi]

Just to clarify for those who could miss it, by P(17), Lurker means
SUM(P(x)|0<=x<=17), not just evaluating the Poisson probability density function at 17.

I know that the Poisson is used often when studying counts, but just because we are studying counts, does that necessarily mean the Poisson is appropriate to use? I dunno. We are assuming that the distribution of J counts is distributed approximately Poisson. Could someone discuss the evidence for that?

If instead the mean was 11.65 instead of 11.356 we'd end up failing to reject the null hypothesis. That seems fairly close. Not sure what that means though.

[BillHoyt]

Originally posted by T'ai Chi
I know that the Poisson is used often when studying counts, but just because we are studying counts, does that necessarily mean the Poisson is appropriate to use? I dunno. We are assuming that the distribution of J counts is distributed approximately Poisson. Could someone discuss the evidence for that?
I might be able to help more if I understood your question. Poisson is used for events with low probability. Classic uses are events over a geographic region, events over a span of time. Things like cars arriving at an intersection in a five minute interval. Horse deaths per acre. Defects per square foot of sheet metal. In this case, names beginning with "A" or "D" or "J".

An alternative choice might be binomial, but therein lies a catch. The advantage of Poisson is that the moment-generating functions are such that its mean and its variance are the same and therefore, we only need to know the mean, which is about all we have here.

If instead the mean was 11.65 instead of 11.356 we'd end up failing to reject the null hypothesis. That seems fairly close. Not sure what that means though.
Close only counts in horseshoes. It means nothing. The rejection criterion is always binary: you accept or you reject. If you the significance is .049 or .019 or .0499, it still meets the criterion.

Cheers,

[Jeff Corey]

There's something fishy about the Poisson distribution.

[BillHoyt]

Originally posted by Jeff Corey
There's something fishy about the Poisson distribution.

Boo, you're base!

[Jeff Corey]

Originally posted by BillHoyt


Boo, you're base!
So what are you, acid?

[Thanz]

Originally posted by Jeff Corey

So what are you, acid?
psst - he was making a bad fish soup pun.......

[BillHoyt]

Originally posted by Thanz

psst - he was making a bad fish soup pun.......

psst - he may have been returning the favor, but shifting to chemistry...

[Jeff Corey]

That still was a base canard, unless it was duck soup.

[Thanz]

BillHoyt -

I see that you have made several replies, but not to my latest post. Do you agree with what I have said?

Are not even going to bother trying to defend your hypothesis?

[BillHoyt]

Originally posted by Thanz
BillHoyt -

I see that you have made several replies, but not to my latest post. Do you agree with what I have said?

Are not even going to bother trying to defend your hypothesis?
Thanz,

My hypothesis needs no defense. You have confounded it with the test of the hypothesis. These are separate concepts scientifically. Please refer to my post about the hypothesis and the null hypothesis. They specify no test. They need not specify a test.

[Thanz]

Mr. Hoyt -

I have read your post on the hypothesis and the null hypothesis. You are completely missing the point.

Let's start at the beginning. Let's say that you are a researcher, and you want to see if JE was cold reading. You come up with the idea that if he was cold reading, he would probably guess common initials a lot, and less common initials very rarely.

Now you have to figure out the best way to test this theory. You decide that you need to compare it against the normal population instead of just using raw numbers of guesses.

Without regard for whatever data is currently available, what test do you want to do? Do you want to compare one letter? Or do you want to compare ALL letters?

[BillHoyt]

Originally posted by Thanz
Mr. Hoyt -

I have read your post on the hypothesis and the null hypothesis. You are completely missing the point.

Let's start at the beginning. Let's say that you are a researcher, and you want to see if JE was cold reading. You come up with the idea that if he was cold reading, he would probably guess common initials a lot, and less common initials very rarely.

Now you have to figure out the best way to test this theory. You decide that you need to compare it against the normal population instead of just using raw numbers of guesses.

Without regard for whatever data is currently available, what test do you want to do? Do you want to compare one letter? Or do you want to compare ALL letters?

Thanz,

You keep trying to shift the focus away from your assertion about not being able to work with the available data. You keep trying to focus the attention on this question of "wouldn't more data be better?"

I have said, repeatedly, yes. You continue to question me on this. It is irksome.

Now you try an angle of attack based on my hypothesis being wrong. It is not. You are really questioning the test of the hypothesis. Do you understand the distinction? If you have read the hypothesis and null hypothesis, why do you still confuse them with the test?

If I am a researcher in this area, and am confronted with the medium's feint, I am left with no alternative but to uncover whatever untainted evidence is available and see if it is workable. JE plays the medium's feint all the time. His show is heavily edited, he won't subject himself to testing (except by very friendly researchers) and leaves little evidence to scrutinize. With little data available, I look at the hypothesis and the available data and ask myself how can I test the hypothesis?

In the control population, "J" sings out as the, by far, most frequently seen initial. I chose that to test. I chose .05 significance. That poisons the well for further dipping afterwards as has been discussed before. JE's data were remarkable. In a small set (n=85), we would have expected 11 hits on "J". Yet he had 18. That is significant at the .05 level. I reject the null hypothesis.

[Clancie]

Not to divert this, but I'm just curious about something. If "J" showed up far beyond chance in JE readings...and if JE was 100% correct every time he said it, wouldn't it give more credence to the idea that he's not cold reading?

Apart from the argument about the too-small sample size (which now even Bill agrees with, apparently, finally), why would excessive use of "J" support JE as a cold reader if, hypothetically, all the "guesses" were correct? :confused:

[BillHoyt]

Originally posted by Clancie
Apart from the argument about the too-small sample size (which now even Bill agrees with, apparently, finally), why would excessive use of "J" support JE as a cold reader if, hypothetically, all the "guesses" were correct? :confused:
Clancie,

Where did I say that? Show us the post where you think I said that.

[Thanz]

Originally posted by BillHoyt
You keep trying to shift the focus away from your assertion about not being able to work with the available data. You keep trying to focus the attention on this question of "wouldn't more data be better?"
No, I am not shifting the focus to "wouldn't more data be better". I am keeping the focus on "is this data adequate".

I do not feel that a test of one letter alone, whatever the results, is adequate to shed light on whether JE is cold reading. An analysis of all letters is adequate. And we don't have enough data right now to do it. Because the sample size is too small.

Now you try an angle of attack based on my hypothesis being wrong. It is not. You are really questioning the test of the hypothesis. Do you understand the distinction? If you have read the hypothesis and null hypothesis, why do you still confuse them with the test?
Not that the hypothesis is wrong per se. But that the hypothesis is not adequate to tell us anything meaningful about cold reading. It is fine as far as it goes - I just don't think that it goes far enough to tell us anything meaningful. There is just not enough information there. It leaves too much unquestioned.

[BillHoyt]

Originally posted by Thanz

No, I am not shifting the focus to "wouldn't more data be better". I am keeping the focus on "is this data adequate".

I do not feel that a test of one letter alone, whatever the results, is adequate to shed light on whether JE is cold reading. An analysis of all letters is adequate. And we don't have enough data right now to do it. Because the sample size is too small.
This is patently false. Let us say that, in 85 guesses, he guessed "J" 85 times. This would be extraordinary. There would be no other letters whose frequencies we test. Does that mean we don't have enough data? No way.

You cannot make the assertion you made. The adequacy of n depends on the hypothesis, the distribution model, the desired significance and the observations.

I am done discussing this with you.



Not that the hypothesis is wrong per se. But that the hypothesis is not adequate to tell us anything meaningful about cold reading. It is fine as far as it goes - I just don't think that it goes far enough to tell us anything meaningful. There is just not enough information there. It leaves too much unquestioned. [/QUOTE]
Once again, you have confused the hypothesis with the test of the hypothesis. I am not repeating myself here either. I am done discussing this with you.

[Thanz]

Originally posted by BillHoyt

I am done discussing this with you.
Fine with me. Keep your leg in the puddle and keep insisting that you are swimming.

[Clancie]

Posted by Bill Hoyt
You keep trying to focus the attention on this question of "wouldn't more data be better?" I have said, repeatedly, yes.
Okay, re-reading your post, I guess you only saying "more data would be better", but the very small sample size is still enough for what you were doing, in your opinion.

But I'm still interested if you -really- think your results stand up, Bill? Do you really feel that this "hypothesis about 'J'" has added something statistically of value to the discussion of cold reading and JE?

[CFLarsen]

Clancie,

I see you are back to discussing mediumship. Am I correct?

[BillHoyt]

Originally posted by Clancie
But I'm still interested if you -really- think your results stand up, Bill?[/b]
"Stand up" how, Clancie? Do I really think that further data will support the cold reading hypothesis? Yes. Absolutely. The problem will be to find such data. CO cannot be used because it increases the likelihood of warm- or hot-reading.
Do you really feel that this "hypothesis about 'J'" has added something statistically of value to the discussion of cold reading and JE?
Yes. It adds a rejection (at the .05 level) of the hypothesis that JE's name and initial guesses match the names in the population. He overused the most frequently found forename initial, "J" significantly. If JE is cold-reading we would expect him to concentrate on those initials that are most abundant in the population. We see exactly that in this instance.

Cheers,

[Clancie]

Posted by Bill Hoyt

If JE is cold-reading we would expect him to concentrate on those initials that are most abundant in the population. We see exactly that in this instance.
Suppose he was correct 100% of the time? Do you still think the "over representation" of "J's" would still support your cold reading hypothesis if every 'J' name he got (and/or intial*) was very significant to the sitters?

[T'ai Chi]

Originally posted by BillHoyt
I might be able to help more if I understood your question. Poisson is used for events with low probability. Classic uses are events over a geographic region, events over a span of time. Things like cars arriving at an intersection in a five minute interval. Horse deaths per acre. Defects per square foot of sheet metal. In this case, names beginning with "A" or "D" or "J".


I'm saying that I know the Poisson is used in some cases when you have counts. What I'm wondering, is just because we have counts, does that mean a Poisson is appropriate? That is, do we have any histogram of J counts to show that it is even remotely distributed like a Poisson? I don't doubt that it is, but I just don't want to say 'because it is a count, let's use the Poisson', when that might not be a good distributional assumption.


Close only counts in horseshoes. It means nothing. The rejection criterion is always binary: you accept or you reject. If you the significance is .049 or .019 or .0499, it still meets the criterion.


Close definitely counts in statistics and the conclusions you make from it. I hope you are not claiming that (if alpha = .05) the conclusions from a p-value of .0499 are the same, because it meets the criterion as the conclusions from a p-value of .019.

[Lurker]

Originally posted by BillHoyt

Yes. It adds a rejection (at the .05 level) of the hypothesis that JE's name and initial guesses match the names in the population. He overused the most frequently found forename initial, "J" significantly. If JE is cold-reading we would expect him to concentrate on those initials that are most abundant in the population. We see exactly that in this instance.



What have we learned from Bill's test? I will agree with Bill that it is a good test to perform and provides some interesting results.

We see that JE guesses the letter J at a higher proportion than the average (taken from the census). From this sample, this is indisputable.

Now, we have to ask why. Bill Hoyt jumps to a conclusion that it is consistent with cold reading. Certainly a possibility and at the risk of offending Thanz I would consider it probable.

But there are other possibilities as well. Bill's conclusion is predicated on the notion that JE is familiar with the proportion of "J" or that he has learned over the years that "J" guesses seem to work more.

Perhaps JE is partial to a letter which starts his own name? Perhaps his "frames of reference" that the spirits read his mind to use have better "J" representations than certain other letters. I merely throw out other ideas to illustrate that the analysis does not indicate cold reading necessarily. It merely means he uses J more than the average. Reasons behind his overuse of J still remain a mystery.

That all being said, I will reiterate I think the reason is cold reading.

Lurker

[CFLarsen]

Lurker,

I need a clarification.

Are you saying that JE "tunes in" (or whatever we should call it) to people who have more J's in their circle of relationships? He simply gets messages for that group more often than for others?

How is that falsifiable?

[Lurker]

Tai Chi:

Your questioning whether the Poisson Distribution is valid in defining this problem is a good point.

In what I have seen there are three parameters that a distribution should have to qualify as a Poisson Distribution.

http://www.maths.unsw.edu.au/ForStudents/courses/math2899/handouts/lec4.pdf

1. N>=100

2. P<=0.01

3. NP<=20

You see, Bill, the problem is the Poisson Distribution has no upper limit. Clearly for a sample of 85 guesses 85 would be a hard limit. But the Poisson does not account for this. Further, this error is not just at the upper limit but appears in each value.

Perhaps we should be using the Binomial Distribution instead? It may not change the end results much but since p is relatively high we might want to consider it.

Just throwing my ignorant two cents into the arena.

Lurker

[BillHoyt]

Originally posted by Lurker
Tai Chi:

Your questioning whether the Poisson Distribution is valid in defining this problem is a good point.

In what I have seen there are three parameters that a distribution should have to qualify as a Poisson Distribution.

http://www.maths.unsw.edu.au/ForStudents/courses/math2899/handouts/lec4.pdf

1. N>=100

2. P<=0.01

3. NP<=20

You see, Bill, the problem is the Poisson Distribution has no upper limit. Clearly for a sample of 85 guesses 85 would be a hard limit. But the Poisson does not account for this. Further, this error is not just at the upper limit but appears in each value.

Perhaps we should be using the Binomial Distribution instead? It may not change the end results much but since p is relatively high we might want to consider it.

Just throwing my ignorant two cents into the arena.

Lurker
Lurker,

Please read that document for understanding this time. Please. This is as politely as I can put it.

[Lurker]

Originally posted by CFLarsen
Lurker,

I need a clarification.

Are you saying that JE "tunes in" (or whatever we should call it) to people who have more J's in their circle of relationships? He simply gets messages for that group more often than for others?

How is that falsifiable?

Glad to help clarify, I was not terribly clear in my meaning.

John Edward says the spirits read his mind and look for symbols that are in JE's frame of reference. We know he has a limited number of references. Possibly some come through better than others. for example, when he gets a "J" symbol is it a picture of him which he knows means a "J" name? Perhaps it is easier for spirits to use certain symbols in JE's lexicon. This would result in:

1. Spirits purposely choosing to communicate "J" names more often (remember it does not have to be the deceased. It can be ANYONE!) to indicate their presence.

or

2. Spirits who choose "J" symbols are more easily heard by JE.

or

3. ?

None of this is falsefiable. Just postulating possiblities. Much like cold reading is a possiblity in why he gets "J" names more often.

Lurker

P.S. And since I have your attention, I have read the Larsen List thread and generally agree with TLN. I may not always agree with your methods but you certainly ask good questions. Ones which certain believers wriggle to avoid.

I also did a quick check on some of the languages you used and from what I saw you use "possibility" when promulgating a debunking theory. It is humorous that certain believers will not even explore teh "possiblities" that you put forth. Very telling...

[Lurker]

Originally posted by BillHoyt

Lurker,

Please read that document for understanding this time. Please. This is as politely as I can put it.

Ah, the restrictions were for the approximation of the mean.

Still, the rest applies. Since you put for the Poisson Distribution, care to provide support that it is accurate for the problem at hand?

And why not the Binomial?

Lurker

[CFLarsen]

Lurker,

Thanks.

I don't agree that JE has a limited number of references. It seems that he can use anything! :D

[BillHoyt]

Originally posted by Lurker


Ah, the restrictions were for the approximation of the mean.

Still, the rest applies. Since you put for the Poisson Distribution, care to provide support that it is accurate for the problem at hand?

And why not the Binomial?

Lurker
Lurker,

Your own source yields the answer. The "conditions" you cited were not to qualify a distribution as Poisson. They cite conditions under which the Poisson is a good approximation of the Binomial.

np <=20 is our condition. Poisson and Binomial in this range are almost identical. (Look at the tables in your source document.)

[Lurker]

Bill:

Clearly the Poisson is best used when the probability is small, wouldn't you say?

Clearly the Poisson is best used when N is large, wouldn't you say?

Clearly this is selfevident. If you cannot see that then you truly do not understand the Poisson Distribution and I cannot help you.

Lurker

[Lurker]

Originally posted by CFLarsen
Lurker,

Thanks.

I don't agree that JE has a limited number of references. It seems that he can use anything! :D

Devil's advocate, my friend. Devil's advocate...

Lurker

[BillHoyt]

Originally posted by Lurker
Bill:

Clearly the Poisson is best used when the probability is small, wouldn't you say?

Clearly the Poisson is best used when N is large, wouldn't you say?

Clearly this is selfevident. If you cannot see that then you truly do not understand the Poisson Distribution and I cannot help you.

Lurker

Okay, Lurker, go back to your source. In the order of your statements, here are my responses.

Bull. Balderdash. Wrong. You need to read your source and understand it says nothing of the sort. Go back and read it. Several times if necessary. It says no such things!

[Lurker]

I posted the source as an example.

Since you refuse to see the light I will go through it quite carefully so even an obstinate man can understand.

Let us say N=100
P=0.9

u=90

Use Poisson. What do you get for the probability that the count will be higher than 100? Since I am a nice guy, I will give you the answer. The answer is 1-0.8651=.1349

13.49% according to Poisson that in one hundred tries that you will get OVER 100 positive results.

Now the rest of us can see that is obviously impossible. Do you see the problem in this example?

If so, can you see that as we descend from this example to the problem we have used it for this error is diminished? The question is, how much?

Lurker

[Lurker]

You see Bill, the Poisson Distribution is merely an approximation fo a more accurate distribution (binomial). So the limits in the example that you called Bull are guidelines as to whether the Poisson is accurately predicting the binomial.

You aren't under the impression that Poisson is more accurate than the Binomial, are you?

Lurker

[Clancie]

Lurker,

I know nothing of statistics, but perhaps you would be kind enough to answer my question since Bill hasn't.

If JE uses "J" names and initials more often than they are expected in the general population--and, hypothetically, all of these "J" names were correct and meaningful to the sitter--does their overrepresentation in his readings still support the cold reading hypothesis?

And, if he's right about 'J', shouldn't Bill's results be consistent even if the sample increases? (I ask because there are two other LKL transcripts available on the web from 1998).

And...if the analysis of "J" is correct, can all the 26 letters be accurately calculated from the same sample, from using 78 names and initials only?

[Lurker]

What? No snappy rejoinder Bill? Did my p=0.9 show you the light? Turn the light on, Bill.

Lurker

[BillHoyt]

Originally posted by Lurker
I posted the source as an example.

Since you refuse to see the light I will go through it quite carefully so even an obstinate man can understand.

Let us say N=100
P=0.9

u=90

Use Poisson. What do you get for the probability that the count will be higher than 100? Since I am a nice guy, I will give you the answer. The answer is 1-0.8651=.1349

13.49% according to Poisson that in one hundred tries that you will get OVER 100 positive results.

Now the rest of us can see that is obviously impossible. Do you see the problem in this example?

If so, can you see that as we descend from this example to the problem we have used it for this error is diminished? The question is, how much?

Lurker

ROTFLMAO! You flaming ignoramous! Holy sh**. I don't believe it. Holy sh**.

Oh, wow. You've just proven Poisson doesn't work for anything! Wow! Why did that idiot french guy invent it? BTW, fool, did you find any pairs of n and p for which this doesn't happen? What do you make of that?

Tell me where n appears in the Poisson equation. Hmm. Why not?

Holy sh**. What a stooge. You don't understand the difference between a pdf and an expectation function, do you? Holy sh**! No wonder this has gone on for pages. Holy sh**. You haven't even bothered to educate yourself. You've simply searched for bits that help support your argument. You never truly considered that you needed to learn something! Holy sh**.

Drink heavily this weekend. You're going to need it.

I'm done with you. If you don't think you need to rethink this, then be my guest. But, as far as I'm concerned, go away.

[Lurker]

Originally posted by Clancie
Lurker,

Perhaps you could answer my question since Bill hasn't. [QUOTE]

I'll give it a whirl.

[QUOTE]

If JE uses "J" names and initials more often than they are expected in the general population--and, hypothetically, all of these "J" names were correct and meaningful to the sitter--does their overrepresentation in his readings still support the cold reading hypothesis?



It does not matter whether the names were meaningful to the sitter or not. Immaterial. The hypothesis that Bill has proposed is that cold readers would use popular letters more often than mere chance (in the case of "J" 13%) in order to get more hits. So, yes, the stats that he ran does lend support to his hypothesis. It does not PROVE it, merely supports it.

But, I did provide some other reasons that we may see the result that we did in the stats. I was reaching, but I am trying to show that there are other possible explanation other than cold reading. It is not an EITHER/OR scenario.


And, if valid, shouldn't Bill's results be consistent even if the sample increases? (I ask because there are two other LKL transcripts available on the web from 1998).

And (since I know nothing of statistics, I'm wondering)....if the analysis of "J" is correct, can all the 26 letters be accurately calculated from the same sample, from using 78 names and initials only?

I would suggest we run the same test on the other two transcripts without using the previous data. If you can;

1. Count the "J" name guesses
2. Count the total number of guesses

then I will run the numbers and see what happens. Essentially follow the same method Bill outlined. I look forward to it.

Lurker

[Lurker]

Originally posted by BillHoyt




I guess I better go through this even slower than I thought necessary.

So Bill, how exactly did you calculate the mean for the Poisson in your "J" example? If I recall you took 0.1336*85 and arrived at 11.356. Hmm? Is that correct? Let me input the variable names for the formula you used. p*N

So, Mr Hoyt, I guess I am a stooge for following your methodology. Otherwise, please explain how you arrived at the mean for the Poisson?

Also, please give a rough idea of the limitations of the Poisson Distribution. I am starting to think you simply plug numbers into a calculator and hope for the best.

Lurker

[Lurker]

This bears repeating...


So Bill, how exactly did you calculate the mean for the Poisson in your "J" example? If I recall you took 0.1336*85 and arrived at 11.356. Hmm? Is that correct? Let me input the variable names for the formula you used. p*N


Lurker

[Lurker]

In Bill's own words:

Originally posted by BillHoyt


...According to the US census data presented earlier, "J" surnames are 13.36% of the total population. In this analysis of 85 JE name guesses, I counted 18 "J" names. I calculated the expected number of "J"s (formally, the "expectation function") as 11.05.

I used the Poisson function to model the population. With an expected mean of 11.05, ...[/B]

Mea culpa. I used mean instead of Bill terms, "expected number" or "expected mean".

Now, how did you avoid using N here? I specifically see you getting the expected mean of 11.05 by multiplying 85*0.13. We'll ignore your roundoff. Isn't N*p being used here? Show me where I am wrong.

Lurker

[Lurker]

ROTFLMAO! You flaming ignoramous! Holy sh**. I don't believe it. Holy sh**.

Oh, wow. You've just proven Poisson doesn't work for anything! Wow! Why that idiot french guy invent it? BTW, fool, did you find any pairs of n and p for which this doesn't happen? What do you make of that?


No, I showed it does not work in all cases. What is so hard to understand about that? And yes, if we integrate the Poisson Distribution from one above the sample size to infinity and call that value A, as A approaches zero I would contend that the Poisson Distribution is getting more accurate.



Tell me where n appears in the Poisson equation. Hmm. Why not?



Um, in N*p to arrive at the expected mean which IS in the equation and which is exactly what you used in your "J" example.


Holy sh**. What a stooge. You don't understand the difference between a pdf and an expectation function, do you? Holy sh**! No wonder this has gone on for pages. Holy sh**. You haven't even bothered to educate yourself. You've simply searched for bits that help support your argument. You never truly considered that you needed to learn something! Holy sh**.

Drink heavily this weekend. You're going to need it.

I'm done with you. If you don't think you need to rethink this, then be my guest. But, as far as I'm concerned, go away.

Your name calling is merely avoiding the inevitable. First off, why is the example I provided inaccurate? You refuted nothing yet.

Evidence, it is about evidence, Bill. Not personalities.

Lurker

[Lurker]

Bill:

Here is a website that shows that p should be small.

http://galton.uchicago.edu/~artigas/Stat251/Handouts/poisson.pdf

It says the Poisson approximation to the Binomial Distribution is good when small.

Go through it, Bill. Don't have such blind faith in the Poisson Distribution.

Lurker

[T'ai Chi]

Originally posted by BillHoyt

ROTFLMAO! You flaming ignoramous! Holy sh**. I don't believe it. Holy sh**.

Oh, wow. You've just proven Poisson doesn't work for anything! Wow! Why that idiot french guy invent it? BTW, fool, did you find any pairs of n and p for which this doesn't happen? What do you make of that?

Tell me where n appears in the Poisson equation. Hmm. Why not?

Holy sh**. What a stooge. You don't understand the difference between a pdf and an expectation function, do you? Holy sh**! No wonder this has gone on for pages. Holy sh**. You haven't even bothered to educate yourself. You've simply searched for bits that help support your argument. You never truly considered that you needed to learn something! Holy sh**.

Drink heavily this weekend. You're going to need it.

I'm done with you. If you don't think you need to rethink this, then be my guest. But, as far as I'm concerned, go away.

So much for just offering the idea that someone could be incorrect in their reasoning. :roll:

[Lurker]

Bill:

From http://www.computer.org/cise/cs2001/c3078abs.htm

"We scientists and engineers often use Poisson's probability distribution to characterize the statistics of rare events whose average number is small. Using it correctly is crucial if we are to validate claims of discovery of new phenomena, such as a new fundamental particle (few candidate collision events among millions), a remote galaxy (few photons in the telescope among the billio