Need Help with Randomizing for Experiment

[Amapola]

I'm trying to set up a proper experiment. A friend of ours believes that praying for water will make it "better" and that plants watered with prayed for water will do better than plants that are watered with normal water - ie, not prayed for water.

His original suggestion was to have two plants, water one with the prayed for water and the other with regular water. However that did not seem like enough plants to me.

I went out and bought 20 identical pots. I have filled them all with the same type of dirt - it is aged manure cleaned from my animal pens and then allowed to sit all winter. I have soaked all 20 pots with water and planted each pot with two bean seeds. These are pole beans - the variety is Kentucky Wonder. I put two seeds in each pot, spaced apart, and will re-seed if one or more fails to come up.

According to our friend we wait until the plants are up and thriving, and then we can begin the experiment.

What I am proposing to do is water 8 with the prayed for water; 8 with "normal" water, and the remaining 4 I will water with "cursed" water. (Hey - if praying works, why not cursing?) I need to go over my cursing protocol with our friend to make sure it meets his standards of "cursed".

My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water? Only I will know. My husband and our friend certainly will not know and I do not plan on telling anyone else until the experiment is done. So how do I randomize it? Please help, and thanks!

[bpesta22]

I'm trying to set up a proper experiment. A friend of ours believes that praying for water will make it "better" and that plants watered with prayed for water will do better than plants that are watered with normal water - ie, not prayed for water.

His original suggestion was to have two plants, water one with the prayed for water and the other with regular water. However that did not seem like enough plants to me.

I went out and bought 20 identical pots. I have filled them all with the same type of dirt - it is aged manure cleaned from my animal pens and then allowed to sit all winter. I have soaked all 20 pots with water and planted each pot with two bean seeds. These are pole beans - the variety is Kentucky Wonder. I put two seeds in each pot, spaced apart, and will re-seed if one or more fails to come up.

According to our friend we wait until the plants are up and thriving, and then we can begin the experiment.

What I am proposing to do is water 8 with the prayed for water; 8 with "normal" water, and the remaining 4 I will water with "cursed" water. (Hey - if praying works, why not cursing?) I need to go over my cursing protocol with our friend to make sure it meets his standards of "cursed".

My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water? Only I will know. My husband and our friend certainly will not know and I do not plan on telling anyone else until the experiment is done. So how do I randomize it? Please help, and thanks!

I think the experiment is interesting but it lacks statistical power and wouldn't be a fair test of prayer. I'd drop the n=4 cursed plants and go with 10 versus 10, though I still don't think failure would prove anything (though success would).

How large of an effect does your friend think prayer has on plant growth?

80% power is considered an acceptable minimum. For the study you propose, you would need 393 plants in each group to detect a small effect (.2 standard deviations bigger growth in the prayer group); 64 in each group for a medium effect (.5 sds) and 26 in each if the effect were large (.8).

With the 8 and 8 comparison as you have it, and assuming a medium effect, you only have .26 power.

This means that even if prayer works, there's a 74% chance your experiment won't be sensitive enough to show it.

This calculator lets you play around with values to see how many plants you'd need to get reasonable power:

http://www.dssresearch.com/toolkit/spcalc/power_a2.asp (bumping it to 10 per group gets you about .30 power)

Perhaps changing the experiment would be a good idea. For example, instead of charting plant growth, see if your friend can pick which of the 10 got prayer and which didnt after so many days or weeks of watering. You're testing a slightly different hypothesis, but you would certainly have power here to see if a believer can spot the effects of his/her prayer working.

[Graham Jackman]

I'm trying to set up a proper experiment. A friend of ours believes that praying for water will make it "better" and that plants watered with prayed for water will do better than plants that are watered with normal water - ie, not prayed for water.

His original suggestion was to have two plants, water one with the prayed for water and the other with regular water. However that did not seem like enough plants to me.

I went out and bought 20 identical pots. I have filled them all with the same type of dirt - it is aged manure cleaned from my animal pens and then allowed to sit all winter. I have soaked all 20 pots with water and planted each pot with two bean seeds. These are pole beans - the variety is Kentucky Wonder. I put two seeds in each pot, spaced apart, and will re-seed if one or more fails to come up.

According to our friend we wait until the plants are up and thriving, and then we can begin the experiment.

What I am proposing to do is water 8 with the prayed for water; 8 with "normal" water, and the remaining 4 I will water with "cursed" water. (Hey - if praying works, why not cursing?) I need to go over my cursing protocol with our friend to make sure it meets his standards of "cursed".

My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water? Only I will know. My husband and our friend certainly will not know and I do not plan on telling anyone else until the experiment is done. So how do I randomize it? Please help, and thanks!

perhaps the simplest approach would be to get your husband and friend to divide the plants into 3 groups that they agree are reasonably similar. Securely label them with a hidden label, so that the group can be revealed at the end of the experiment. Then randomly number them in sequence and keep a secret record of the group to which each plant belongs. keep all plants in the same area and maybe arrange to move their relative positions every couple of days to minimise variations ddue to their exact position. Watering would best be done in such a way that if all plants are watered at the same time, no plant is likely to dry out or drown. Sit back and wait.

[HawaiiBigSis]

Label the pots A through T. Make corresponding slips of paper. Put the slips of paper into a hat. After shaking the pieces of paper thoroughly, draw the pieces of paper out, assigning one to each group until all pots are assigned. Arrange the pots alphabetically, preferably as close together as possible, so as to exclude any differences in terms of light, music, and other ambient characteristics, and water each according to your list.

I really don't have a clue how to do it mathematically, but that's how I would randomize...

[bokonon]

I'd say skip the cursing, and just go 10 to 10 with holy vs unholy water.

As to the randomizing, I'd say let your friend mix up the pots while you're not in the room (to eliminate any suspicion later that you put "puny" seeds in one set), then number the pots. Do this now, while they're all just indistinguishable dirtbags.

Once the pots are fairly numbered, flip a coin to decide whether odd or even pots will be favored with the good water.

To be completely fair, a third party should handle the coin flip and the watering, so that neither you nor your friend will know which plants got which water. At the agreed-upon time, the numbers can be covered, and your friend can try to identify which plants were favored by prayer. I'd say if he correctly guesses 8 of the 10, prayer wins.

[bpesta22]

maybe use grass seed and bump up the n size dramatically. Plant 100 in identical containers and start from the first watering with prayer versus non. Honestly, the 10/10 comparison is not worth doing as the certain failure can be written off as poor power.

One thing to consider though is how fun it might be to measure the length of 200 different blades of grass...

[bpesta22]

It would be the binomial distribution for the guessing game.

Each trial would be graded right or wrong. I don't think it would be fair to analyze the probability on just the ones the person picked as prayed for, as there are 10 other "trials" also (the 10 not prayed for).

With n = 20, p = .5 the probability of getting at least

10 right is .59
11 right is .42
12 right is .25
13 right is .13
14 right is .057
15 right is .02

14 right would do it.


If you did do it on just the 10 prayed for then 8 out of 10 would be the magic number, but I'm not sure that's appropriate.

http://rockem.stat.sc.edu/prototype/calculators/index.php3?dist=Binomial

[bpesta22]

sorry for mass posting here. If anyone knows:

would calculating the p on just the 10 picked be kosher? This might be the only type of experimental design where it seems ok (despite what I said above) to analyze only half the observations. 8 of 10 has the same p as 14 of 20.

And, after picking any 10, the fate of the other 10 is sealed (if 8 of 10 picked were correct, than 8 of 10 not picked were correct too).

Edited!

but wait then, that would be 16 of 20 correct which has a much smaller p value than 8 of 10.

This is like the monty hall problem / question. Fascinating, or maybe I need to go to bed.

Hey you could also do signal detection analysis to separate your prayer's true sensitivity at detecting god's work from his/her personality (is she conservative when picking-- must really really look like god touched it before picking it, or liberal-- as long as it sorta looks like god's work, she'll pick it).

The hits would be the % correct on the prayed for ones.
False alarms would be the % (out of 10) where she picked one that wasn't prayed for.

Sensitivity would be hits minus false alarms and you'd predict it to be zero.

Personality would be hits + false alarms. Values greater than 1 suggests she's the "just needs it to sorta look like god's work for me to pick it" type believer.

[bpesta22]

Yeah, doing it the 8/10 way would unfairly penalize the prayer person.

You think she's performing at p=.05 when correctly picking 8 out of 10. But, translating that to 16 out of 20 to account for all observations actually has a p value of .0059. That's 10 x less likely than you would have concluded her performance to be!

[69dodge]

My question is this: How do I properly decide which pots get the cursed water, which the prayed for water and which the normal water? I want to be certain I am doing this correctly. Is there a way to randomly decide which pot gets which water?

Label the pots with the numbers 1 through 20. Then, generate a list of the numbers 1 through 20, in random order (http://www.random.org/sequences/?min=1&max=20&format=html&rnd=new), and use the first four, the next eight, and the last eight, respectively.

I agree with others in this thread that splitting the twenty pots into ten and ten is probably better than into four, eight and eight. If you and your friend agree that cursing might be worse than doing nothing but wouldn't be better, you could simply use ten cursed pots and ten blessed ones.

[69dodge]

It would be the binomial distribution for the guessing game.

I don't think so. The guesses aren't independent. The guesser knows that ten pots were blessed and ten weren't, so there's no chance that he will guess otherwise.

Here's what I've come up with: The number of correct guesses, out of twenty, will certainly be even. (Every wrong guess that a cursed pot was blessed will be accompanied by another wrong guess that a blessed pot was cursed, because the guesser knows that there are ten pots of each kind.) Assuming that there's no real difference between the pots, the probability of making exactly 2n correct guesses is $\binom{10}{n}^2/\binom{20}{10}$, so the probability of at least 14 correct guesses is 0.0894, and of at least 16 is 0.0115.

[CFLarsen]

and wouldn't be a fair test of prayer.

How do you design a fair test for prayer experiments?

I just prayed for all of the pots. Some more than others. Sorry, amapola!!

As to the randomizing, I'd say let your friend mix up the pots while you're not in the room (to eliminate any suspicion later that you put "puny" seeds in one set)

Never leave anyone alone with the setup. If there is suspicion that one can fiddle, then all can fiddle.

Who will supervise the whole set while growing? I can make the right ones wither from a distance - just give me 5 seconds and a syringe filled with battery acid. Won't be detected.

[Jeff Corey]

[QUOTE=bpesta22;3652443]sorry for mass posting here. If anyone knows:

would calculating the p on just the 10 picked be kosher? This might be the only type of experimental design where it seems ok (despite what I said above) to analyze only half the observations. 8 of 10 has the same p as 14 of 20.
Why do that? Keep it simple and stick to the design you proposed originally. One minor quibble
is that with 14 hits out of 20, p is not less than .05. It is .05765914. Go for 13 hits where p= .02069473.

[CFLarsen]

Are all statisticians incontinent?

They always have to p....

[Jeff Corey]

Bragging about the significance of your results is known as "p waving".

[fls]

I think it would be useful to allow the experiment to proceed without blinding - that is, give your friend the opportunity to 'see' dramatic differences in the growth of the plants watered with prayed for water. Then when he is convinced he sees a difference, present the plants to him one-by-one in a blinded manner and ask which group the plant belongs to. You may have to remove the plant from the pot and present it in a different position (such as upside-down) in order to avoid any subtle clues (like scratches on the pot or the shape of the plant). What you want is for him to understand the difference between what we see when our biases are allowed to influence our perceptions and what we see when they are not.

It is difficult to guarantee identical growing conditions (I spent several summers working at an agricultural research centre growing winter wheat so I have (too much :)) direct experience with this problem), so randomization is necessary to try to counteract that effect. I agree with others to drop the idea of cursed water. And there is no necessity to use 10 of each if you are simply asking him which group each plant falls in to at the end (rather than comparing groups). Any method of generating a random result will do. Simply roll the die for each pot and label it with the water it is to receive. You should end up with a reasonably balanced number of prayed-for and normal.

There would need to be a number of other rigorous controls if this were a real experiment (like making sure the two kinds of water are stored in the same manner), but it sounds like the point is really to teach your friend something about perception.

Linda

[CFLarsen]

Bragging about the significance of your results is known as "p waving".

No wonder their p is so small...

[bpesta22]

Dodge; thanks for your argument.

I still think it's the binomial distribution though as the probability of guessing right on any of the 20 trials is still .50.

Even if you know it's 10 prayed /10 not, and stop at trial 18 because you've picked 10 that you think were prayed for, the probability that 19 and 20 were not prayed for is still .5 each.

Your strategy would affect which plants you pick (i.e., guess no to all plants after guessing yes on the 10th) but not the probability of being right on any trial.

jc; I agree, though I think you mean at least 15?

[bpesta22]

Bragging about the significance of your results is known as "p waving".

What's it mean that my p is bigger than yours? Whatever; lets make sure we keep our p's orthogonal to each other...:boxedin:

[Jeff Corey]

Dodge; thanks for your argument.

I still think it's the binomial distribution though as the probability of guessing right on any of the 20 trials is still .50.

Even if you know it's 10 prayed /10 not, and stop at trial 18 because you've picked 10 that you think were prayed for, the probability that 19 and 20 were not prayed for is still .5 each.

Your strategy would affect which plants you pick (i.e., guess no to all plants after guessing yes on the 10th) but not the probability of being right on any trial.

jc; I agree, though I think you mean at least 15?
Yeah, I saw that too late to edit it. It's Saturday AM, after all.

[bpesta22]

Yeah, I saw that too late to edit it. It's Saturday AM, after all.

Morning p's are more significant than p's any other time of the day.

[Amapola]

OK, so it sounds like I should drop the cursing and just go with prayed for/not prayed for water. I can change that now easily because we can't begin until the plants are up and it will take 10 days to 2 weeks for them to germinate.

I won't be able to change them around once the plants are growing. These are pole beans; they will quickly grow about 6 - 10' high on a trellis. Yes, they are all right next to each other so should receive the same approximate growing conditions. There is no real concern about the plants being messed with (except that *Claus* has already started praying! :mad:;)) because I live way up in the mountains in such an isolated area. The guy who believes in the prayer lives about 70 miles away. I would know if he came up here, as he would have to ask for directions!

Actually we are performing the experiment at his insistence, because we (Mr. Amapola and I) do *NOT* believe in prayer and he does. He is convinced that we will see the light as this experiment progresses. He thought it would be OK to use only two plants but I pointed out this was not enough. Unfortunately, I don't have the resources to use 393 plants in each group! :D

As to how we are supposed to keep the water: he has asked us to get a container and boil it for 30 minutes, then put the water in it. We originally had thought of using a gallon glass jar, but with so many plants I think we will switch to a 5-gallon container which is completely enclosed and has a spigot on one side. I'll try and work out with our friend a proper way to sterlize this container. It was specifically made for containing potable water for camping etc. so I think there is a way to do this. As far as I can tell, he can pray for the water from 70 miles away. He has not indicated that he needs to see the water, or come in contact with it. When we first moved here I learned the procedure considered correct for getting a clean water sample out of the tap: using a match on the spigot for a set number of minutes, letting the water run for a set number of minutes, then taking the sample; and I will use this procedure to fill his container of prayer water, so as to assure it has not been contaminated. (We have a well. There is no concern about chlorine or other additions.)

I like the idea of either picking the numbers out of a hat, or simply flipping a coin. With the pot being in either group A or group B the coin flipping should work just fine, so I think I'll use the coin flipping.

I'll be running the experiment because I don't come in contact with this guy; he works with Mr. Amapola and is more his friend than mine. I only see the guy very occasionally, and not up here where we live. So for this reason, I won't tell Mr. Amapola to which group each pot belongs.

I'm sorry to hear I don't have enough pots to make it a proper experiment. Perhaps I'll try and design another experiment - but *NOT* one where I have to measure blades of wheat grass! :p

[69dodge]

I still think it's the binomial distribution though as the probability of guessing right on any of the 20 trials is still .50.

Yes, each guess individually has probability 1/2 of being correct, but this is not enough to ensure that the number of correct guesses is distributed binomially. For that, the guesses also need to be independent, and here they aren't.

(Consider an extreme example: A single coin is secretly flipped, to determine whether pots 1-10 or 11-20 are prayed for. The guesser, knowing this, will then guess either that pots 1-10 were prayed for, or that pots 11-20 were. Here too, each pot has probability 1/2 of being guessed correctly. But the number of correct guesses is definitely either 0 or 20, which is not binomial at all. Guessing all 20 pots correctly wouldn't be significant, because doing so has the very large probability of 1/2.)

But I suppose it doesn't make that much of a difference here, practically speaking. Though we disagree about the exact p-values, we agree that 16 correct guesses are enough, whereas 14 are not.

[Jeff Corey]

...I like the idea of either picking the numbers out of a hat, or simply flipping a coin. With the pot being in either group A or group B the coin flipping should work just fine, so I think I'll use the coin flipping...
I'd strongly advise that you pick the numbers out of a hat or use some other method of sampling without replacement. This would insure you have ten plants in each group and be able to set p = .5 in the binomial test.
In class experiments with 20 subjects, I use a deck of 20 cards, 10 red and 10 black suits, shuffle them and assign conditions based on which color they draw.

[69dodge]

Actually we are performing the experiment at his insistence, because we (Mr. Amapola and I) do *NOT* believe in prayer and he does. He is convinced that we will see the light as this experiment progresses. [...] I'll be running the experiment because I don't come in contact with this guy; he works with Mr. Amapola and is more his friend than mine. I only see the guy very occasionally, and not up here where we live. So for this reason, I won't tell Mr. Amapola to which group each pot belongs.

Have you thought about how exactly it will be decided, at the end of the experiment, whether prayer in fact helped the plants it was applied to?

Or is the guy just assuming that the difference between the two groups of plants will be so large as to be completely obvious to you?

[fls]

Thanks for clarifying the situation.

Do you really need to go through the process of blinding and randomization then? If he is not involved in viewing the experiment, you don't have to prove anything to him. And I agree that two groups of 10 are fine if it's just you looking at the plants (that extra layer of uncertainty is not necessary).

Linda

[Jeff Corey]

Have you thought about how exactly it will be decided, at the end of the experiment, whether prayer in fact helped the plants it was applied to?

Or is the guy just assuming that the difference between the two groups of plants will be so large as to be completely obvious to you?
One objective measure would be to weight the plants (but not the way the cops weigh pot plants, roots, dirt and all).

[bpesta22]

Not sure if you intend on doing this, but if you boil the water or do anything else to it besides pray, then the control group water should also be boiled, etc.

Start with a big drum that contains the water for the whole experiment (after you've boiled it filtered it or whatever you're doing) then mix into two containers right before you start watering and let the prayers begin.


Dodge, I still respectfully disagree. It's not the guesses that matter when it comes to the independence assumption, it's the probability of being correct on each trial, which does not change.

[Amapola]

He's assuming there will be a really huge difference - that it will be a "better" plant than normal. Somewhere around here I have a link he gave me to some goofy website that talks about praying for water and how the prayer supposedly changes the structure of the water. I'll look for it and post it when I find it.

I agree though - we need to devise a way to show the praying worked. I just thought that if the guy comes out here and looks at the 20 pots and can not tell the difference between them, that would demonstrate it did not work. I see now I should ask him how I'll know the difference.

Thanks Dr. Corey - that sounds like a better idea, I'll get 20 cards with 10 black, 10 read, shuffle them, and then just walk along the row of pots and turn up one card at a time, and assign the pot I'm next to to that group. You are right, if I use the coin flipping, I may not end up with exactly 10 in each group.

[bpesta22]

You could use coin flipping and when you get to 10 in any group, the rest would be of the other group. Just as random

This would be a good case study in experimental design. The idea is totally simple, but implementing it to fairly test the idea is not.

[Amapola]

Thanks for clarifying the situation.

Do you really need to go through the process of blinding and randomization then? If he is not involved in viewing the experiment, you don't have to prove anything to him. And I agree that two groups of 10 are fine if it's just you looking at the plants (that extra layer of uncertainty is not necessary).

Linda

The reason I am doing the randomizing is this: I am assuming I will *NOT* be able to discern any difference whatsoever. And once we tell our friend that, I am also fairly certain he will want to come up and see for himself. So I don't want all the prayed for ones right next to each other, or all in a clump. Also, I want to be certain the plants have a "fair" chance, so I want them to be randomized in the row in case one end has a problem or something.

Bpesta22, we're not going to boil the water... we were going to boil the container, in order to avoid contamination. After that it would just be the water that comes out of our well. I think it's OK, because we are not measuring against anything other than "normal" water, and the normal water that has been prayed for, if that makes any sense...

I should also say I will use two identical containers (a pint each, I think) and give all the pots the same amount of water each day, one container being designated for the normal water and one for the prayed for water.

I think I will take pictures as I go along. Perhaps I can get the local paper to print up the story afterwards... :D

[Amapola]

You could use coin flipping and when you get to 10 in any group, the rest would be of the other group. Just as random

This would be a good case study in experimental design. The idea is totally simple, but implementing it to fairly test the idea is not.

I was just thinking this very thing... :D

[CFLarsen]

There is no real concern about the plants being messed with (except that *Claus* has already started praying! :mad:;))

Ask your friend how he would control for that.

[Amapola]

Hey, I found that link I was talking about: linky-poo. (http://www.life-enthusiast.com/twilight/research_emoto.htm)

Maybe some of you can make better sense of this link than I can. They photographed water as it crystallized and the crystals were all different. Isn't that the way it is supposed to occur? I thought I learned as a kid that snowflakes don't all look alike... am I wrong here? :con2:

Claus, I will ask him if other people praying will mess things up, and warn him that some mad Dane might be praying for who knows what... ;)

Of course the whole thing is remarkably silly, but this guy is a very good friend of Mr. Amapola, and a very earnest and nice person, so we are taking him as seriously as we can. I am doing my best to honestly set up a good experiment that will have results (one way or the other!) that were not arrived at by cheating or faulty methodology, as far as it is possible within my means. I appreciate all the suggestions. You guys have covered things I did not think about that should be thought of. Even controlling for *other* people praying ought to be taken into consideration.

[ben m]

Let me propose a blinding method which would solve most of the problems, including judgement and tampering issues.

(1) Label each plant with a letter A-T.
(2) Label 20 paper cups with the letters A-T.
(3) In secret, a trusted 3rd party (Person C) should randomly draw 10 cups to be "prayed for"; this should be written down and kept by Person C but not shown to anyone.
(4) Every watering-period, the experimenters provide Person C with a bucket of holy water and a bucket of regular water. In secret, person C fills each cup with an appropriate type of water, then leaves the room.
(5) The experimenters return and water each plant from its designated cup.
(6) The experimenters randomly shuffle the plants on their shelf so that crowding/light differences don't have too large an effect.
(7) Repeat steps 4-5 the agreed-upon number of times.
(8) At the end of the experiment, the experimenters "rate" each plant on whatever scales they like (Height? Mass? Number of leaves?), or subjectively rank the plants to identify the 10 best.
(9) Person C returns and reveals which plants had gotten holy water. This list is compared with the best-plants list determined in Step 8.

In this case, even a maximally-biased, cheating experimenter can't change the results; they don't know which plants to discreetly kill or fertilize or shadow, nor which ones to overrate in the evaluation step.

But it's more important to agree ahead of time on how stats work; you don't want him or her to guess 12/20 right and start claiming victory.

[Jekyll]

You could use coin flipping and when you get to 10 in any group, the rest would be of the other group. Just as random

This would be a good case study in experimental design. The idea is totally simple, but implementing it to fairly test the idea is not.

I would probably split the plants into ten pairs and give one of each pair holy water and the other unholy.
You can assign the splitting of the pairs by coin flip.

Guessing the pairs correctly will then have a symmetric binomial distribution of the size 10 and is nice and tractable.

[69dodge]

I'd strongly advise that you pick the numbers out of a hat or use some other method of sampling without replacement. This would insure you have ten plants in each group and be able to set p = .5 in the binomial test.

I agree about sampling without replacement. (The link I gave in post #10 is to a webpage that can be used to do that automatically, without a hat. It lists the numbers from 1 to 20 in random order. Pray for the first ten listed, don't pray for the last ten.)

You agree with bpesta22 that a binomial test is appropriate? I still disagree. Can you explain your reasoning? Where are the independent Bernoulli trials?

Not sure if you intend on doing this, but if you boil the water or do anything else to it besides pray, then the control group water should also be boiled, etc.

Start with a big drum that contains the water for the whole experiment (after you've boiled it filtered it or whatever you're doing) then mix into two containers right before you start watering and let the prayers begin.

Right, that makes sense. Everything should be the same, except that half the water gets prayed for and half doesn't.

Dodge, I still respectfully disagree. It's not the guesses that matter when it comes to the independence assumption, it's the probability of being correct on each trial, which does not change.

:confused: Are we talking about the same setup?

There are twenty plants. Ten of them were given prayed-for water. Ten were given regular water. The guy knows this; he just doesn't know which are which. He will try to decide which are which.

There aren't really twenty separate trials. He has one big job to do: separate the twenty plants into two groups of ten each.

If we think of his guessing the status of each plant as a separate trial, the probability of being correct on the next trial does change, depending on whether the previous trials were correct. For example, if he guesses that the first ten plants were prayed for, and he is told that those guesses were correct, he is sure to guess correctly about the remaining ten plants. The last ten trials no longer have their original probability (1/2) of being correct.

I agree though - we need to devise a way to show the praying worked. I just thought that if the guy comes out here and looks at the 20 pots and can not tell the difference between them, that would demonstrate it did not work. I see now I should ask him how I'll know the difference.

No, that's ok. You said he lived far away, so I wasn't sure what the plan was. If he can look at the plants himself, that's the best. You can let him use whatever method he wants, to pick out the prayed-for plants.

But you and he should try to agree beforehand on how many correct picks are "enough". Even if prayer doesn't affect the plants at all, so that in effect he'll simply be guessing, he'll get a few right anyway, just by chance.

Does he think he'll be able to make at least 16 correct identifications out of 20? That is, out of the 10 plants he says were prayed for, he's allowed one or two mistakes, but no more?

I would probably split the plants into ten pairs and give one of each pair holy water and the other unholy.
You can assign the splitting of the pairs by coin flip.

Guessing the pairs correctly will then have a symmetric binomial distribution of the size 10 and is nice and tractable.

That would work. But only one mistake would be allowed then, rather than two, for the same level of significance (about 1%).

[Jekyll]

That would work. But only one mistake would be allowed then, rather than two, for the same level of significance (about 1%).

Sure, and I agree with your earlier post (which I missed the first time around) about the distribution being:

$\frac{\binom{10}{n/2}\times\binom{10}{10-n/2}}{ \binom{20}{10}}$
for the probability of correctly labelling an even number n, of plants in the original partitioning problem.

But it seems to me, there's a direct trade off between the power of the test, and how many people can follow the maths. :D

[Jeff Corey]

I agree about sampling without replacement. (The link I gave in post #10 is to a webpage that can be used to do that automatically, without a hat. It lists the numbers from 1 to 20 in random order. Pray for the first ten listed, don't pray for the last ten.)

You agree with bpesta22 that a binomial test is appropriate? I still disagree. Can you explain your reasoning? Where are the independent Bernoulli trials?



Right, that makes sense. Everything should be the same, except that half the water gets prayed for and half doesn't.



:confused: Are we talking about the same setup?

There are twenty plants. Ten of them were given prayed-for water. Ten were given regular water. The guy knows this; he just doesn't know which are which. He will try to decide which are which.

There aren't really twenty separate trials. He has one big job to do: separate the twenty plants into two groups of ten each.

If we think of his guessing the status of each plant as a separate trial, the probability of being correct on the next trial does change, depending on whether the previous trials were correct. For example, if he guesses that the first ten plants were prayed for, and he is told that those guesses were correct, he is sure to guess correctly about the remaining ten plants. The last ten trials no longer have their original probability (1/2) of being correct...
I am assuming that he will make all his guesses in one sitting and not be given feedback until all guesses are recorded. This would make each guess independent of the previous ones, especially if he were not told how many were in each group.
So a Binomial or Sign test should be appropriate.

[Amapola]

The way Dr. Corey describes is the way I had planned to do it - let the guy walk around and look at all the plants, and say one by one which plant belongs in which group. When he was finished, then I would produce the map that showed which plants were given the prayed for water and which the normal water. With only me knowing which were which, I would probably have my husband go around with him and record his answers.

69dodge, I'll check out that link! That's a nice high-tech way to get the numbers.

The more I think about this the more convinced I become that the guy will drop out and refuse to say which plants belong in which group before the end. Maybe I need more faith in his faith.

[JJM]

{snip} In class experiments with 20 subjects, I use a deck of 20 cards, 10 red and 10 black suits, shuffle them and assign conditions based on which color they draw.That sounds good. I would have suggested labeling cards 'P' for "prayer" and 'P' for "plain."

{snip} I'll be running the experiment because I don't come in contact with this guy; he works with Mr. Amapola and is more his friend than mine. I only see the guy very occasionally, and not up here where we live. So for this reason, I won't tell Mr. Amapola to which group each pot belongs. {snip}You definitely have to get the believer to come up and determine the outcome, all by himself. Whatever criterion, or criteria, for efficacy must be specified before he makes his choices. The more criteria he chooses, the more likely it is he will find a difference, and modified, more-rigorous statistics have to be applied.

For the statistically-challenged (e.g., me) it is best to choose one criterion. On the other hand, that allows the true believer to rationalize that he simply chose the wrong measure. And that is what believers do in the face of negative results- they rationalize (see Randi's book "Flim-Flam").

{snip} Do you really need to go through the process of blinding and randomization then? If he is not involved in viewing the experiment, you don't have to prove anything to him. And I agree that two groups of 10 are fine if it's just you looking at the plants (that extra layer of uncertainty is not necessary).

LindaI agree; but the application of randomization is simple, and it removes one avenue of post-hoc rationalization.

[HawaiiBigSis]

<snip>The more I think about this the more convinced I become that the guy will drop out and refuse to say which plants belong in which group before the end. Maybe I need more faith in his faith.
In the end, you'll get a bunch of beans, which is a nice enough reward for his faith, or lack thereof, yeah?

Have you decided how long to run the test? All the way to fruit, or will you stop sometime before that? Maybe the number of beans per plant would be a determination factor that eliminates the need to decide whether any particular plant is healthier than another.

[Jeff Corey]

[QUOTE=HawaiiBigSis;3655841]In the end, you'll get a bunch of beans, which is a nice enough reward for his faith, or lack thereof, yeah?
"It ain't worth a hill of beans" might apply here.

[Hokulele]

The way Dr. Corey describes is the way I had planned to do it - let the guy walk around and look at all the plants, and say one by one which plant belongs in which group. When he was finished, then I would produce the map that showed which plants were given the prayed for water and which the normal water. With only me knowing which were which, I would probably have my husband go around with him and record his answers.

69dodge, I'll check out that link! That's a nice high-tech way to get the numbers.

The more I think about this the more convinced I become that the guy will drop out and refuse to say which plants belong in which group before the end. Maybe I need more faith in his faith.


I vote that you mess with his head. Have only 1 pot use the prayed for water and see if he can pick it out. :cool:

[Amapola]

[QUOTE=HawaiiBigSis;3655841]In the end, you'll get a bunch of beans, which is a nice enough reward for his faith, or lack thereof, yeah?
"It ain't worth a hill of beans" might apply here.

Except it will be... and a hill of beans might be all I end up with!

I vote that you mess with his head. Have only 1 pot use the prayed for water and see if he can pick it out. :cool:

You are evil. I like it. :D

However I will stick to what I have agreed to do which is water half of them with holy water. I'm such a boring person...

[69dodge]

I am assuming that he will make all his guesses in one sitting and not be given feedback until all guesses are recorded. This would make each guess independent of the previous ones, especially if he were not told how many were in each group.
So a Binomial or Sign test should be appropriate.

Right, he won't actually be given feedback until he's done making all his guesses. But the various guesses still aren't independent, because, if he were given feedback on earlier guesses, he would be able to do better on later ones. Which is why he shouldn't be given feedback.

That's for an experiment where everyone knows that there are sure to be ten plants in each group. I think this is what Amapola intends to do.

But a binomial test is appropriate if each plant is independently assigned to a group, as by a coin flip, so that the two groups might not end up being the same size. (fls suggested doing the experiment this way.) And, in this case, it's perfectly ok to give him feedback after guessing the status of each plant, because doing so won't help him make better guesses about subsequent plants.

[bpesta22]

Right, he won't actually be given feedback until he's done making all his guesses. But the various guesses still aren't independent, because, if he were given feedback on earlier guesses, he would be able to do better on later ones. Which is why he shouldn't be given feedback.

That's for an experiment where everyone knows that there are sure to be ten plants in each group. I think this is what Amapola intends to do.

But a binomial test is appropriate if each plant is independently assigned to a group, as by a coin flip, so that the two groups might not end up being the same size. (fls suggested doing the experiment this way.) And, in this case, it's perfectly ok to give him feedback after guessing the status of each plant, because doing so won't help him make better guesses about subsequent plants.

I think giving him feedback after each guess would bias the test in the way you argued above.

With no feedback, we likely have to agree to disagree on our probability arguments (though I submit the independence assumption applies only to the P correct of each trial, which would nt be violated unless they guy got feedback after each -- for example, the p of getting trial 20 correct would be 1.0).

[69dodge]

I submit the independence assumption applies only to the P correct of each trial, which would nt be violated unless they guy got feedback after each -- for example, the p of getting trial 20 correct would be 1.0.

Independence of events doesn't mean that they have the same probability. Two events with the same probability might be independent or not. Two events with different probabilities might be independent or not.

By definition, two events are independent if the probability that both occur together is equal to the product of their individual probabilities.

Instead of twenty plants that are split ten and ten, suppose there are only two, one prayed for and one not. Either the first was and the second wasn't, or vice versa. The guy knows that these are the only two possibilities, and will guess one of them. The probability is 1/2 that he guesses correctly about the first plant. The probability is also 1/2 that he guesses correctly about the second plant. If the two events (correct guess about plant 1, and correct guess about plant 2) were independent, the probability that both his guesses are simultaneously correct would be 1/4. The binomial distribution, which assumes independence, says it's 1/4. But it isn't 1/4. It's 1/2. Either he gets both plants right, or both plants wrong. There's no chance that he'll get one right and one wrong, whereas, according to the binomial distribution, such an event should have probability 1/2.

[bpesta22]

Independence of events doesn't mean that they have the same probability. Two events with the same probability might be independent or not. Two events with different probabilities might be independent or not.

By definition, two events are independent if the probability that both occur together is equal to the product of their individual probabilities.

Instead of twenty plants that are split ten and ten, suppose there are only two, one prayed for and one not. Either the first was and the second wasn't, or vice versa. The guy knows that these are the only two possibilities, and will guess one of them. The probability is 1/2 that he guesses correctly about the first plant. The probability is also 1/2 that he guesses correctly about the second plant. If the two events (correct guess about plant 1, and correct guess about plant 2) were independent, the probability that both his guesses are simultaneously correct would be 1/4. The binomial distribution, which assumes independence, says it's 1/4. But it isn't 1/4. It's 1/2. Either he gets both plants right, or both plants wrong. There's no chance that he'll get one right and one wrong, whereas, according to the binomial distribution, such an event should have probability 1/2.

Dodge, thanks, but I still disagree (which is why I said perhaps we need to agree to disagree). With just two scenarios-- and no feedback-- the probability is 1/4 that he'd get both right (or both wrong. and, it's 1/2 that he'd get at least one right.

With feedback, then we leave the realm of probability for trial 2 and use deductive certainty to guarantee that trial is correct. With feedback and only 2 trials, assuming the selector is rational, it's a 50/50 chance, as you said.

With feedback and 20 trials, you could use deduction to get at least the last trial correct for sure, and maybe a few more (the maximum you could get right deductively would be 10, in the odd chance all of one type were ordered first).

That's why I said don't give feedback, and I think we agree on the points above.

Without feedback, the probability on each trial is independent. That you *guessed* prayed for 10 times already by trial 20 has no affect on the *probability* that trial 20 is a prayed for bean.

I'm saying, without feedback, no possible guessing strategy would make it so the probability of getting it right on this trial depends on the probability of getting it right on any other trial.

This I think is where we disagree-- the independence isn't concerned with how people select on each trial, but only with the probability of success on each trial (which I claim cannot vary by guessing strategy, unless the selector gets feedback).

B

[Jekyll]

Dodge, thanks, but I still disagree (which is why I said perhaps we need to agree to disagree). With just two scenarios-- and no feedback-- the probability is 1/4 that he'd get both right (or both wrong. and, it's 1/2 that he'd get at least one right.

Is your definition of feedback that the guesser knows how many of the total plants have been given holy water or something else?

Assuming the guesser already knows that only one of the two plants has been watered, there are only two guesses he can legitimately make: That the first plant was watered or the second plant was watered. So he has a 50% chance of being 100% right, without any additional information being offered.

[bpesta22]

Is your definition of feedback that the guesser knows how many of the total plants have been given holy water or something else?

Assuming the guesser already knows that only one of the two plants has been watered, there are only two guesses he can legitimately make: That the first plant was watered or the second plant was watered. So he has a 50% chance of being 100% right, without any additional information being offered.

I agree on the 2 trial scenario where the selector is told whether his first pick is right before he makes the second pick.

By feedback in the 20 trial scenario, I meant that after each pick, he is told whether he's right or wrong. This would violate the independence assumption for at least the last trial.


Without such feedback, my assertion is that this is squarely the binomial distribution with no violation of the independence assumption.

Knowing that 10 are prayed for and 10 are not does not violate the independence assumption.

[Jekyll]

I agree on the 2 trial scenario where the selector is told whether his first pick is right before he makes the second pick.

By feedback in the 20 trial scenario, I meant that after each pick, he is told whether he's right or wrong. This would violate the independence assumption for at least the last trial.

Ok, so if we stick with the 2 plant scenario, assume the selector knows that there is only 1 holy plant and nothing else, he is given no further feedback, what would his guesses look like?

I can only see him making 2 possible guesses:

"Plant one is holy and plant two is not."
or
"Plant two is holy and plant one is not."

But if you think the binomial distribution applies, you must think he can some how make a choice like:
"Both plant one and two are holy."
or,
"Neither plant one nor two are holy."

as these are the only way to get the 50% right answers, predicted by the binomial distribution.

[bpesta22]

Ok, so if we stick with the 2 plant scenario, assume the selector knows that there is only 1 holy plant and nothing else, he is given no further feedback, what would his guesses look like?

I can only see him making 2 possible guesses:

"Plant one is holy and plant two is not."
or
"Plant two is holy and plant one is not."

But if you think the binomial distribution applies, you must think he can some how make a choice like:
"Both plant one and two are holy."
or,
"Neither plant one nor two are holy."

as these are the only way to get the 50% right answers, predicted by the binomial distribution.

Perhaps we're agreeing here but failing to communicate.

I agree that a 2 trial experiment is problematic when one gets feedback (or knows that one is prayed for and one is not). I don't think one can use the same deductions in a 20 trial experiment, with no feedback provided, even if the selector knows that half are prayed for and half are not.

I also think the 2 trial example is logically reduced to a one trial scenario, so I think it is binomial, with p = .5 of getting it right.

[Jekyll]

Perhaps we're agreeing here but failing to communicate.

I agree that a 2 trial experiment is problematic when one gets feedback (or knows that one is prayed for and one is not). I don't think one can use the same deductions in a 20 trial experiment, with no feedback provided, even if the selector knows that half are prayed for and half are not.

I also think the 2 trial example is logically reduced to a one trial scenario, so I think it is binomial, with p = .5 of getting it right.

I agree, except with the bolded bit.

It is clear to me that the independence assumption is violated in the 20 case scenario, just as it is in the two case scenario.
If we fix the status of the first 19 plants there is only one possible state that the 20th plant can be in. Similarly if we fix the first 18 plants there are at most two different states the remaining two plants can be in.

If the independence assumptions held there would be 2 possible states the last plant could be in after fixing the first 19 of them, and 4 possible states for the last 2 plants after fixing 18.

[Amapola]

OK - bpesta, 69Dodge and Jekyll - am I thinking about this wrong? To me, there will be 10 prayed-for pots. So if he can guess all 10 prayed for ones, he gets 10 out of 10. But if he only guesses 5 of the prayed for ones, he gets 5 out of 10. I'm leaving the other 10, the normal ones or the control group, out of his correct/not correct guesses. That's the way I had planned to record the results. (So I would see a trial with only two pots as he either gets it right or wrong - he either chooses the correct prayed for pot, or chooses the wrong pot.)

I'm striving to set this up so that he gets no feedback as he is guessing, not even unconscious signals from Mr. Amapola.

But am I wrong in only considering the 10 that he will have prayed for?

If this is not the right way to go about it I will of course change it.

[bpesta22]

I agree, except with the bolded bit.

It is clear to me that the independence assumption is violated in the 20 case scenario, just as it is in the two case scenario.
If we fix the status of the first 19 plants there is only one possible state that the 20th plant can be in. Similarly if we fix the first 18 plants there are at most two different states the remaining two plants can be in.

If the independence assumptions held there would be 2 possible states the last plant could be in after fixing the first 19 of them, and 4 possible states for the last 2 plants after fixing 18.


Why not fix the states of the last 19 plants and complain about the status of the first one? Or , why not fix the status of 1-9 and 11-20 and claim 10 is violated?

It's clear that there's n-1 degrees of freedom here, but show me how just knowing that changes anything?

Of all possible states for the last plant, half of them include it being prayed for, and the other half include it not being prayed for. p=w/t = .50 for each of 20 trials in the population (for every scenario where the last plant must be prayed for there are equally probably scenarios where the last plant must be a control).

Show me how just knowing that it's broken down 10 and 10 alone can give the selector an advantage where the null is not .50 for any trial.

It seems like the pop quiz fallacy? The pop quiz cant be on the last day of the class, because it wouldn't then be a surprise, etc.

I could be wrong and would concede if someone could explain how just knowing that it's 10 and 10 gives you any advantage whatsover or clues you in on what to guess for pot 20?

[bpesta22]

Amp

I think wrong is too strong a word as it seems like either approach would yield the same conclusions except in a very improbable borderline case.

I think it's an academic debate at this point; still submitting that it's a 20 trial binomial test-- not a 10 trial one on just the prayed for beans.

Consider, we agreed that 8/10 would pass in your version and 14/20 in my version.

In one, your subject needs 80% accuracy to win. In the other, just 70% accuracy does it!

As JC suggested, if we make it 15 of 20 to pass, then that's just 75% accuracy, but that's considerably less probable then the 80% accuracy rate needed to pass the 10 trial test. Edited: so it would be easier to pass the 80% accuracy test than the 75% accuracy test!

So, there's psychological advantage to going with n=20. "C'mon you just need 70% (or 75%) accuracy to win! That's like getting a C in a class. Is prayer that weak that it can't even produce a C?"

[69dodge]

OK - bpesta, 69Dodge and Jekyll - am I thinking about this wrong? To me, there will be 10 prayed-for pots. So if he can guess all 10 prayed for ones, he gets 10 out of 10. But if he only guesses 5 of the prayed for ones, he gets 5 out of 10. I'm leaving the other 10, the normal ones or the control group, out of his correct/not correct guesses. That's the way I had planned to record the results. (So I would see a trial with only two pots as he either gets it right or wrong - he either chooses the correct prayed for pot, or chooses the wrong pot.)

I'm striving to set this up so that he gets no feedback as he is guessing, not even unconscious signals from Mr. Amapola.

But am I wrong in only considering the 10 that he will have prayed for?

If this is not the right way to go about it I will of course change it.

That's fine.

The number of correct guesses about normal pots will always be the same as the number of correct guesses about prayed-for pots. So there's no need to record both of them explicitly.

(For example, suppose he correctly guesses, about eight of the prayed-for pots, that they were prayed for. Then, he's got two more "prayed-for" guesses left. They will necessarily be made about two of the normal pots. And so, the remaining eight normal pots will be guessed correctly as being normal.)

[Jekyll]

..snip..

It's clear that there's n-1 degrees of freedom here, but show me how just knowing that changes anything?
..snip..

Actually, it's more restrictive than that.

Show me how just knowing that it's broken down 10 and 10 alone can give the selector an advantage where the null is not .50 for any trial.

Ok, so under the binomial distribution there are 2^20 = 1048576 possible guesses he can make about which pots are being prayed for. Only one of these guesses will be entirely right.

If we restrict ourselves to just the cases where there are 10 prayed for plants there are 20C10 =184756 possible guesses.
Still exactly one of these guesses will be right.

So if the guesser knows that there are exactly 10 pots being prayed for he is almost 6 times more likely to be completely right than the binomial distribution leads us to believe.

[BobK]

Given the small plant population, I wouldn't use normal water at all.

Use the assumption that prayed water is better than normal, and cursed water is worse than or equal to normal.

If there is a difference in growth, it should be most noticeable just using prayed and cursed water.

If the results happen to show cursed as being best, it might put your friend in a state of cognitive dissonance. :)

[Jekyll]

OK - bpesta, 69Dodge and Jekyll - am I thinking about this wrong? To me, there will be 10 prayed-for pots. So if he can guess all 10 prayed for ones, he gets 10 out of 10. But if he only guesses 5 of the prayed for ones, he gets 5 out of 10. I'm leaving the other 10, the normal ones or the control group, out of his correct/not correct guesses. That's the way I had planned to record the results. (So I would see a trial with only two pots as he either gets it right or wrong - he either chooses the correct prayed for pot, or chooses the wrong pot.)

I'm striving to set this up so that he gets no feedback as he is guessing, not even unconscious signals from Mr. Amapola.

Your method is fine, you're recording enough data that anyone could recreate the full outcome if they needed to.

We're just arguing about what the numbers mean, and what we should count as a success.

We don't want to give unearned credit to god for making the flowers grow, or needlessly oppress Christians.

[bpesta22]

Actually, it's more restrictive than that.

Ok, so under the binomial distribution there are 2^20 = 1048576 possible guesses he can make about which pots are being prayed for. Only one of these guesses will be entirely right.

If we restrict ourselves to just the cases where there are 10 prayed for plants there are 20C10 =184756 possible guesses.
Still exactly one of these guesses will be right.

So if the guesser knows that there are exactly 10 pots being prayed for he is almost 6 times more likely to be completely right than the binomial distribution leads us to believe.

J-- just because it's hard to read a post's tone, realize I am enjoying the debate here and not trying to be combative.

That said, of course, you're completely wrong.
:D

The probability of being completely right isn't what we're testing and is so close to zero that it wouldn't matter. Plus, with no feedback, even if one managed to get the first 19 correct, it'd still be a 50/50 guess as to what plant 20 was.

What matters-- my contention-- is of those restricted scenarios where it's 10 and 10, exactly how many of them include the prayed for plant at trial 20 (I'm guessing exactly half). Since p = .50 for each trial whether you know it's 10/10 or not, I'm asserting the independence assumption is not violated.

[bpesta22]

Oh and I am praying for this guy to do it to counteract claus praying against him.

[bokonon]

I'm not going to work out the math, but I do agree that, as stated, it's a 10-trial rather than a 20-trial. He may correctly choose all 10 holy water plants, but for each one he misses, he will also necessarily miss one well water plant. I'm still comfortable with calling 80% (8 of ten correct) or better a win for prayer, and I suspect it will be a win in his mind if he correctly guesses 6 of ten. In that case, you might suggest (or, indeed, have prepared in advance) a second experiment along the lines Hokulele suggests, with only 1 plant in the group being "favored." After all, it's the water he's praying (for/at); the number of plants you subsequently pour it on is not material.

[69dodge]

I could be wrong and would concede if someone could explain how just knowing that it's 10 and 10 gives you any advantage whatsover or clues you in on what to guess for pot 20?

Looking at pot 20 alone, I have no clue as to what I ought to guess about it.

But looking at all twenty pots together, I know that I must make ten "prayed-for" guesses and ten "normal" guesses.

That eliminates a whole bunch of wrong sets-of-twenty-guesses that I might have made, had I not known that the twenty pots were split ten-and-ten.

With no extra conditions, there are 220 = 1,048,576 ways to make twenty binary guesses. But if ten must be "yes" and ten must be "no", there are only $\binom{20}{10}$ = 184,756 ways.

still submitting that it's a 20 trial binomial test-- not a 10 trial one on just the prayed for beans.

I'm saying that it's not either of those. It's not binomial at all. My post #11 contains what I think is the correct formula.

[Jeff Corey]

OK - bpesta, 69Dodge and Jekyll - am I thinking about this wrong? To me, there will be 10 prayed-for pots. So if he can guess all 10 prayed for ones, he gets 10 out of 10. But if he only guesses 5 of the prayed for ones, he gets 5 out of 10. I'm leaving the other 10, the normal ones or the control group, out of his correct/not correct guesses. That's the way I had planned to record the results. (So I would see a trial with only two pots as he either gets it right or wrong - he either chooses the correct prayed for pot, or chooses the wrong pot.)

I'm striving to set this up so that he gets no feedback as he is guessing, not even unconscious signals from Mr. Amapola.

But am I wrong in only considering the 10 that he will have prayed for?...
Yes, you are tossing out data. You need to record how many are correctly and incorrectly identified in each category.
............Prayed For Not Prayed For

Correct ______ _______

Incorrect ______ ________

[bpesta22]

Controversy!

We should pray for guidance or a statistician. I'm going to ask one today, if he's in.

of those restriced 184576 ways, half of those have prayed-for beans at trial 20 and the other half do not. Indeed, for any trial, the same is true. It's a mini-me version of a the more dispersed (non-restricted) binomial, but it's a still binomial. Can you plot the restricted version and see if it's bell shaped?

If it's not still a binomial (will defer to a statistician here) I assert that the test is equivalent to a binomial test.

I agree with JC (hell freezing) that technically it would be inappropriate to analyze only the 10 and that all 20 should be factored in (note for example that 8 of 10 right is a different passing rate than is 15 or 20).

[69dodge]

J-- just because it's hard to read a post's tone, realize I am enjoying the debate here and not trying to be combative.

Likewise.

That said, of course, you're completely wrong.
:D

And . . . likewise.

:p

Since p = .50 for each trial whether you know it's 10/10 or not, I'm asserting the independence assumption is not violated.

But that's not what independence means! It doesn't mean that all the probabilities are the same. It is an extra condition, which might or might not be satisfied, even where all the probabilities are definitely the same.

[bpesta22]

best I can tell, the independence assumption means that the p of getting it right on any trial does not depend on the p of getting it right on any other trial. And, it doesn't, unless you get feedback after every trial.

Sure 10/10 reduces the number of possible distributions, but they're still bell shaped (plotting frequency of occurence by 0-20 correct guesses), with p=.50 for any trial.

[Jeff Corey]

How would this be any different from a test for dowsing with 20 covered buckets, half filled with water? If the water witch was correct without feedback 15 times, wouldn't that be significant with p = .02?

[69dodge]

Can you plot the restricted version and see if it's bell shaped?

..

[69dodge]

How would this be any different from a test for dowsing with 20 covered buckets, half filled with water? If the water witch was correct without feedback 15 times, wouldn't that be significant with p = .02?

How can the dowser be correct 15 times?

If ten buckets actually have water, and ten are guessed to have water, the number of correct guesses, out of twenty, has to be even.

That alone should tell you that something is funny, that the situation is not the usual binomial one.

[GreedyAlgorithm]

best I can tell, the independence assumption means that the p of getting it right on any trial does not depend on the p of getting it right on any other trial. And, it doesn't, unless you get feedback after every trial.

Sure 10/10 reduces the number of possible distributions, but they're still bell shaped (plotting frequency of occurence by 0-20 correct guesses), with p=.50 for any trial.
That's not what the independence assumption means. Try Wikipedia (http://en.wikipedia.org/wiki/Statistical_independence), for instance:
In probability theory, to say that two events are independent, intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs.
It says that if you know the result of a trial, then your probability for the next trial is changed. It does not say that if you know the probability of the results of a trial, then your probability for the next trial is changed.

In general if I have N trials and my guess for each is independently good or bad with probability p=0.5, that's a binomial distribution on the total number of good guesses with mean Np=N/2 and variance Np(1-p)=N/4. As N grows large this will be approximated by a normal distribution N(N/2,N/4).

But if I have N trials and I know that exactly N/2 are of one type and N/2 are of another, then my guess for each is good or bad with probability p=0.5, but my guesses aren't independent. What does this do? The mean total number of good guesses is still Np. My intuition says the variance will be different but I'm not sure. So let's check. I will randomly guess N/2 of the N to be one type and the other N/2 to be the other type. Of the first N/2 I will get k correct with probability (N/2)Ck * (N/2)C(N/2-k) / NC(N/2). Of the second N/2 I will get another k correct because I will get N/2-k of them incorrect due to my first guesses.

Where have I seen that before? Well, it's certainly hypergeometric (http://en.wikipedia.org/wiki/Hypergeometric_distribution) Here m=N/2 and n=N/2. We see that the mean for that is nm/N = N^2/4N = N/4, which is good since we actually get twice the mean correct (k and then k again), for N/4 * 2 = N/2. How about the variance? It's n*(m/N)*(1-m/N)*(N-n)/(N-1). That's N/2*(1/2)*(1/2)*(N/2)/(N-1) or 1/16 * N^2/(N-1). But remember this is the variance of k and we want the variance of 2k, so it's actually 1/4 * N^2/(N-1). As N grows large that's N(N/2,N^2/4(N-1)) which is nearly exactly N(N/2,N/4).

Conclusion (tl;dr)
In the limit these two methods are very, very close to each other. But for smallish N, where approximating with a normal distribution isn't good enough anyway, they are very different distributions. N=2 is smallish; I claim N=20 is smallish.

[bpesta22]

How can the dowser be correct 15 times?

If ten buckets actually have water, and ten are guessed to have water, the number of correct guesses, out of twenty, has to be even.

That alone should tell you that something is funny, that the situation is not the usual binomial one.

He got 7 of 10 "yes" guesses right.

and 8 of 10 "no" guesses right.

15/20 and binomial.

[bpesta22]

Greedy, the independence assumption has nothing to do with whether one's GUESSES are independent; the focus is only on the outcome's probability, as I understand it.

The outcome's probability (whether for example pot 20 is prayed for) is independent of any other trial's p when the selector only knows that it's 10/10 but gets no feedback.

I think it's binomial with a smaller variance, as you point out (less sure about this point than the two above).

[GreedyAlgorithm]

He got 7 of 10 "yes" guesses right.

and 8 of 10 "no" guesses right.

15/20 and binomial.

Um. Let's see. Suppose we line up the buckets afterwards, keeping his guesses tagged to the buckets, with buckets with water first and no water second:

WWWWWWWWWWNNNNNNNNNN

You can't tag those with 7/10 yes guesses right and 8/10 no guesses right.

WWWWWWWWWWNNNNNNNNNN
YYYYYYY YYY

There are the 7 yes guesses right. Notice there are not enough buckets left to get 8 no guesses right. If there are 10 buckets with water out of 20 buckets and "yes" is guessed 10 times, the number of correct guesses must be even.

[GreedyAlgorithm]

I think it's binomial with a smaller variance, as you point out (less sure about this point than the two above).

No, it's not binomial with a smaller variance. As N grows large, the two are very closely approximated by normal distributions, and the second has slightly higher variance (divide by N-1 makes it larger). The second is not binomial. Lots of things are approximated well by normal distributions, that doesn't mean they're the same thing. If it's known that 10 of the 20 are of one type instead of each chosen randomly with p=0.5, then the distribution on number of correct guesses is hypergeometric, not binomial. In fact the Wikipedia page gives conditions on which hypergeometric distributions are well approximated by binomial distributions: "If N and m are large compared to n and p is not close to 0 or 1". But uh oh, now I see that it says also
If n is large, N and m are large compared to n and p is not close to 0 or 1, then [the hypergeometric distribution is well approximated by Φ(np,np(1-p))] where Φ is the standard normal distribution function]
But here m is not large compared to n. Maybe this isn't even well approximated by a normal distribution as N grows large.

[bokonon]

The "covered bucket" example is a different experiment. The only way he could get 15/20 right is if he treats them all independently, and doesn't keep track well enough to have 10 "yes" guesses and 10 "no" guesses. This shouldn't be the case with the plants, because the plants are being compared to each other, and the prayer knows there are 10 of each.
Whatever criteria he uses -- biggest, greenest, bushiest, beaniest, or some combination of factors -- he should be guessing 10 prayer buckets. If he guesses 11 prayer buckets, then yes, the number of "wrong" guesses can be factored in as well, and the total taken from 20 possibilities. If (as should be the case) his "Yes" guesses equal his "No" guesses, every incorrect "Yes" will force an incorrect "No", and you can take either group, or both totaled, without changing the percentages.

[bpesta22]

Greedy, Dodge and Bok

I concede the point re it being impossible to get 15 correct in the scenario outlined above. I guess, unlike flipping coins, where one just might get 20 tails, here the most one can get is 10, as constrained by the experiment.

I still feel the independence assumption is not violated for reasons I posted above (whether the distribution be binomial or hypo allergenic). The guesses might be dependent but the probability on any trial is not (with no feedback provided).

Given that being wrong on the 15/20 thing increases the probability I am wrong here too, I'm still not ready to concede the point. Any help?

Changing your guessing strategy based on knowing that it's 10/10 is one thing, but I see no way where that can change the probability that any pot (even pot 20) was prayed for or not.

Also, why wouldn't this now be a binomial distribution with n=10 (if the above already explained that, my apologies for not noticing).

B

[GreedyAlgorithm]

I still feel the independence assumption is not violated for reasons I posted above (whether the distribution be binomial or hypo allergenic). The guesses might be dependent but the probability on any trial is not (with no feedback provided).
I think you simply have a mistaken idea of what independence is. Independence is a statement about the joint distribution over trials, not about what each trial looks like independently. The definition of independence is that A and B are independent given background information X exactly when P(AB|X) = P(A|X)P(B|X).

So is one pot prayed independent of another pot prayed? We know that P(A|X)=0.5 and P(B|X)=0.5, so is P(AB|X)=0.25?

P(AB|X) = P(A|BX)*P(B|X) = 0.5*P(A|BX). As expected, the condition is that P(A|BX) must still be 0.5 to make them be independent. But it's not, since if B, then there are only 9 prayed pots left out of 19 and P(A|BX) = 9/19. Not independent, even though P(A|X)=0.5.

Also, why wouldn't this now be a binomial distribution with n=10 (if the above already explained that, my apologies for not noticing).
Ah. The explanation is that the guesser is not choosing 1 pot from 10 prayed and 10 not-prayed pots, 10 times. Then it would be binomial with n=10, p=0.5. Instead the guesser is choosing 10 pots known to be distinct from 20 pots. In the first case he could choose the same pot more than once. In this he cannot. That is where the difference from binomial comes in.

[Jeff Corey]

How can the dowser be correct 15 times?

If ten buckets actually have water, and ten are guessed to have water, the number of correct guesses, out of twenty, has to be even.

That alone should tell you that something is funny, that the situation is not the usual binomial one.
Maybe I'm wrong, but if there are ten buckets with water and ten without and the water witch gets 7 right and 3 wrong when there's water and 8 right and 2 wrong when not, he has 15 right. An odd number. Right?

[GreedyAlgorithm]

Maybe I'm wrong, but if there are ten buckets with water and ten without and the water witch gets 7 right and 3 wrong when there's water and 8 right and 2 wrong when not, he has 15 right. An odd number. Right?

If he gets 7 right and 3 wrong when there's water, that means he guessed "water" 7 times and "no water" 3 times. And if he gets 8 right and 2 wrong when there's not water, that means he guessed "water" 2 times and "no water" 8 times. That's a total of "water" 9 times and "no water" 11 times. The only way to get an odd number is to guess "water" a number of times other than 10.

[bpesta22]

I think you simply have a mistaken idea of what independence is. Independence is a statement about the joint distribution over trials, not about what each trial looks like independently. The definition of independence is that A and B are independent given background information X exactly when P(AB|X) = P(A|X)P(B|X).

So is one pot prayed independent of another pot prayed? We know that P(A|X)=0.5 and P(B|X)=0.5, so is P(AB|X)=0.25?

P(AB|X) = P(A|BX)*P(B|X) = 0.5*P(A|BX). As expected, the condition is that P(A|BX) must still be 0.5 to make them be independent. But it's not, since if B, then there are only 9 prayed pots left out of 19 and P(A|BX) = 9/19. Not independent, even though P(A|X)=0.5.


Ah. The explanation is that the guesser is not choosing 1 pot from 10 prayed and 10 not-prayed pots, 10 times. Then it would be binomial with n=10, p=0.5. Instead the guesser is choosing 10 pots known to be distinct from 20 pots. In the first case he could choose the same pot more than once. In this he cannot. That is where the difference from binomial comes in.



See, I don't think the guesser has what you're calling "background info x". Without it, the trials are independent. The guesser knows that it's 10/10. That's it. But, your X appears to be feedback on each trial, which will not be given.

Just to summarize where I'm at, still admitting that I might be wrong:

1) I don't see how a guessing strategy can influence the probability of an outcome. The plants were either prayed for or not. You know that of the 20, 10 fall in each category. You don't know-- unless prayer works-- which 10 are which.

2) There are 20 place markers here. 10 can be filled with prayed for plants; 10 with non prayed for plants. The selector even knows this.

3) It follows that once 10 of either type have been placed, whatever trials remain must be of the other type. I understand and agree (I characterized it as a degrees of freedom deal).

This is the argument that the trials are not independent? Fine, but the guesser is not privy to when in the sequence 10 of one type have occurred (because he's not getting trial by trial feedback). So, he can't use the info to his advantage, which means he can't change the probability on later trials.

similarly:

4) Someone above calculated that there are x number of possible "samples" that meet the requirement in #3 above (the 10/10 deal).

For all of these samples, once 10 of either type have been placed, the remaining trials must be of the other type.

5) I suspect there are exactly as many samples where the trial 20 plant is prayed for as there are where the trial 20 plant is not prayed for (i.e., that the probability at trial 20 is .50, SUMMED across all possible samples that meet the 10/10 requirement).

Given that, I concluded that the trials are independent. Without getting feedback on each trial, there is no X in your posted formula for the guesser to use.

Since any sample of 10/10 is equally likely to be the sample you're asked to judge, and since those samples are normally distributed, it's p = .5 that you get any single trial correct, and that does not depend on how you guessed on prior trials. Nor can you use a sampling without replacement strategy as you don't know the true placement on any trial til after you've guess on all trials.

Thoughts? Thanks!

[69dodge]

best I can tell, the independence assumption means that the p of getting it right on any trial does not depend on the p of getting it right on any other trial.

Ok, but how can we decide whether one thing depends on another? Only by changing the first---if not in reality then at least in imagination---and seeing whether the second changes too.

Suppose I go to the post office to mail a half-ounce letter. I'm told that the postage is 41 cents. From this information alone, can I tell whether the postage depends on the weight of the letter?

No. I have to ask, Would the postage be any different if the letter weighed two ounces?

It doesn't weight two ounces. It weighs half an ounce. That isn't going to change in reality. But, in order to decide whether the postage does or does not depend on the weight, I need to consider an imaginary scenario in which the weight differs from its actual value.

Similarly, to determine whether two events are independent, it is not enough to look at what their probabilities are. I need to look at what the probability of one would be, if the probability of the other were different. In reality, the guesser is not given feedback, but would feedback be useful to him if it were given?

[Jeff Corey]

If he gets 7 right and 3 wrong when there's water, that means he guessed "water" 7 times and "no water" 3 times. And if he gets 8 right and 2 wrong when there's not water, that means he guessed "water" 2 times and "no water" 8 times. That's a total of "water" 9 times and "no water" 11 times. The only way to get an odd number is to guess "water" a number of times other than 10.

How about he guessed 6/4 and 9/1. Oops that's still an odd number.

[69dodge]

1) I don't see how a guessing strategy can influence the probability of an outcome. The plants were either prayed for or not. You know that of the 20, 10 fall in each category. You don't know-- unless prayer works-- which 10 are which.

The guesser is not being judged on the correctness any single guess. He's being judged on the total number of correct guesses. He has no strategy that will improve his chances of guessing correctly about any single plant considered in isolation, but the overall strategy of making ten "prayed" guesses and ten "normal" guesses will tend to increase the total number of correct guesses, compared to the strategy of making twenty independent guesses, e.g. by flipping a coin for each guess.

ETA: I should rephrase this. The expected number of correct guesess is still 10. So, in that sense, it does not "tend to increase to total number of correct guesses". But it increases the probability that the number of correct guesses will be high. (Balancing this, it also increases the probability that the number of correct guesses will be low. The middle cases lose out, then. I think that's right. I should make another graph.)

ETA again: I'm having trouble making a helpful graph. But never mind whether I can give an intuitive explanation. The numbers are what they are. I didn't make them up. I calculated them. I stand by post #11.

2) There are 20 place markers here. 10 can be filled with prayed for plants; 10 with non prayed for plants. The selector even knows this.

3) It follows that once 10 of either type have been placed, whatever trials remain must be of the other type. I understand and agree (I characterized it as a degrees of freedom deal).

This is the argument that the trials are not independent? Fine, but the guesser is not privy to when in the sequence 10 of one type have occurred (because he's not getting trial by trial feedback). So, he can't use the info to his advantage, which means he can't change the probability on later trials.

He can avoid making five "prayed" guesses and fifteen "normal" guesses, for example. That's an advantage. He's eliminated from consideration a sequence of guesses that's sure to be wrong.

similarly:

4) Someone above calculated that there are x number of possible "samples" that meet the requirement in #3 above (the 10/10 deal).

For all of these samples, once 10 of either type have been placed, the remaining trials must be of the other type.

5) I suspect there are exactly as many samples where the trial 20 plant is prayed for as there are where the trial 20 plant is not prayed for (i.e., that the probability at trial 20 is .50, SUMMED across all possible samples that meet the 10/10 requirement).

Yes, that's right.

Given that, I concluded that the trials are independent. Without getting feedback on each trial, there is no X in your posted formula for the guesser to use.

No. Still not independent.

Since any sample of 10/10 is equally likely to be the sample you're asked to judge,

Yes.

and since those samples are normally distributed,

No.

it's p = .5 that you get any single trial correct,

Yes.

and that does not depend on how you guessed on prior trials.

It depends on whether your previous guesses were in fact correct, even though, at the time, you don't know whether they were or not.

Nor can you use a sampling without replacement strategy as you don't know the true placement on any trial til after you've guess on all trials.

In effect, you do use a sampling without replacement strategy, by making exactly ten "prayed" guesses and ten "normal" guesses rather than any other combination.

[GreedyAlgorithm]

See, I don't think the guesser has what you're calling "background info x". Without it, the trials are independent. The guesser knows that it's 10/10. That's it. But, your X appears to be feedback on each trial, which will not be given.
Nope. X is just the background info everyone agrees the guesser has, namely that 10 of the 20 pots use prayed water. It's only there to remind us that the guesser does know something, that he knows and will try if he can to take into account the given information - exactly 10 of 20 are prayed for.

1) I don't see how a guessing strategy can influence the probability of an outcome. The plants were either prayed for or not. You know that of the 20, 10 fall in each category. You don't know-- unless prayer works-- which 10 are which.

2) There are 20 place markers here. 10 can be filled with prayed for plants; 10 with non prayed for plants. The selector even knows this.

3) It follows that once 10 of either type have been placed, whatever trials remain must be of the other type. I understand and agree (I characterized it as a degrees of freedom deal).
Correct on all counts. But also note that "once 10 of either type have been placed, whatever trials remain must be of the other type" is not everything that follows.

This is the argument that the trials are not independent? Fine, but the guesser is not privy to when in the sequence 10 of one type have occurred (because he's not getting trial by trial feedback). So, he can't use the info to his advantage, which means he can't change the probability on later trials.
On each trial, he has a 0.5 chance of being right. Everyone agrees about this. What we don't agree about, apparently, is whether he has a 25% chance of being right twice in a row for the first two trials. I claim he doesn't.

4) Someone above calculated that there are x number of possible "samples" that meet the requirement in #3 above (the 10/10 deal).

For all of these samples, once 10 of either type have been placed, the remaining trials must be of the other type.

5) I suspect there are exactly as many samples where the trial 20 plant is prayed for as there are where the trial 20 plant is not prayed for (i.e., that the probability at trial 20 is .50, SUMMED across all possible samples that meet the 10/10 requirement).
You'd be right on both counts.

Given that, I concluded that the trials are independent. Without getting feedback on each trial, there is no X in your posted formula for the guesser to use.
This is where you're going wrong. (Side note: X is what we agreed his information was. I didn't introduce information you explicitly said he doesn't get, don't worry.) That's not enough to conclude the trials are independent. Just use the simple example of dependent variables we saw earlier this thread - two coins, you know one is heads and the other tails. There are exactly as many samples where the trial 2 coin is heads as there are where the trial 2 coin is tails (i.e. the probability at trial 2 is .50, SUMMED across all possible samples that meet the one-heads/one-tails requirement).

And yet the two trials are not independent. This shows that at least you cannot conclude "Given that" that the trials are independent.

Since any sample of 10/10 is equally likely to be the sample you're asked to judge, and since those samples are normally distributed, it's p = .5 that you get any single trial correct, and that does not depend on how you guessed on prior trials. Nor can you use a sampling without replacement strategy as you don't know the true placement on any trial til after you've guess on all trials.
Yes. It's p=0.5 that you get any single trial correct. Everyone agrees. It doesn't depend on prior trial guessing. Clearly. There is no way possible to have a greater that p=0.5 chance to get any single trial correct. Just like in the two coin example. The point is that guesses are not independent. In the two coin example, you're either going to get 0 or 2 guesses correct. Each single trial has p=0.5 of being correct but you cannot possible get one right and one wrong (given the background information X - not "whether you got trial 1 right", but rather "don't guess heads for both coins, silly").

You have to separate your intuitions about the means, which appear to be correct, from the facts about the variances (and higher moments).

Can we make this easier so that everyone can show explicit calculations? Apparently the 2-coin example was too simple (though it's perfectly fine). Shall we continue the discussion as if there were 4 plants, 2 of which should get prayed-for water? Here's what happens.

Probability of getting k guesses right:
k=0: 1/16
k=1: 4/16
k=2: 6/16
k=3: 4/16
k=4: 1/16

Probability of getting k guesses right:
k=0: 1/6
k=1: 0/6
k=2: 4/6
k=3: 0/6
k=4: 1/6

Are these the two distributions in question or do you have a different one that you're thinking of?

[69dodge]

ETA: [...] So, in that sense, it does not "tend to increase to total number of correct guesses".

Bah. I edited in a typo, instead of editing one out. "to total number" should be "the total number", of course.

[Jekyll]

See, I don't think the guesser has what you're calling "background info x". Without it, the trials are independent. The guesser knows that it's 10/10. That's it. But, your X appears to be feedback on each trial, which will not be given.

Just to summarize where I'm at, still admitting that I might be wrong:

1) I don't see how a guessing strategy can influence the probability of an outcome. The plants were either prayed for or not. You know that of the 20, 10 fall in each category. You don't know-- unless prayer works-- which 10 are which.

2) There are 20 place markers here. 10 can be filled with prayed for plants; 10 with non prayed for plants. The selector even knows this.

3) It follows that once 10 of either type have been placed, whatever trials remain must be of the other type. I understand and agree (I characterized it as a degrees of freedom deal).

This is the argument that the trials are not independent? Fine, but the guesser is not privy to when in the sequence 10 of one type have occurred (because he's not getting trial by trial feedback). So, he can't use the info to his advantage, which means he can't change the probability on later trials.

This is fun...

Ok, let's try a simpler analogous problem.

Assume we have 20 plants and we are going to make either the first 10 holy or the last 10 holy.

If you know this, and you want to make a guess about what's going on, you can either guess:
1) First 10 holy.
or
2)Second 10 holy.

So it's no harder to guess correctly than it is to guess a single coin toss. Given the knowledge of what one plant's status is, we would know exactly what every other plant's status is like. There is no independence here.

Despite this, the bit in bold below is still correct, but your conclusion is wrong.


5) I suspect there are exactly as many samples where the trial 20 plant is prayed for as there are where the trial 20 plant is not prayed for (i.e., that the probability at trial 20 is .50, SUMMED across all possible samples that meet the 10/10 requirement).

Given that, I concluded that the trials are independent. Without getting feedback on each trial, there is no X in your posted formula for the guesser to use.


The problem is, you're using the wrong definition of independence.
Normally people say that if,

$P(X=x)\neq P(X=x|Y=y)$ for any x and y then X and Y aren't independent, because then some knowledge of Y may change the probability that X=x .

You on the other hand are averaging across all possible fixings of Y, when you test for independence:

$P(X=x) = \frac{\sum_{y} P(X=x|Y=y)}{|Y|}$ Which is always going to be true, whether or not X is dependant on Y.

[Jeff Corey]

If he gets 7 right and 3 wrong when there's water, that means he guessed "water" 7 times and "no water" 3 times. And if he gets 8 right and 2 wrong when there's not water, that means he guessed "water" 2 times and "no water" 8 times. That's a total of "water" 9 times and "no water" 11 times. The only way to get an odd number is to guess "water" a number of times other than 10.
I don't think so. It seems to me that it is possible to get any number from 0 to 20 correct. Every guess is either right or wrong. Right?

[GreedyAlgorithm]

I don't think so. It seems to me that it is possible to get any number from 0 to 20 correct. Every guess is either right or wrong. Right?
Here are the assumptions: There are 20 pots. Exactly 10 will be god-pots. You know both of those things but of course don't know which pots are which. Thus you pick 10 at random and guess that they are the god-pots, then guess that the other 10 are not god-pots.

Claim: The number of correct guesses is even.

Proof: Suppose the number of pots which you guessed are god-pots that are actually god-pots is k. Then you guessed (incorrectly) that exactly 10-k normal pots were god pots. Therefore you guessed that the rest of the normal pots were normal pots: 10-(10-k)=k. So you made k correct god-pot guesses and k correct normal pot guesses for a total of 2k correct guesses, which is even.

[bpesta22]

I suspect you guys are right, I just can't get my head around it and need more time to think (and sulk!).

Very educational and jref-mission like.

I shall be back though with another bad argument, I suspect :)

[bpesta22]

Ok, before I go gently into the night, lemme see what this looks like format-wise then come back to it.

4 trials / plants. 2 are Gods 2 are not:


1,2,3,4
t,t,t,t
t,t,t,f
t,t,f,t
t,t,f,f **
t,f,t,t
t,f,t,f **
t,f,f,t **
t,f,f,f
f,t,t,t
f,t,t,f **
f,t,f,t **
f,t,f,f
f,f,t,t **
f,f,t,f
f,f,f,t
f,f,f,f

So, I agree, without the constraint that half be god plants, there's 16 possible placements. With the constraint there's only 6.

Roger that. More later..

[bpesta22]

ok, still thinking. Here's the 6 possible trials of 4 plants where 2 are prayed for and 2 are not:

1,2,3,4
t,t,f,f
t,f,t,f
t,f,f,t
f,t,t,f
f,t,f,t
f,f,t,t

Any series is equally likely to be the one picked for the guesser.

For any of the 4 trials, p = .5 as looking down the columns there are always three t's and three f's.

[GreedyAlgorithm]

ok, still thinking. Here's the 6 possible trials of 4 plants where 2 are prayed for and 2 are not:

1,2,3,4
t,t,f,f
t,f,t,f
t,f,f,t
f,t,t,f
f,t,f,t
f,f,t,t

Any series is equally likely to be the one picked for the guesser.

For any of the 4 trials, p = .5 as looking down the columns there are always three t's and three f's.
Okay, let's take the last step. Those 6 possibilities are also the six ways you could guess. Here's a table of possibilities, numbered:
1,2,3,4
1: t,t,f,f
2: t,f,t,f
3: t,f,f,t
4: f,t,t,f
5: f,t,f,t
6: f,f,t,t
Now let's see how many you get right if you, say, guess pattern 4 when the reality is pattern 6. In that case you'd correctly guess "f" for the first trial and correctly guess "t" for the third trial: 2 correct guesses.
1 2 3 4 5 6
-----------
1 |4 2 2 2 2 0
2 |2 4 2 2 0 2
3 |2 2 4 0 2 2
4 |2 2 0 4 2 2
5 |2 0 2 2 4 2
6 |0 2 2 2 2 4
The expected number of correct guesses is still 2, but as soon as you start some kind of aggregate statistics like "how likely is it you guess at least 2 correctly?" wonkiness appears. The answer to that question is clearly 5/6, but if it was binomial the answer would be 6/16+4/16+1/16 = 11/16 < 5/6. So if you do the math like it was binomial then you'd conclude, if asking that question, that the guesser got luckier (i.e. prayer was more likely to have worked) than he really did.

ETA: Or if you say that half the total correct guesses is binomial in N/2, then you get 2/4+1/4 = 3/4 < 5/6 for the probability of guessing 2*1 or more.

[Hokulele]

I do have a procedural question and I am not sure if it was covered in all the math here. Amapola, were you planning to have him walk down the row and say "Prayed" or "Non-prayed" for each pot? Would you allow him to go back and change his mind? For example, if he uses up his 10 "Prayed" options before reaching the end of the row and one of the last plants looks healthier, can he go back and reassign a pot to "Non-Prayed"? Would this affect the results of the experiment at all? I could see the selection process giving him an excuse regardless of how it was carried out.

Has this already been covered?

[Jeff Corey]

Here are the assumptions: There are 20 pots. Exactly 10 will be god-pots. You know both of those things....
That's the problem here. The second assuption is not necessarily true. As I said in post#39, they would be independent, "especially if he did not know how many were in each group."

[GreedyAlgorithm]

That's the problem here. The second assuption is not necessarily true. As I said in post#39, they would be independent, "especially if he did not know how many were in each group."

Okay, let's look at post 39 then. Here are the relevant parts: 69dodge's setup, and your response.

There are twenty plants. Ten of them were given prayed-for water. Ten were given regular water. The guy knows this; he just doesn't know which are which. He will try to decide which are which.
I am assuming that he will make all his guesses in one sitting and not be given feedback until all guesses are recorded. This would make each guess independent of the previous ones, especially if he were not told how many were in each group.
Yes, he will make all his guesses in one sitting. Yes, he will not be given feedback until all guesses are recorded. No, this would not make each guess independent of the previous ones. Yes, if he were not told (i.e. if the experiment is set up differently than 69dodge stipulates) how many were in each group, then each guess will be independent of the previous ones.

Sure, fine. Change the experiment, change the results. I've got no problem with that. Let's summarize:

20 pots, each chosen to be a god-pot or not independently with probability 0.5, and the guesser does not know this: binomial.

20 pots, each chosen to be a god-pot or not independently with probability 0.5, and the guesser knows this: binomial.

20 pots, a random 10 of which are chosen to be a god-pot and the others not, and the guesser does not know this: binomial.

20 pots, a random 10 of which are chosen to be a god-pot and the others not, and the guesser knows this: not binomial.

ETA: if a random 10 are chosen and the guesser doesn't know, the results are binomial from the standpoint of the guesser. But if the guesser declares he will guess in a certain way (such as always guessing 10 to be god-pots) then the experimenters' probabilities are no longer binomial. Anyone who has both pieces of information can no longer model it with a binomial distribution.

[Amapola]

I do have a procedural question and I am not sure if it was covered in all the math here. Amapola, were you planning to have him walk down the row and say "Prayed" or "Non-prayed" for each pot? Would you allow him to go back and change his mind? For example, if he uses up his 10 "Prayed" options before reaching the end of the row and one of the last plants looks healthier, can he go back and reassign a pot to "Non-Prayed"? Would this affect the results of the experiment at all? I could see the selection process giving him an excuse regardless of how it was carried out.

Has this already been covered?

I've been quoted! *Amapola faints*

I don't see how his changing his mind would affect results but on the other hand some of this math is really over my head... I don't think it would matter, because he would finally have to make his choices (whatever they are) and the results will still (in my view) be pure guess work. (Maybe I shouldn't be so doubtful about the power of prayer! ;)) But the way I am seeing it is, the coin doesn't "know" you changed your mind mid-flip; it will still either be heads or tails no matter how unsure the guesser might be.

[Jeff Corey]

Okay, let's look at post 39 then. Here are the relevant parts: 69dodge's setup, and your response.



Yes, he will make all his guesses in one sitting. Yes, he will not be given feedback until all guesses are recorded. No, this would not make each guess independent of the previous ones. Yes, if he were not told (i.e. if the experiment is set up differently than 69dodge stipulates) how many were in each group, then each guess will be independent of the previous ones.

Sure, fine. Change the experiment, change the results. I've got no problem with that. Let's summarize:

20 pots, each chosen to be a god-pot or not independently with probability 0.5, and the guesser does not know this: binomial.

20 pots, each chosen to be a god-pot or not independently with probability 0.5, and the guesser knows this: binomial.

20 pots, a random 10 of which are chosen to be a god-pot and the others not, and the guesser does not know this: binomial.

20 pots, a random 10 of which are chosen to be a god-pot and the others not, and the guesser knows this: not binomial.

ETA: if a random 10 are chosen and the guesser doesn't know, the results are binomial from the standpoint of the guesser. But if the guesser declares he will guess in a certain way (such as always guessing 10 to be god-pots) then the experimenters' probabilities are no longer binomial. Anyone who has both pieces of information can no longer model it with a binomial distribution.

I was suggesting number 3 as part of the procedure, maybe I should have been most explicit.
I don't get part about the guesser declaring he will guess in a certain way being any different from deciding he will guess in a certain way and not declaring it or having a bias to guess in a certain way and not realizing it.

[GreedyAlgorithm]

I was suggesting number 3 as part of the procedure, maybe I should have been most explicit.
I don't get part about the guesser declaring he will guess in a certain way being any different from deciding he will guess in a certain way and not declaring it or having a bias to guess in a certain way and not realizing it.

The difference is that if he declares it, then someone knows both pieces of information. Probability is a function of the information available to the one computing the probability, so if someone knows both the experimenter's randomization technique and the guesser's randomization technique, it's possible they will compute probabilities differently than if they didn't know one or the other. In scenario 1 the computed probabilities will be the same; in scenario 3 they will not.

Similarly if the guesser does this experiment (scenario 3) a hundred times and each time picks exactly 10 god-pots, there is very good grounds for re-doing the probabilities given that he will guess this way. To be thorough you might set up a different experiment (scenario 1) to see if he is psychically (or whatever) picking up on the fact that there are exactly 10 god-pots. Then if he keeps guessing 10, you know it's a matter of how he likes to guess, and if he doesn't, something fishy is going on. What's happened? Basically the experimenter has found out empirically that the guesser has a non-independent method of guessing, rather than the guesser straight-out just declaring it. Either way it's time to recalculate the probabilities as soon as you know.

[fls]

I originally made my suggestion about assigning God-pots or not based on a die roll (rather than a draw without replacement) in order to avoid the argument that has occupied the last two pages. However, it has been enlightening to now discover who I can and can not trust when it comes to these issues. ;)

Linda

[bpesta22]

I originally made my suggestion about assigning God-pots or not based on a die roll (rather than a draw without replacement) in order to avoid the argument that has occupied the last two pages. However, it has been enlightening to now discover who I can and can not trust when it comes to these issues. ;)

Linda

Meanie :blush:

Given the discussion so far, then, would a better way to do it be to have a series of as many trial as you want where one of the plants is selected at random, and the guy guesses. The plant is replaced and another randomly s