Putting "The Secret" to the test

[rcronk]

I have a few relatives that believe that "The Secret" (Law of attraction) works. I don't think it works. So, all parties involved in the debate are currently heavily biased.

So what I'd like to do is come up with a simple unbiased scientific test for them to try to see if it works or not. I'll explain the test below and I'd like some feedback about my proposed test so I can make sure I'm not missing anything. I'd also like to be able to explain why "The Secret" seems to work because of confirmation bias and I need some help with those terms and definitions in the context of "The Secret".

The test
I will generate with a pseudo-random number generator (because I'm a software engineer geek) a list of 30 pseudo-random values before the test begins. Each value will be either "Positive" or "Negative/Neutral". Each value represents one day (or other unit of time agreed upon) and the test subject will think positively on each day that has a "Positive" next to it, and will either not think about it or think negatively about it on each day that has a "Negative/Neutral" next to it. Each day, the level of success will be recorded. Success must be measured objectively (e.g. amount of money made/lost, number of free bicycles that appear, etc.). The Success or Failure must not be affected directly or indirectly by the test subject (e.g. consciously or subconsciously working harder to market an Internet product on the positive days than they do on the negative days and then counting income from those sales as their success/failure) - it has to be something that is not related to the test subject. At the end of 30 days (or 30 hours, or whatever), I will average the positive and negative days' success/failure results and calculate the standard deviation and see if there's a statistically significant difference between positive thinking and negative thinking days.

Help
1. Did I miss anything in the process or requirements? I wrote them in paragraph form - is there a more scientific form of writing it to make it more easily expressed in a standard way?

2. Is calculating the mean and standard deviation all that would be required to analyze the result and derive a conclusion from it? Do I need to calculate the standard error? Help me out on the statistical requirements needed to show significant difference or not.

3. Confirmation bias is when a person more easily accepts things that match their existing beliefs while being skeptic of (or ignoring) things that don't match their preexisting beliefs, right? This would apply to people who already believe in "The Secret". There's also the issue of remembering the days it worked well and forgetting the days it didn't. Is that behavior also called confirmation bias or does that have a different name?

4. Any other help in setting up and executing the test or explaining the results of the test and why people seem to think it works when it really doesn't would be appreciated. In fact, it would be nice to have them give me a subjective answer to whether they thought "The Secret" worked during those 30 days at the end of it and compare that subjective answer to the objective data and then use the confirmation bias and other explanations to explain the discrepancy.

Thanks in advance for all of your input. In fact, if we could get this test nailed down, we could then have "believers" and "skeptics" run the test whether they be members of the forum (skeptics) or friends of members of the forum (believers) to run the test and post their results.

[jimtron]

One problem I foresee: if believers in The Secret take your test, and don't see results, they'll say, "it would have happened, but the test wasn't long enough." Or "maybe I didn't get what I was wishing for, but maybe there's something even better that my positive thoughts are going to get me--maybe something down the line. Or maybe I would have got hit by a car, but didn't as a result of positive thinking."

Etc. My experience is that people who believe in this kind of stuff want to believe, regardless of evidence. I would be curious to hear what the author says though, about how long it should take to get what you wish for, and if there's any way to test this. Or what evidence she has that it really works. And why she was able to reveal the secret when so many others for kept it secret for centuries.

I'd be curious though, to see what your relatives say about taking a test like this. Have you asked them yet?

[rcronk]

I think the test length and measurements can be figured out beforehand and adjusted if necessary. I'm really not looking to change any true believers' minds. I agree that they most likely won't be swayed. I would like to affect fence sitters though, and a side effect would be that it would get them to put up or shut up - run the test and show me the money, or let's talk about politics or something useful instead of "The Secret".

I know of one relative who is a true believer who may not take the test, but I have another relative who is a fence sitter who has been receptive to some other skepticism and so I think he would be willing to run it and may even be swayed by the results. I think his belief is more of a general "karma" or "do unto others as you would have them do unto you" or "you reap what you sew"-style belief in "The Secret" though. I wanted to run it by the forum first and refine it and get it right before even asking them though.

Do you have any help or answers to my OP questions? Thanks!

[Limbo]

rcronk I think you should take the test as well, along with everyone else.

[Delvo]

The problem is that the book/DVD actually doesn't set them up to expect anything quite so stark as bicycles magically appearing, even though the video seems to show it happening; the words that the people in it say are different. They describe the process in a more subtle way, like Christians saying that the Lord works in mysterious ways.

For example, one guy's story is that he thought a lot about a certain house he saw in an aerial photograph in a magazine or some source like that. It represented financial success to him. But even according to "The Secret", he didn't just wake up in that house one day with it having magically become his. He worked on running his business for the following years, increased his wealth significantly, paid his bills regularly, and developed the kind of credit rating and account balances that enabled him to eventually sign a mortgage. By this time, he'd put the photograph away and forgotten about it, or at least its details, but the next time he took it out again, he realized that it was the view from above of exactly the same house he was now living in.

They know not to expect world-changing flashes of light to happen before their eyes and leave their manifested dreams immediately sitting in front of them. They know to expect the results to take unpredictable amounts of time and come from process which look mundane to the faithless, such as a man getting a fancy new house after several years of successful business management and responsible financial behavior. It's just that they'll look at those mundane processes and interpret them as being driven by some unseen mystical force in the background, or they'll look at the lack of success yet and say something's in the works to bring it along later. So disproving stark, instantaneous miracles won't affect them because it's disproving something they already don't think anyway.

[Delvo]

PS: Unless you follow a subject around all day judging whether it's positive or negative yourself, you have no source of that judgement other than what they say, and they already know whether they're supposed to think of positive or negative things, so they know what results they're supposed to get.

[rcronk]

I have more faith in the test subjects than you do. They already have some claimed daily objective (monetary) results that should be easy to work with. And sure, I'll likely take the challenge too just for fun.So, can anyone help me with the questions in the OP? Thanks.

[Hokulele]

I have more faith in the test subjects that you do. They already have some claimed daily objective (monetary) results that should be easy to work with. And sure, I'll likely take the challenge too just for fun.So, can anyone help me with the questions in the OP? Thanks.


If that's the case, you may want to keep it simple (and quantifiable) and just focus on the money aspect. Keep a financial diary for the "Positive" and "Neutral/Negative" dates and see if there is any correlation. Don't worry about magic bicycles or the like, just dollars and cents. For the financial diary, the more complete they can make it, the better. For example, if they do put X numbers of hours into marketing a product, have them record that as well as the dollars made. With any luck (ha ha!), they will see how what they had been attributing to "The Secret" can often be attributed to hard work and personal effort.

Hmm, thinking further on this, maybe have two variables and see how they correlate with income. Have one set of numbers generated for "The Secret" and another for "working your butt off". Both should be randomized so that there are days where you do one, the other, both, or neither. See which type of day fares best!

[rcronk]

I'm going to put forth a theory that the "work your butt off" variable will win. Thanks for the ideas.

I really need help on the rest of the OP questions too. Any statistics wizards here?

[Hokulele]

I'm going to put forth a theory that the "work your butt off" variable will win.


Well, that was kind of the point of the suggestion. ;)

[fls]

Can you give some specific example of things your relatives point to as evidence that it works?

Can the effects be transferred - i.e. does it work if someone has positive thoughts for someone else? That would make it much easier to blind.

Linda

[creativecritter41]

I watched the Secret years ago and thought.. hmmmm. Yet, here I am without my great wealth. lol.

Not too long after watching the movie, I was in a grocery store and had stepped into one of the dreaded lines that just weren't moving. There was a shorter line to my right with a gentleman that I could see was getting really irritated at having to wait in his equally slow line. With the movie still fresh in my mind, I found myself thinking about the Secret.

I thought, ok, I won't get any more ticked off than I am getting and just let it happen. I won't focus on the negative that is happening. (according to the Secret, that only helps bring forth the mojo) I actually felt calmer and even though my line must have had three or four more ppl in front of me, I still managed to get my few items while that poor angry guy was still waiting. He did manage a laugh when I said I beat him even with my longer line.

The Secret at work, or a better cashier..? That is my only brush with the Secret. I still can't manage my road rage.. why, oh why do slow people want to drive in front of me and why, oh why do fast people want to drive behind me?!?

:)

[WDB]

If there is no way of making the experiment blind, for example as suggested someone else thinks positive thoughts on behalf of the subject then I don't think you can really consider any of the results to be significant.

[Ladewig]

Each value represents one day (or other unit of time agreed upon) and the test subject will think positively on each day that has a "Positive" next to it, and will either not think about it or think negatively about it on each day that has a "Negative/Neutral" next to it.

If I were a believer, I would be too excited about the philosophy to not think about it for an entire day.

[ScottXSI]

How about you just go find a bunch of homeless people, Junkies, people dying of Aids or people starving to death in a war torn nation then introduce them to the concepts of the secret, then keep track of them for a year to see if they find places to live and get jobs, kick their habits, not die or have the war end and food to be plentiful. Harsh reality will be the proof. ;)

[rcronk]

The example I have from one potential test subject is that he sets up some marketing on the Internet - like click through ads - and then just leaves it alone for a while and lets it make some money. He said (to the true believer) that he had been thinking positively about it lately and he had made more money than before - "it's working". So, since he's not doing anything from day to day about it except thinking a certain way about it, the test should be blind enough. And since he's already doing it, the test is ready to go - the only thing missing is to sprinkle some random "neutral/negative" days in there and check the $$$ each day correlating it to the positive/negative day types.

So, again, please, can someone help me out with the questions in the OP? I just want to get the experiment set up correctly first and once we have it refined and laid out properly (with the help of all of you who can help me with the questions in the OP), then I'll take it to the test subjects and see what happens. I just need scientist-minded and statistic-minded persons to review my OP and give me some help, that's all.

I'll post the results here. In fact, I'm thinking about tracking a stock and thinking positively or negatively about that stock for 30 units of time and posting the results and doing the calculations here as a test run.

Again, thanks in advance for helping me with the questions in the OP.

[Ladewig]

The example I have from one potential test subject is that he sets up some marketing on the Internet - like click through ads - and then just leaves it alone for a while and lets it make some money. He said (to the true believer) that he had been thinking positively about it lately and he had made more money than before - "it's working". So, since he's not doing anything from day to day about it except thinking a certain way about it, the test should be blind enough. And since he's already doing it, the test is ready to go - the only thing missing is to sprinkle some random "neutral/negative" days in there and check the $$$ each day correlating it to the positive/negative day types.


What if "The Secret" has a lag time; e.g. positive thoughts today lead to positive results tomorrow. I think your suggested experiment is too flawed to produce useful results.

[fls]

If I were a believer, I wouldn't find your experiment persuasive. If there were no differences between days I was supposed to think positive thoughts and days I wasn't, I would simply point out to you that even on neutral days positive thoughts entered my mind.

I think you would be better off setting it up like the prayer studies. You mentioned stocks. Pick a bunch of stocks, randomly assign believers to think positively about half of them, and then compare the performance of the two groups at the end of the week.

Linda

[rcronk]

What if "The Secret" has a lag time; e.g. positive thoughts today lead to positive results tomorrow. I think your suggested experiment is too flawed to produce useful results.

Is that the only flaw? If so, we can adjust the experiment. Bring solutions and suggestions to the table, not just problems. Could we have the test subject tell us what time period to use based on their past results that have worked?

If I were a believer, I wouldn't find your experiment persuasive. If there were no differences between days I was supposed to think positive thoughts and days I wasn't, I would simply point out to you that even on neutral days positive thoughts entered my mind.

I think you would be better off setting it up like the prayer studies. You mentioned stocks. Pick a bunch of stocks, randomly assign believers to think positively about half of them, and then compare the performance of the two groups at the end of the week.

Linda

As I said previously, this experiment isn't supposed to convince true believers. It's for fence sitters - I have a fence sitter relative who I think will take the test. He's been receptive to 9/11 debunking and other debunking before and he's pretty honest and willing to let go of false beliefs so this test is really tailored for him, but it might be useful in general too.

I will think about the stock idea you gave. I'll think positively about it. ;)

So, anybody want to help out on the statistical calculations, or any other of the questions in the OP?

[fls]

As I said previously, this experiment isn't supposed to convince true believers. It's for fence sitters - I have a fence sitter relative who I think will take the test. He's been receptive to 9/11 debunking and other debunking before and he's pretty honest and willing to let go of false beliefs so this test is really tailored for him, but it might be useful in general too.

Then there's no point in asking us the questions in the OP. Ask him to look at your design and tell you what would be persuasive to him. Come back to us once you have worked out the details.

I will think about the stock idea you gave. I'll think positively about it. ;)

So, anybody want to help out on the statistical calculations, or any other of the questions in the OP?

Statistical calculations depend upon the experimental design. Once you have worked out the design with your relative, I would be willing to help.

Linda

[fls]

So, anybody want to help out on the statistical calculations, or any other of the questions in the OP?

You can compare two groups with a t-test (it's a straightforward test using the means and standard deviations).

Confirmation bias means that you pay attention only to those things that confirm your idea, or you only test whether your idea is confirmed. For example, if you were to test the stereotype that black people like fried chicken, you might survey a bunch of black people and ask if they liked fried chicken. A better test would be a survey of a more general population to see whether most people like fried chicken, regardless of race.

Linda

[rcronk]

Then there's no point in asking us the questions in the OP. Ask him to look at your design and tell you what would be persuasive to him. Come back to us once you have worked out the details.



Statistical calculations depend upon the experimental design. Once you have worked out the design with your relative, I would be willing to help.

Linda

Fair enough - I'll talk to him about what he thinks.

I was looking here at the forum to make sure my test was scientifically sound, not to find out what would convince him. I want to know scientifically if my test as outlined above has holes in it or if I'm not considering all variables, etc. That's all.

In the mean time, would you be willing to help me with the statistical calculations based on what's been described so far? i.e. He has a randomized list of positive/negative days and a dollar amount for each day for 30 days. Thanks.

[rcronk]

You can compare two groups with a t-test (it's a straightforward test using the means and standard deviations).

Confirmation bias means that you pay attention only to those things that confirm your idea, or you only test whether your idea is confirmed. For example, if you were to test the stereotype that black people like fried chicken, you might survey a bunch of black people and ask if they liked fried chicken. A better test would be a survey of a more general population to see whether most people like fried chicken, regardless of race.

Linda

Thanks! I looked up t-test (http://en.wikipedia.org/wiki/Student's_t-test) on wikipedia and it looks like if I can get the positive and negative days to be 15 each (30 total), I can use the simpler calculation:

http://upload.wikimedia.org/math/4/7/3/473a40b099857f4a3e808f9b3fac3fcf.png

where

http://upload.wikimedia.org/math/9/8/3/983aae66d428df8ec00e7abdd7abc2f1.png

Are these the calculations of which you speak? If so, let's assume we have 15 positive days with a mean of $50 and a standard deviation of $5 and 15 negative days with a mean of $51 and a standard deviation of $6. The X-bar1 is one mean, the X-bar2 is the other mean and so I'm subtracting those on the top, then I'm dividing that by uh - what's S^2x1? Sorry, I'm statistically challenged. Do we need the actual data to find the answer or are the two means and two deviations good enough? Let me know and I think with one good example, I'll be ok.

It did mention an excel add-on and I gave it the following limited sample:


positives negatives
$14 $14
$15 $16
$13 $15
$16 $13
$15 $14
$15 $15

and it generated the following information. How do I interpret it?

t-Test: Paired Two Sample for Means
Positives Negatives
Mean 14.667 14.5
Variance 1.067 1.1
Observations 6 6
Pearson Correlation -0.369
Hypothesized Mean Difference 0
df 5
t Stat 0.237
P(T<=t) one-tail 0.411
t Critical one-tail 2.015
P(T<=t) two-tail 0.822
t Critical two-tail 2.570

[Ladewig]

Is that the only flaw?

No. Fls pointed out the other one. Secret believers hold that even subconscious thoughts influence results. So no matter what time period you use, some positive thoughts might creep in (consciously or unconsciously) on neutral days and thus produce positive results.

Bring solutions and suggestions to the table, not just problems.

I have no idea how to construct a good test of "The Secret," I just know that the one proposed in the opening post has its problems.

[fls]

Thanks! I looked up t-test (http://en.wikipedia.org/wiki/Student's_t-test) on wikipedia and it looks like if I can get the positive and negative days to be 15 each (30 total), I can use the simpler calculation:

http://upload.wikimedia.org/math/4/7/3/473a40b099857f4a3e808f9b3fac3fcf.png

where

http://upload.wikimedia.org/math/9/8/3/983aae66d428df8ec00e7abdd7abc2f1.png

Are these the calculations of which you speak? If so, let's assume we have 15 positive days with a mean of $50 and a standard deviation of $5 and 15 negative days with a mean of $51 and a standard deviation of $6. The X-bar1 is one mean, the X-bar2 is the other mean and so I'm subtracting those on the top, then I'm dividing that by uh - what's S^2x1? Sorry, I'm statistically challenged.

The t-test gives you a sort of standardized version of the difference between the means of two groups.

S is the pooled standard deviation (a sort of average). Standard deviation is simply the square root of the variance. Pooled standard deviation is the square root of the pooled variance. And the pooled variance, when the two groups are the same size, is just an average of the two variances. In this case, square the two standard deviations (to get the variances), take the average of the two numbers and then take the square root.

(25+36)/2=30.5

square root of 30.5=5.52

S=5.52

n=15 (the number of measurements (days) in each group)

So the top is the difference between the averages (51-50=1).

And the bottom is the standard error of that difference (5.52xsqrt(2/15)=2.02).

The t-value is 1/2.02=0.50

The larger the t-value, the more significant (unusual due to chance) the difference.

You then look this value up on a t-table of critical values (http://www.sjsu.edu/faculty/gerstman/StatPrimer/t-table.pdf) to find out just how significant the value is. Start with the degrees of freedom (df), which is n-1 or 15-1=14. Look across the row next to the 14 until you find a value larger than our t-value of 0.50. In this case, it's the first column (ignore the column of zeros). The number at the top of the column (0.50 for two-tailed testing, 0.25 for one-tailed testing) is the minimum probability that you would see this difference due to chance. So we can see that this t-value isn't at all unusual, since the probability is greater than 0.50 that we would see a difference of this size (or smaller) due to chance.

Do we need the actual data to find the answer or are the two means and two deviations good enough?

The means and standard deviations are good enough (along with the sample size).

Let me know and I think with one good example, I'll be ok.

It did mention an excel add-on and I gave it the following limited sample:


positives negatives
$14 $14
$15 $16
$13 $15
$16 $13
$15 $14
$15 $15

and it generated the following information. How do I interpret it?

t-Test: Paired Two Sample for Means
Positives Negatives
Mean 14.667 14.5
Variance 1.067 1.1
Observations 6 6
Pearson Correlation -0.369
Hypothesized Mean Difference 0
df 5
t Stat 0.237
P(T<=t) one-tail 0.411
t Critical one-tail 2.015
P(T<=t) two-tail 0.822
t Critical two-tail 2.570

You should be able to interpret this, now. Is the t-value significant (unusual)?

Linda

[rcronk]

No. Fls pointed out the other one. Secret believers hold that even subconscious thoughts influence results. So no matter what time period you use, some positive thoughts might creep in (consciously or unconsciously) on neutral days and thus produce positive results.

I have no idea how to construct a good test of "The Secret," I just know that the one proposed in the opening post has its problems.

Understood. Thanks.

The t-test gives you a sort of standardized version of the difference between the means of two groups.

S is the pooled standard deviation (a sort of average). Standard deviation is simply the square root of the variance. Pooled standard deviation is the square root of the pooled variance. And the pooled variance, when the two groups are the same size, is just an average of the two variances. In this case, square the two standard deviations (to get the variances), take the average of the two numbers and then take the square root.

(25+36)/2=30.5

square root of 30.5=5.52

S=5.52

n=15 (the number of measurements (days) in each group)

So the top is the difference between the averages (51-50=1).

And the bottom is the standard error of that difference (5.52xsqrt(2/15)=2.02).

The t-value is 1/2.02=0.50

The larger the t-value, the more significant (unusual due to chance) the difference.

You then look this value up on a t-table of critical values (http://www.sjsu.edu/faculty/gerstman/StatPrimer/t-table.pdf) to find out just how significant the value is. Start with the degrees of freedom (df), which is n-1 or 15-1=14. Look across the row next to the 14 until you find a value larger than our t-value of 0.50. In this case, it's the first column (ignore the column of zeros). The number at the top of the column (0.50 for two-tailed testing, 0.25 for one-tailed testing) is the minimum probability that you would see this difference due to chance. So we can see that this t-value isn't at all unusual, since the probability is greater than 0.50 that we would see a difference of this size (or smaller) due to chance.

The means and standard deviations are good enough (along with the sample size).

You should be able to interpret this, now. Is the t-value significant (unusual)?

Linda

Thanks! That was extremely helpful! From what I read about one-tailed and two-tailed tests, it would seem this is a two tailed test since either mean could be larger. In that case, I assume that we're looking at the t critical two tail value (is that the right one?) of 2.570 and since we have 6 samples, we have 5 degrees of freedom and that means, looking up that value on the table, we have a 5% chance this is due to random chance - so this would seem like a significant difference. I'm surprised that the excel add-on didn't look up the value in the table (I assume those are calculated) and give me the % chance as well.

I will make a simple spreadsheet that will take two sets of values and give the % chance as output. Is there a formula that would give me the % chance when given the t value and the degrees of freedom as inputs? Thanks again for your help.

[rcronk]

Ok, to make sure I'm understanding the process, I have made a spreadsheet that calculates the t-test values for the above data sets instead of having the excel add-on do it for me and I'm coming up with different values. Here's what I have so far:


positive data set: 14, 15, 13, 16, 15, 15
negative data set: 14, 16, 15, 13, 14, 15

positive mean: 14.667
positive variance: 1.067
negative mean: 14.500
negative variance: 1.100

difference of mean: 0.167 (top of t-test fraction)

average of variance: 1.083
S (pooled stdev): 1.041 (sqrt(average of variance))
S * sqrt(2/6): 0.601 (bottom of t-test fraction)

t-test value: 0.277 (diff of mean / S * sqrt(2/6))


The 0.277 doesn't match anything in the add-on. Which is right? Where did I go wrong?

[fls]

In that case, I assume that we're looking at the t critical two tail value (is that the right one?) of 2.570 and since we have 6 samples, we have 5 degrees of freedom and that means, looking up that value on the table, we have a 5% chance this is due to random chance - so this would seem like a significant difference. I'm surprised that the excel add-on didn't look up the value in the table (I assume those are calculated) and give me the % chance as well.

What you were looking at is the value from the table you need to exceed (i.e. the critical value) for a p=0.05 (Excel was just trying to be helpful), not the calculated t-value from the data set. The calculated t-value is called the "t stat" and was 0.237. The p-value or percent chance was the "P one-tail" of 0.411 and the "P two-tail" of 0.822.

I will make a simple spreadsheet that will take two sets of values and give the % chance as output. Is there a formula that would give me the % chance when given the t value and the degrees of freedom as inputs? Thanks again for your help.

I hope it's clearer that Excel did that for you.

Linda

[fls]

Ok, to make sure I'm understanding the process, I have made a spreadsheet that calculates the t-test values for the above data sets instead of having the excel add-on do it for me and I'm coming up with different values. Here's what I have so far:


positive data set: 14, 15, 13, 16, 15, 15
negative data set: 14, 16, 15, 13, 14, 15

positive mean: 14.667
positive variance: 1.067
negative mean: 14.500
negative variance: 1.100

difference of mean: 0.167 (top of t-test fraction)

average of variance: 1.083
S (pooled stdev): 1.041 (sqrt(average of variance))
S * sqrt(2/6): 0.601 (bottom of t-test fraction)

t-test value: 0.277 (diff of mean / S * sqrt(2/6))


The 0.277 doesn't match anything in the add-on. Which is right? Where did I go wrong?

It's close to the "t stat" of 0.237. I don't know if the difference is due to differences in rounding?

Linda

[rcronk]

It's close to the "t stat" of 0.237. I don't know if the difference is due to differences in rounding?

Excel should be doing no rounding at all during calculations - only as it displays the values. I was hoping you could calculate the T value yourself and see if it matches my number or the "t stat" number (or neither).

I appreciate your help - this is exactly what I was looking for as far as the statistics help is concerned. Thanks again.

[TMiguel]

ERR! Ok lets start tackle this things.

Despite of what you may thing computer generated random experiments are not random at all. There is no ambiguous mechanism that could allow a random outcome inside a computer. The random function is essentially the access of memory of junk section of memory. You may say it is random because you don’t really know what the outcome will be, but the outcome can manipulated or unwillingly biased depending of what you have been doing before.

The nature of “the secret” fallacy is mainly associated whit the belief that a conscious observer because he is conscious, has any influence what so ever in the outcome of an “ambiguous” experiment (this already tells what it is). What you may not know is that this sort of principle is also used in scientific litter that many are fooled into believe that it is in fact credible science. In fact by wrapping things up in complex scientific background has quantum theory (that most of the people don’t understand) and whit the help whit a misunderstanding of the principle of uncertainty, it has become on of the most scientifically credible nonsense (for illiterate people) so far.

For that to be true an event must have a mechanism of decision in order to respond to the arrangement of brain synapses, determine that they are orientated to that specific event and that the entity having those synapses is a conscious observer (something that is not exactly that well defined) and only then materialize the result and never before then. And even worst it must know before hand that will be experimented by a conscious observer in order act accordingly to any sort of indirect experimentation.

Before trying to solve a problem, one must determine first if there is a problem. How they have come to this conclusion falls apart completely because they can’t define what a conscious observer is, neither establish an unequivocal cause and effect relation nor a reason for preferential relation of reality to them (psychic). And so they can not determine whit any degree of certainty that so can or not be true, consequentially one can determine that such event is manufactured and the result of imagination and so no matter for credibility what so ever.

But to us we already know that it is ********. To make an experiment to debunk such claim is relatively trivial; all it takes is a group of people independently making the same experience of producing more of a determinate event by will. If there is a really cause effect relations all experiences must be successful, if fails (and it will eventually fail) then there is no direct cause and effect relations and so wrong. But I have no doubt that the general result of the experiment will not be much better then what it is expected by chance.

[fls]

Excel should be doing no rounding at all during calculations - only as it displays the values. I was hoping you could calculate the T value yourself and see if it matches my number or the "t stat" number (or neither).

I get 0.277350098. I looked through the formulas Excel uses for the t-test and I didn't see anything fishy. I don't know why the number is different unless Excel rounds differently for my formulas than it does for theirs.

ETA: It was quite a while ago that I read about how Excel handles data, so I don't remember the details. My calculations were somewhat stepwise instead of putting everything into one long formula (if only because it's easy to push the SUM button for some of the steps). I wonder if putting a SUM into a cell and then referring to that cell introduces some rounding (not just in the display, but in how Excel 'remembers' that number)?

Anyway, the important part is that you were able to calculate a t-value on your own!

Linda

[rcronk]

Despite of what you may thing computer generated random experiments are not random at all. There is no ambiguous mechanism that could allow a random outcome inside a computer. The random function is essentially the access of memory of junk section of memory. You may say it is random because you don’t really know what the outcome will be, but the outcome can manipulated or unwillingly biased depending of what you have been doing before.

That's why I called the numbers pseudo-random earlier. Pseudo-random is random enough for this test. All that is really required is that it is completely disconnected from any of the variables in the test.

The nature of “the secret” fallacy is mainly associated whit the belief that a conscious observer because he is conscious, has any influence what so ever in the outcome of an “ambiguous” experiment (this already tells what it is). What you may not know is that this sort of principle is also used in scientific litter that many are fooled into believe that it is in fact credible science. In fact by wrapping things up in complex scientific background has quantum theory (that most of the people don’t understand) and whit the help whit a misunderstanding of the principle of uncertainty, it has become on of the most scientifically credible nonsense (for illiterate people) so far.

For that to be true an event must have a mechanism of decision in order to respond to the arrangement of brain synapses, determine that they are orientated to that specific event and that the entity having those synapses is a conscious observer (something that is not exactly that well defined) and only then materialize the result and never before then. And even worst it must know before hand that will be experimented by a conscious observer in order act accordingly to any sort of indirect experimentation.

Uh, I'm not sure what you're getting at here. Could you summarize?

Before trying to solve a problem, one must determine first if there is a problem. How they have come to this conclusion falls apart completely because they can’t define what a conscious observer is, neither establish an unequivocal cause and effect relation nor a reason for preferential relation of reality to them (psychic). And so they can not determine whit any degree of certainty that so can or not be true, consequentially one can determine that such event is manufactured and the result of imagination and so no matter for credibility what so ever.

I think it's not at deep as you're making it out to be. Relatives of mine are trusting in something that I think is untrue. I'd like to point out to them that it's untrue and I think at least one of them would be willing to run a simple objective test for me that would show it to be untrue. It doesn't really get any deeper than that.

But to us we already know that it is ********. To make an experiment to debunk such claim is relatively trivial; all it takes is a group of people independently making the same experience of producing more of a determinate event by will. If there is a really cause effect relations all experiences must be successful, if fails (and it will eventually fail) then there is no direct cause and effect relations and so wrong. But I have no doubt that the general result of the experiment will not be much better then what it is expected by chance.

I also assume that the result we get will be within the expected results we'd get by chance. My point of posting here is to make sure my test is scientifically sound and get some help with the statistics and I'm getting some help here so I'm thankful.

[rcronk]

I get 0.277350098. I looked through the formulas Excel uses for the t-test and I didn't see anything fishy. I don't know why the number is different unless Excel rounds differently for my formulas than it does for theirs.

ETA: It was quite a while ago that I read about how Excel handles data, so I don't remember the details. My calculations were somewhat stepwise instead of putting everything into one long formula (if only because it's easy to push the SUM button for some of the steps). I wonder if putting a SUM into a cell and then referring to that cell introduces some rounding (not just in the display, but in how Excel 'remembers' that number)?

Anyway, the important part is that you were able to calculate a t-value on your own!

Linda

Yahoo! Thanks for confirming my calculations.

I just did a rounding test and there is no rounding internally to excel on calculations even when referring to a cell that has a format applied to it that shows rounding. The format only affects the display of the number. If I refer to the number in another cell, the full precision is displayed even though the source cell shows rounding because of its format. I hope that made sense. Because it doesn't round, doing it stepwise (which is what I also did) versus doing it all in one big equation shouldn't matter at all and should end up being virtually the same operation.

One more thing. On the wikipedia t-test page, the formula we just used was described as being the formula for "Equal sample sizes, equal variance". Is the "equal variance" part of this description describing the difference in variance of the two data sets? I don't think so, since the formula we just used takes the average of the two variances, implying a difference in variance. So why is it described as "equal variance".

[fls]

Yahoo! Thanks for confirming my calculations.

I just did a rounding test and there is no rounding internally to excel on calculations even when referring to a cell that has a format applied to it that shows rounding. The format only affects the display of the number. If I refer to the number in another cell, the full precision is displayed even though the source cell shows rounding because of its format. I hope that made sense. Because it doesn't round, doing it stepwise (which is what I also did) versus doing it all in one big equation shouldn't matter at all and should end up being virtually the same operation.

Okay. That was what I was wondering.

One more thing. On the wikipedia t-test page, the formula we just used was described as being the formula for "Equal sample sizes, equal variance". Is the "equal variance" part of this description describing the difference in variance of the two data sets? I don't think so, since the formula we just used takes the average of the two variances, implying a difference in variance. So why is it described as "equal variance".

Well, by equal they mean similar. I did try both formulas when I was calculating the number, but it didn't make a difference.

Linda

[dudalb]

What concerns me about "The Secret" is some of it's advocates are edging into a form of faith healing involving "Positive Visualization". That is scary.

[rcronk]

Well, by equal they mean similar. I did try both formulas when I was calculating the number, but it didn't make a difference.

Weird. So do you know what the formula is to generate that t-test table that you linked to? That way I could have two data sets as input and have a percentage chance as output and the t-test would just be another intermediate value. Would this be what the percent value is from excel? Does the percent value in excel calculate the percent chance or does it do a table look up like we had to? I just want to make this as simple as possible - paste in two data sets and a percentage pops up in a cell through formulas I have created. Thanks again.

[rcronk]

What concerns me about "The Secret" is some of it's advocates are edging into a form of faith healing involving "Positive Visualization". That is scary.

I smell a lawsuit if they really get into that side of things.

[Ladewig]

ERR! Ok lets start tackle this things.

Despite of what you may thing computer generated random experiments are not random at all. There is no ambiguous mechanism that could allow a random outcome inside a computer. The random function is essentially the access of memory of junk section of memory. You may say it is random because you don’t really know what the outcome will be, but the outcome can manipulated

They can be manipulated? If I used a spreadsheet program that generated 14 "pseudo" random numbers ranging from one to ten, surely the manipulations required to produce all fives would be more than astronomically complex. I question whether it could be done at all.

or unwillingly biased depending of what you have been doing before.

But even if they were "unwillingly biased" what difference would that make to the task of assigning some days as positive days and assigning all other days as non-positive days?

[Ladewig]

The nature of “the secret” fallacy is mainly associated whit the belief that a conscious observer because he is conscious, has any influence what so ever in the outcome of an “ambiguous” experiment (this already tells what it is). What you may not know is that this sort of principle is also used in scientific litter that many are fooled into believe that it is in fact credible science. In fact by wrapping things up in complex scientific background has quantum theory (that most of the people don’t understand) and whit the help whit a misunderstanding of the principle of uncertainty, it has become on of the most scientifically credible nonsense (for illiterate people) so far.

For that to be true an event must have a mechanism of decision in order to respond to the arrangement of brain synapses, determine that they are orientated to that specific event and that the entity having those synapses is a conscious observer (something that is not exactly that well defined) and only then materialize the result and never before then. And even worst it must know before hand that will be experimented by a conscious observer in order act accordingly to any sort of indirect experimentation.

Before trying to solve a problem, one must determine first if there is a problem. How they have come to this conclusion falls apart completely because they can’t define what a conscious observer is, neither establish an unequivocal cause and effect relation nor a reason for preferential relation of reality to them (psychic). And so they can not determine whit any degree of certainty that so can or not be true, consequentially one can determine that such event is manufactured and the result of imagination and so no matter for credibility what so ever.



Are you sure that "The Secret's" authors included all that in their explanation? I know that that stuff is the core of "What the Bleep Do We Know?" but I wasn't aware that it was part of "The Secret."


But to us we already know that it is ********. To make an experiment to debunk such claim is relatively trivial;

If it really were that trivial, then this thread would not already be 40 posts long.

all it takes is a group of people independently making the same experience of producing more of a determinate event by will. If there is a really cause effect relations all experiences must be successful,

All experiences? What if there is a measurable effect half the time or only 20% of the time. Then the statistics become all the more important in determining if the effect occurs greater than chance.

But I have no doubt that the general result of the experiment will not be much better then what it is expected by chance.


I agree that if the experiment is run long enough then the results will resemble chance, but that does not diminish the usefulness of such a test.

[fls]

The 0.277 doesn't match anything in the add-on. Which is right? Where did I go wrong?

D'oh! I just realized what the problem is. You asked Excel to do a paired t-test instead of a regular t-test. Try running it as a two sample, equal variance type of t-test.

Linda

[Ladewig]

I want to retract my previous statement that constructing a test is a non-trivial matter. An exceptionally simple test would be to plant 12 seeds in a row and tell the Secret believes to use their special positive vibes to make the odd number plants grow taller than the even numbered plants. If one attains positive results at the end of a previously specified time period, then tell them to make the even numbered plants overtake the odd numbered ones. Of course, the person watering and measuring the plants should not know which ones were the target plants.

[TMiguel]

Are you sure that "The Secret's" authors included all that in their explanation? I know that that stuff is the core of "What the Bleep Do We Know?" but I wasn't aware that it was part of "The Secret."


There are some common people involved, might flash some lights on the subject. If you don' think that it's part of the secret then take a closer look.

[Ladewig]

There are some common people involved, might flash some lights on the subject. If you don' think that it's part of the secret then take a closer look.

I just went to the official website and couldn't find anything like that. I also read the Washington Post (http://www.washingtonpost.com/wp-dyn/content/article/2007/04/06/AR2007040601819.html) review and again, didn't see anything like that.

Does anyone reading this thread have a citation for the idea that The Secret is "associated with the belief that a conscious observer because he is conscious, has any influence what so ever in the outcome of an 'ambiguous' experiment"?

. . . . . . . . .

Rcronk, would you be able to convince the others that The Secret is not true by pointing out the offensive conclusions to the Law of Attraction? Specifically, that people dying from terminal illnesses,1 people dying from famines, and people dying from genocidal purges are all attracting those things and if they were just more optimistic then they would not die? One of the experts cited in the book appeared on Nightline and was asked if the starving children of Darfur had "manifested" or "visualized" their own misery. He replied, "I think the country probably has."

_ _ _ _ _ _ _
(1) Actual affirmations from the website: "Know that there is no such thing as incurable."
"As you love completely and feel the joy within you, disease cannot exist."

[fls]

So do you know what the formula is to generate that t-test table that you linked to?

The formula is here:

http://www.answers.com/topic/student-s-t-distribution
http://www.danielsoper.com/statkb/topic08.aspx

Linda

[TMiguel]

So check this video: http://www.youtube.com/watch?v=_b1GKGWJbE8 at 12'36"
and this video: http://www.youtube.com/watch?v=SrKvzPXULME at 3'52"

And see if there something familiar between those two, or am I just imagining things?
What you didn't know is that "The Secret" and "What the bleep do we know" is in fact complementary.

[fls]

Does anyone reading this thread have a citation for the idea that The Secret is "associated with the belief that a conscious observer because he is conscious, has any influence what so ever in the outcome of an 'ambiguous' experiment"?

Here's an example.

http://www.experiencefestival.com/a/Lawof_Attraction/id/5615550
http://www.experiencefestival.com/a/Subconscious/id/22148

Linda

[ExMinister]

I just went to the official website and couldn't find anything like that. I also read the Washington Post (http://www.washingtonpost.com/wp-dyn/content/article/2007/04/06/AR2007040601819.html) review and again, didn't see anything like that.

Does anyone reading this thread have a citation for the idea that The Secret is "associated with the belief that a conscious observer because he is conscious, has any influence what so ever in the outcome of an 'ambiguous' experiment"?

. . . . . . . . .





I just skimmed through the book and the closest I can find to that type of statement was this, from Fred Alan Wolf, "Quantum physics really begins to point to this discovery. It says you can't have a Universe without mind entering into it, and that the mind is actually shaping the very thing that is being perceived."

And, slightly less relevant, from R. Byrne: "I never studied science or physics at school and yet when I read complex books on quantum physics I understood them perfectly because I wanted to understand them. The study of quantum physics helped me to have a deeper understanding of The Secret, on an energetic level. For many people, their belief is strengthened when they see the perfect correlation between the knowledge of The Secret and the theories of new science...In simple terms, all energy vibrates at a frequency. Being energy, you also vibrate at a frequency and what determines your frequency at any time is whatever you are thinking and feeling...As you focus on what you want, you are changing the vibration of the atoms of that thing, and you are causing it to vibrate TO you...The law of attraction says that like attracts like. You are an energy magnet, so you electrically energize everything to you and electrically energize yourself to everything you want."

Mostly though the tone is motivational.

[Ocelot]

I don't think the OP's test will work. From what little I know about the secret it doesn't seem to claim immediate results. The windfall on a negative/neutral day may be attributed to the previousday's positive thoughts.

One alternative. Twelve people pony up £11 of their own money and are given £1. That totals £12 to place on a roulette table with a £1 minimum bet, according to specific betting instuctions

The instructions are
Place just one bet at a time,
Place the minimum £1 bet each time.
Place bets on each spin.
Place the bets in the following sequence.
1st Red
2nd Black
3rd Odd
4th Even
5th High
6th Low
7th Red
8th Black
9th Odd
10th Even
11th High
12th Low

After their 12 bets the players will be allowed to keep any winnings. On average the laws of statistics expects players to lose 32 pence. (Single Zero table)

6 of the players will be practitioners of the secret and must use their power of attraction to maximise their winnings.

6 of the player will be skeptics who belive that the odds will be unaffected by their state of mind and are playing merely for the expected 64 pence profit. They will be required to repeat "This is a mugs game. I'm not going to win. My luck is awful" through their turns.

Each player visits the table seperately ina random order.

Each group of six would be expected to end up with a total of around £70, a loss of £2.

The odds of the secret practitioners ending up with a total of £98 or more, a profit of £26, would be less than one in a thousand. Suitable odds to pass a preliminary test for the MDC. It would clearly uphold the hypothesis that positive thinking brings positive results.

The odds of the secret practitioners ending up with a total of £86 or more, a profit of £14, would be less than one in twenty. A significant result worthy of further examination.

Any less would be considered a null result.

The odds of the 6 skeptics ending up with a total of £42 or less (a loss of £30) would be less than one in one thousand. It would clearly uphold the hypothesis that negative thinking leads to negative results.

The odds that the sceptics would end up with a total of £54 or less (a loss of £18) would be less than one in twenty. A significant result worthy of further examination.

It should be acknowledged that this is effectively two tests in one whihc nearly doubles the odds of getting at least one significant result.

The odds of the skeptics losing at least £18 in total is actually 3.74% and the odds of the believers winning at least £14 in total is actually 3.89% so the combined odds of one, the other or both happening is 7.48% slightly higher than the one in twenty level of significance (closer to 3 in 40)

To fully uphold the message of the secret, both sides must be proven.

[TMiguel]

I just skimmed through the book and the closest I can find to that type of statement was this, from Fred Alan Wolf, "Quantum physics really begins to point to this discovery. It says you can't have a Universe without mind entering into it, and that the mind is actually shaping the very thing that is being perceived."

And, slightly less relevant, from R. Byrne: "I never studied science or physics at school and yet when I read complex books on quantum physics I understood them perfectly because I wanted to understand them. The study of quantum physics helped me to have a deeper understanding of The Secret, on an energetic level. For many people, their belief is strengthened when they see the perfect correlation between the knowledge of The Secret and the theories of new science...In simple terms, all energy vibrates at a frequency. Being energy, you also vibrate at a frequency and what determines your frequency at any time is whatever you are thinking and feeling...As you focus on what you want, you are changing the vibration of the atoms of that thing, and you are causing it to vibrate TO you...The law of attraction says that like attracts like. You are an energy magnet, so you electrically energize everything to you and electrically energize yourself to everything you want."

Mostly though the tone is motivational.

If he really read a serious academic quantum mechanics book, he wouldn’t understand a thing, simply because it uses much more complex mathematics and principles that a scientific illiterate couldn’t even imagine, much less know or understand.
A very common theme between pseudo-scientific claims is vibrations, if I give him a bucket full of vibrations he couldn’t tell what it is, but you know vibrations and all that ambiguous stuff (I don’t know what that is but who cares, seams scientific enough to me).

[rcronk]

D'oh! I just realized what the problem is. You asked Excel to do a paired t-test instead of a regular t-test. Try running it as a two sample, equal variance type of t-test.

Linda

That worked - thanks!

What's the difference between a "paired" and a "regular" t-test?

Also, in the scenario at hand, are we doing a one-tail or two-tail? Why?

Thanks again for all of your help.

Rcronk, would you be able to convince the others that The Secret is not true by pointing out the offensive conclusions to the Law of Attraction? Specifically, that people dying from terminal illnesses,1 people dying from famines, and people dying from genocidal purges are all attracting those things and if they were just more optimistic then they would not die? One of the experts cited in the book appeared on Nightline and was asked if the starving children of Darfur had "manifested" or "visualized" their own misery. He replied, "I think the country probably has."

_ _ _ _ _ _ _
(1) Actual affirmations from the website: "Know that there is no such thing as incurable."
"As you love completely and feel the joy within you, disease cannot exist."

I think a great test would be to inject the person who said that with the HIV virus and let them cure themselves by thinking happy thoughts. They should put up or shut up. Certainly there are advantages to positive thinking in general, but this is silly.

The formula is here:

http://www.answers.com/topic/student-s-t-distribution
http://www.danielsoper.com/statkb/topic08.aspx

Linda

Is this the formula of which you speak:

http://www.danielsoper.com/statkb/img/t_pdf_integral.jpg

If so, I can see why people just use a table. ;)

[Ladewig]

I see that I was wrong in claiming that The Secret does not encompass the idea that observers create reality.

[TMiguel]

I see that I was wrong in claiming that The Secret does not encompass the idea that observers create reality.

I wouldn't belived it either if I haven't seen it already.

[fls]

That worked - thanks!

What's the difference between a "paired" and a "regular" t-test?

You use a paired test when the data are related in some way - like a before and after measurement in the same person, or with the study design I suggested where you measure the movement day-by-day on two sets of stocks. When in doubt, do a regular t-test, as it's valid either way.

Also, in the scenario at hand, are we doing a one-tail or two-tail? Why?

Well, you've put your foot in it now. :)

The effect of one-tailed testing is to double your chances of getting a significant result, if your result happens to lie in a particular direction. However, unless there is effectively (based on the question you are asking) no chance of a large difference in the opposite direction, you have just doubled your chances of getting a false positive. Some people (not me) say that you do a one-tailed test when your hypothesis specifies a particular direction (i.e. whether a drug performs better than nothing, whether positive thinking improves outcomes). Other people (including me) say that you use one-tailed testing sparingly, when the logic of the comparison suggests that any differences will occur in only one direction. The test I use is to ask myself, "if the results come back with an unexpectedly large difference in the opposite direction, am I going to ignore this or am I going to try to use it to modify my explanation. Parapsychologists use one-tailed testing a lot, making it twice as likely they will find 'significant' differences simply due to chance. But the validity of their choice is put to the lie when they use results in the opposite direction to also support the idea of psi (i.e. 'psi missing').

Is this the formula of which you speak:

http://www.danielsoper.com/statkb/img/t_pdf_integral.jpg

Yup.

If so, I can see why people just use a table. ;)

Yup. :)

Linda

[Ladewig]

I don't think the OP's test will work. From what little I know about the secret it doesn't seem to claim immediate results. The windfall on a negative/neutral day may be attributed to the previousday's positive thoughts.

But your test is subject to the same criticism.

If your tests require people to bet red, black, odd, even and the outcome is black, red, black, odd, even, then the believers can say that they just fell behind in the spins and that the target sequence is showing up but it is just lagging behind by one spin. An easier approach is to have the believers root for black for 12 consecutive spins.

[TMiguel]

But your test is subject to the same criticism.

If your tests require people to bet red, black, odd, even and the outcome is black, red, black, odd, even, then the believers can say that they just fell behind in the spins and that the target sequence is showing up but it is just lagging behind by one spin. An easier approach is to have the believers root for black for 12 consecutive spins.

Better yet, make a binary or ternary test based always on the same achievement and discard the first 10 attempts.

[dudalb]

But your test is subject to the same criticism.

If your tests require people to bet red, black, odd, even and the outcome is black, red, black, odd, even, then the believers can say that they just fell behind in the spins and that the target sequence is showing up but it is just lagging behind by one spin. An easier approach is to have the believers root for black for 12 consecutive spins.

And that mentality is why Casinos make so much money off of Slot Machines.

[Ocelot]

But your test is subject to the same criticism.

If your tests require people to bet red, black, odd, even and the outcome is black, red, black, odd, even, then the believers can say that they just fell behind in the spins and that the target sequence is showing up but it is just lagging behind by one spin. An easier approach is to have the believers root for black for 12 consecutive spins.

Perhaps. My thoughts were that they could spend days prior to the event positively visualising, not a specific sequence of spins but a positive financial outcome.

The sequence of bets was put in that way so that a believer might not think that they were being asked to do the impossible - make a certain aspect of the spin repeat 12 times. As we all know, this hotch potch of different types evens bets has exactly the same liklihood of paying results as continually focusing on evens, but the idea is to make a test ammenable to their prior agreement that it would work. The fundamantal difference between this and the OP test is that it compares practitioners with skeptics rather than comparing practitioners who are actively "reinforcing their luck" with practitioners who are still in receipt of luck that they've been working on previously but are currently having a day off.

Yes there will allways be a post hoc rationalisation available to the sufficiently imaginative however this removes at least one of them. Also the idea that a person who sincerly believes that negative thoughts will be harmful will be prepared to take that very action they believe to be harmful is another quibble that this test avoids. You know - "Well I did get a big bonus at work on a neutral negative day, but I think I may have accidentally positively visualised it as I've gotten into the habit so we'll count that as a success."

Honestly the sequence of bets aren't really that important. It's that it's comparing believers vs skeptics rather than beleivers on one day vs beleivers on their "day off"

[Sickly Crypsis]

http://www.youtube.com/watch?v=usbNJMUZSwo

The Chaser "tested" it on "The Chaser's war on everything" (Think of them as the Australian version of the onion)

Quite funny