(Split thread) Criticisms of "Quirkology" studies

[skeptigirl]

Split from: http://forums.randi.org/showthread.php?t=90387

The lying test was a bad one.

http://www.richardwiseman.com/quirkology/lyingexpt.html

No one would find the movie "The Core" their favorite. And the Willis move was a pretty good one. They need another test than wasn't so obvious. No wonder science folks got it right. They would have known how bad "The Core" was.

[skeptigirl]

And this one on how fast city walkers average must not apply the same to individuals.

http://www.richardwiseman.com/quirkology/pace_home.htmpeople in fast-moving cities are less likely to help othersI am a very fast walker and I hate people going slow in front of me if they block my way. But I will almost always stop to help even when I'm driving.

[skeptigirl]

I'm going to state my skepticality at this one on surnames.

http://www.quirkology.com/ (You'll have to click on the link from the home page)

I've seen these kind of studies and with small effects you find a number of studies will have different results. I'm not sure how one determines the margin of error in such a study but I suspect it isn't taken into account when the results support the hypothesis.

And anyone can pick out leaders with names that fit the expected trend. Just because there is a Bush and a Blair doesn't mean you couldn't have just as easily found a Zinni and a Washington.

[skeptigirl]

Not that I didn't enjoy the website mind you. Those comments just came to my skeptical mind as I was perusing the Quirkology website. The video quirks are great fun. Mods made this thread which is fine, but it isn't like I was so annoyed at the website I would have started a thread over it.

[GreedyAlgorithm]

No one would find the movie "The Core" their favorite.
That was my reasoning when I did this.

And the Willis move was a pretty good one.
Not significant; effect completely washed out by <see above>.

[tkingdoll]

Split from: http://forums.randi.org/showthread.php?t=90387

The lying test was a bad one.

http://www.richardwiseman.com/quirkology/lyingexpt.html

No one would find the movie "The Core" their favorite. And the Willis move was a pretty good one. They need another test than wasn't so obvious. No wonder science folks got it right. They would have known how bad "The Core" was.

No no, that's exactly the point. The test subject was given complete control over the movies he chose. This was a test of how good a liar he is.

He chose a really poor 'fake' favourite. That tells us a huge amount about what a good/bad liar he is.

The aliases, stories, examples we choose to weave into our falsehoods are one of the most important parts of lying. That he was unable to come up with a decent cover story is massively important.

The same thing is true in everyday life. Lousy liars are often caught out because they cannot judge what a plausible-sounding lie is.

ETA: this same experiment was also done by Richard a few years ago with Sir Robin Day. He chose extremely convincing movies (I've seen the footage) and I believe the results were 50/50. I will confirm that and let you know though.

[tkingdoll]

And this one on how fast city walkers average must not apply the same to individuals.

http://www.richardwiseman.com/quirkology/pace_home.htmI am a very fast walker and I hate people going slow in front of me if they block my way. But I will almost always stop to help even when I'm driving.

Yeah, I'm not a fan of this study myself. I think all it shows us is correlation. I'm not sure anything particularly meaningful can be derived from any of the correlations though (example, heart disease). Pace of life may be the cause, it may not. We don't know and this study doesn't tell us, it just shows us that in cities where pace of life has increased, so has heart disease. The link is speculation.

Although to be fair it's not wild speculation. I mean, I don't find it unreasonable that people who are always in a hurry/have a fast-paced lifestyle also have heart-related illnesses.

[tkingdoll]

I'm going to state my skepticality at this one on surnames.

http://www.quirkology.com/ (You'll have to click on the link from the home page)

I've seen these kind of studies and with small effects you find a number of studies will have different results. I'm not sure how one determines the margin of error in such a study but I suspect it isn't taken into account when the results support the hypothesis.



That's an interesting question, I will find out. 15,000 is a decent enough sample for me to be comfortable with Richard's study, but I find this one rather odd:

In 1999, Nicholas Christenfeld and his colleagues from the University of California, San Diego, uncovered evidence suggesting that a person's initials might affect perhaps the most important aspect of their life - the moment of their death. Using a large, computerised database of death certificates, they identified people whose initials formed a positive-sounding word (such as A.C.E., H.U.G. and J.O.Y.), and those that had very negative connotations, like P.I.G., B.U.M. and D.I.E. Using factors such as race, year of death and socio-economic status as controls, the researchers discovered that men with positive initials lived approximately four and a half years longer than average, whereas those with negative initials died about three years early.

[Ivor the Engineer]

I take exception to this statement:

Finally, we found that people with a science background were the best lie detectors, followed by visual artists, engineers, and finally mathematicians.

So what do engineers have if not "a science background"?

[tkingdoll]

I take exception to this statement:



So what do engineers have if not "a science background"?

That's referring to the way people self-identified in the study. It should say 'natural or life sciences', to be fair. However, most people think of natural or life sciences when they think of 'science', I suspect, so for the sake of an editorial shortcut I can see why it's there.
The actual questions (asked of New Scientist readers) were these:

In general, I would say that I have a background in one of the natural or life sciences, such as physics, biology, or chemistry.

In general, I would say that I have a background in mathematics.

In general, I would say that I have a background in one of the visual arts, such as painting, drawing, design, or photography.

In general, I would say that I have a background in one of the social sciences, such as psychology or sociology.

In general, I would say that I have a background in engineering.

With a rating for each from 'strongly agree' down to 'strongly disagree'. Hope that clarifies :)

[Ivor the Engineer]

That's referring to the way people self-identified in the study. It should say 'natural or life sciences', to be fair. However, most people think of natural or life sciences when they think of 'science', I suspect, so for the sake of an editorial shortcut I can see why it's there.
The actual questions (asked of New Scientist readers) were these:

In general, I would say that I have a background in one of the natural or life sciences, such as physics, biology, or chemistry.

In general, I would say that I have a background in mathematics.

In general, I would say that I have a background in one of the visual arts, such as painting, drawing, design, or photography.

In general, I would say that I have a background in one of the social sciences, such as psychology or sociology.

In general, I would say that I have a background in engineering.

With a rating for each from 'strongly agree' down to 'strongly disagree'. Hope that clarifies :)

Yep.

Must be all the mathematics engineers have to do that makes them poor at picking out liers.:D

[skeptigirl]

That's an interesting question, I will find out. 15,000 is a decent enough sample for me to be comfortable with Richard's study, but I find this one rather odd:The sample size is big enough but the effect was small according to the Quirk summary. I was referring to whether "small' was significant.

These studies on the effects of the full Moon (http://www.ncbi.nlm.nih.gov/sites/entrez?Db=pubmed&DbFrom=pubmed&Cmd=Link&LinkName=pubmed_pubmed&LinkReadableName=Related%20Articles&IdsFromResult=17630892&ordinalpos=1&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsP anel.Pubmed_RVAbstractPlus) had a few with some 'significant' results, also always rather small. And the same happens with some "clusters" of various things like cancers. The fact 5 people in a building get a rare brain cancer does not always mean something in the building actually caused the cluster. You have to be careful interpreting results when you are looking at rare events and small effects.

[skeptigirl]

No no, that's exactly the point. The test subject was given complete control over the movies he chose. This was a test of how good a liar he is.

He chose a really poor 'fake' favourite. That tells us a huge amount about what a good/bad liar he is.

The aliases, stories, examples we choose to weave into our falsehoods are one of the most important parts of lying. That he was unable to come up with a decent cover story is massively important.

The same thing is true in everyday life. Lousy liars are often caught out because they cannot judge what a plausible-sounding lie is.

ETA: this same experiment was also done by Richard a few years ago with Sir Robin Day. He chose extremely convincing movies (I've seen the footage) and I believe the results were 50/50. I will confirm that and let you know though.While there are better and worse liars, there is also some pretty good science that a number of cues do reveal lying. I don't know how good this reference (http://www.blifaloo.com/info/lies.php) is because I am not an expert on the subject, but it talks about the right things. I have seen it demonstrated by someone who was very good and I have had some success myself by watching certain facial inconsistencies. People really do nod yes while saying no when they lie. But to be really consistent at lie detecting you have to be very observant. Some of the indicators are very subtle.

[CFLarsen]

ETA: this same experiment was also done by Richard a few years ago with Sir Robin Day.

You have a Sir Robin Day?

Do you also have a Sir Bedevere the Strangely Flatulent Day?

[LostAngeles]

Yep.

Must be all the mathematics engineers have to do that makes them poor at picking out liers.:D

In order to find the volume of a red rubber ball, the mathematician finds its radius and then uses 4/3(pi)r^3. The physicist fills up a bucket of water, drops the ball in and measures the displacement. The engineer looks it up in a table of red rubber ball volumes.

:p

Give an engineer a table of liars and a few hours and I'm sure he could find at least one liar in the U.S. Senat...

[tkingdoll]

You have a Sir Robin Day?



We did, he's dead now. But alive, he looked exactly like you would expect:

http://www.museum.tv/archives/etv/D/htmlD/dayrobin/dayrobinIMAGE/dayrobin.jpg

[CFLarsen]

We did, he's dead now. But alive, he looked exactly like you would expect:

http://www.museum.tv/archives/etv/D/htmlD/dayrobin/dayrobinIMAGE/dayrobin.jpg

He doesn't look alive......

[tkingdoll]

He doesn't look alive......

As expected.