IS Forum
Forum Index Register Members List Events Search Today's Posts Mark Forums Read Help

Go Back   International Skeptics Forum » General Topics » Science, Mathematics, Medicine, and Technology
 


Welcome to the International Skeptics Forum, where we discuss skepticism, critical thinking, the paranormal and science in a friendly but lively way. You are currently viewing the forum as a guest, which means you are missing out on discussing matters that are of interest to you. Please consider registering so you can gain full use of the forum features and interact with other Members. Registration is simple, fast and free! Click here to register today.
Tags cognitive dissonance , probability

Reply
Old 9th April 2008, 12:24 PM   #1
GreedyAlgorithm
Muse
 
GreedyAlgorithm's Avatar
 
Join Date: Aug 2005
Posts: 569
Cognitive dissonance studies' mistake

FCP: Free Choice Paradigm
CD: Cognitive Dissonance
Originally Posted by M. Keith Chen
In one of the simplest forms of a FCP, the object of study is the shifts in a subject’s choices. In a recent FCP of this type (Egan, Santos & Bloom 2007), the experiment begins with subjects rating a number of objects on a five-point scale. Then, three objects that are rated equally (say rated 4) are chosen for use in a second stage of the experiment. Note, importantly, that the discreteness of the scale leaves open the possibility that these items might not be perfectly equivalent; for example, a subject may truly rate one of the items 4.1, one 4.26, and one 4.3.
In a second stage then, a subject is asked to choose between a randomly chosen two of these items, say A and B. Calling the object which the subject chooses A, the subject is then asked to choose between B (the initially rejected item), and C (the third item that was rated 4). If subjects are more likely to choose C than B in this choice, they are said to suffer from CD.
I argue that this was to be expected in subjects with no CD. In fact, subjects should be expected to choose good C 66% of the time.
The draft paper can be found at http://www.som.yale.edu/faculty/keit...ogDisPaper.pdf. Chen argues that there is a widespread methodological failure in experimental psychology using the FCP.
__________________
while(true);
GreedyAlgorithm is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 12:37 PM   #2
aggle-rithm
Ardent Formulist
 
aggle-rithm's Avatar
 
Join Date: Jun 2005
Location: Austin, TX
Posts: 15,334
I predict that this will cause cognitive dissonance in the minds of many experimental psychologists, who will resolve the dissonance by questioning the paper's validity.
__________________
To understand recursion, you must first understand recursion.

Woo's razor: Never attribute to stupidity that which can be adequately explained by aliens.
aggle-rithm is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 05:03 PM   #3
Dancing David
Penultimate Amazing
 
Dancing David's Avatar
 
Join Date: Mar 2003
Location: central Illinois
Posts: 39,700
I don't think that would be a suprise, there are a number of artifacts to scaling. But if the variation is not measured in a subscale, then how can you know if the dissonant effect if from a .1 or a .9.

there are other influences I would be much more worried about, the tendency of subjects to not represent what they really think in the first place and the stuff where people always seem to chose the middle value if there is an odd number to the scale. There are many different strategies to cope, expanding scales (say twenty point) , have people mark thier position on a line, rather than assign a number.

You could even try a spider web style of decision making where people tried at actually map the variables that effect the choice.

Self answering surveys are a wonder and a mess at the same time. Such as the Beck depression Invenetory, even given a binary (yes,no) choice people will then put down an answer that does not reflect what they say in a later interview. It is wonderful, wierd and fraught with peril.

So it would not suprise me in the least if scaling has a huge source of hiden error.
__________________
I suspect you are a sandwich, metaphorically speaking. -Donn
And a shot rang out. Now Space is doing time... -Ben Burch
You built the toilet - don't complain when people crap in it. _Kid Eager
Never underestimate the power of the Random Number God. More of evolutionary history is His doing than people think. - Dinwar
Dancing David is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 05:16 PM   #4
sol invictus
Philosopher
 
sol invictus's Avatar
 
Join Date: Oct 2007
Posts: 8,613
Originally Posted by GreedyAlgorithm View Post
Chen argues that there is a widespread methodological failure in experimental psychology using the FCP.
I didn't read the paper, but from the abstract you quoted this is nothing but a form of the Monty Hall problem.

Suppose your true preference goes 1,2,3. You're either presented with 1,2, 1,3, or 2,3 with equal probability. In the first case you will choose B, but in the other two you will choose C (just like switching in the MH problem). The correct odds are 2/3, not 66% - but presumably that's what he means.

If people really wrote papers overlooking such an obvious fact, I'm shocked.

Last edited by sol invictus; 9th April 2008 at 05:17 PM.
sol invictus is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 05:22 PM   #5
CapelDodger
Penultimate Amazing
 
CapelDodger's Avatar
 
Join Date: Sep 2001
Location: Cardiff, South Wales
Posts: 25,102
Originally Posted by aggle-rithm View Post
I predict that this will cause cognitive dissonance in the minds of many experimental psychologists, who will resolve the dissonance by questioning the paper's validity.
Or by breaking out in uncontrollable giggles. It's all good.
__________________
It's a poor sort of memory that only works backward - Lewis Carroll (1832-1898)

God can make a cow out of a tree, but has He ever done so? Therefore show some reason why a thing is so, or cease to hold that it is so - William of Conches, c1150
CapelDodger is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 08:36 PM   #6
GreedyAlgorithm
Muse
 
GreedyAlgorithm's Avatar
 
Join Date: Aug 2005
Posts: 569
Originally Posted by sol invictus View Post
I didn't read the paper, but from the abstract you quoted this is nothing but a form of the Monty Hall problem.

Suppose your true preference goes 1,2,3. You're either presented with 1,2, 1,3, or 2,3 with equal probability. In the first case you will choose B, but in the other two you will choose C (just like switching in the MH problem). The correct odds are 2/3, not 66% - but presumably that's what he means.

If people really wrote papers overlooking such an obvious fact, I'm shocked.
I'd be shocked too. So I read some of the referenced papers. Here's the most blatant failure.

The Origins of Cognitive
Dissonance

Evidence From Children and Monkeys
Louisa C. Egan, Laurie R. Santos, and Paul Bloom

This paper shocks me. Here's the childrens' procedure:
Quote:
Setup:
The experimenter assessed children’s preferences for different
stickers using a smiley-face rating scale that included six
faces ... Each child included in the sample
rated stickers until the experimenter was able to identify at least
two triads of stickers for which the child had equal liking (i.e.,
stickers the child had matched to the same face on the scale).

Procedure:
Once a child had rated the stickers, the experimenter randomly
labeled the stickers in each triad as A, B, and C. The child
was then given choices involving each triad of stickers. Each
child participated in one of two conditions, either the choice
condition or the no-choice condition. In the choice condition, the
child was given one choice between A and B ... Next, the child was given a
similar choice between the unchosen alternative (i.e., eitherAor
B, depending on which option the child had chosen) and C (i.e.,
the novel yet equally preferred alternative) ...
In the no-choice condition, each child received either A or B ... randomly ...
After receiving this sticker, the child was given a
choice between the unreceived alternative (again, either A or B,
depending on which one the experimenter had just given the
child) and the equally preferred alternative, C.

Result:
Children in the choice condition were more
likely to prefer option C (mean percentage choice of C = 63.0%)
than were children in the no-choice condition (mean percentage
choice of C = 47.2%). Average choice of C in the choice condition differed reliably from chance, according to a one-sample
t test with a hypothesized mean of 50%, t(14) 5 2.28, p 5 .04,
two-tailed. This was not true for the no-choice condition, t(14)5
0.53, p 5 .60, two-tailed.
They also did a very similar experiment with capuchins.

That one is clearly knock-down a bad test. But most FCP experiments use a different procedure. The paper argues against this more subtle procedure as well, and to my eyes, convincingly.
__________________
while(true);

Last edited by GreedyAlgorithm; 9th April 2008 at 08:37 PM.
GreedyAlgorithm is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 08:45 PM   #7
bpesta22
Cereal Killer
 
bpesta22's Avatar
 
Join Date: Jul 2001
Posts: 4,942
It does seem like an odd way to measure CD-- note, the problem lies with the researcher then and not necessarily the construct.

Also, unless I'm missing something, the paper hasn't been published. It's just a draft?
__________________
Manifest thy bosoms or decamp.
bpesta22 is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 08:49 PM   #8
GreedyAlgorithm
Muse
 
GreedyAlgorithm's Avatar
 
Join Date: Aug 2005
Posts: 569
Originally Posted by bpesta22 View Post
It does seem like an odd way to measure CD-- note, the problem lies with the researcher then and not necessarily the construct.

Also, unless I'm missing something, the paper hasn't been published. It's just a draft?
Originally Posted by GreedyAlgorithm View Post
The draft paper can be found at http://www.som.yale.edu/faculty/keit...ogDisPaper.pdf. Chen argues that there is a widespread methodological failure in experimental psychology using the FCP.
I realize now that linking to and discussing a draft paper might be socially unacceptable. I don't have any actual knowledge about it, so someone please take me to task if I shouldn't.

Yes, true, you should be able to take the same data and analyze it correctly. But it may mean we need to reexamine quite a lot of CD research.
__________________
while(true);
GreedyAlgorithm is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 08:59 PM   #9
bpesta22
Cereal Killer
 
bpesta22's Avatar
 
Join Date: Jul 2001
Posts: 4,942
I dunno that it's unacceptable to discuss it, but until and unless it gets published in a good journal, it would be premature to slam experimental psychology as a whole (theoretically, if it's that bad-- I didn't read it-- it won't get in a good journal, and the field will then be saved!).
__________________
Manifest thy bosoms or decamp.
bpesta22 is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 09:04 PM   #10
bpesta22
Cereal Killer
 
bpesta22's Avatar
 
Join Date: Jul 2001
Posts: 4,942
I do think it's odd (if I skimmed it right) to base the key manipulation on banking on a test's imprecision (that a 4 might be a 4.3 or a 4.1).

I'd be perhaps more impressed if they used something rated a 3 earlier, but subjects now picked it over a 4, because a prior pick on two 4 items had created dissonance. Then we don't need to get into all this probability crap. That would be fairly striking and less complicated-- jmo.

ETA, I see the guy's a management professor. We do state of the art experimental psych research in management, but it's circa 1972 (that said, I bet his salary's 150,000$ + !).
__________________
Manifest thy bosoms or decamp.

Last edited by bpesta22; 9th April 2008 at 09:07 PM. Reason: to edit
bpesta22 is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 9th April 2008, 09:08 PM   #11
GreedyAlgorithm
Muse
 
GreedyAlgorithm's Avatar
 
Join Date: Aug 2005
Posts: 569
Originally Posted by bpesta22 View Post
slam experimental psychology as a whole
Heh, not that extreme. It's basically only slamming protocols based off of Brehm's 1956 paper's assumptions.
__________________
while(true);
GreedyAlgorithm is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Reply

International Skeptics Forum » General Topics » Science, Mathematics, Medicine, and Technology

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -7. The time now is 12:27 AM.
Powered by vBulletin. Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.

This forum began as part of the James Randi Education Foundation (JREF). However, the forum now exists as
an independent entity with no affiliation with or endorsement by the JREF, including the section in reference to "JREF" topics.

Disclaimer: Messages posted in the Forum are solely the opinion of their authors.