JREF Homepage Swift Blog Events Calendar $1 Million Paranormal Challenge The Amaz!ng Meeting Useful Links Support Us
James Randi Educational Foundation JREF Forum
Forum Index Register Members List Events Mark Forums Read Help

Go Back   JREF Forum » General Topics » Science, Mathematics, Medicine, and Technology
Click Here To Donate

Notices


Welcome to the JREF Forum, where we discuss skepticism, critical thinking, the paranormal and science in a friendly but lively way. You are currently viewing the forum as a guest, which means you are missing out on discussing matters that are of interest to you. Please consider registering so you can gain full use of the forum features and interact with other Members. Registration is simple, fast and free! Click here to register today.

Reply
Old 15th April 2009, 02:55 AM   #1
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
Homeopathic Proving- Walach plays at science again

Harald Walach veers erratically between trying to excuse homeopathy from the scrutiny of controlled trials by proposing ever more convoluted reasons why it still works but can't work in a trial. This is the fridge light bulb theory of homeopathic trials- we just have to believe the light goes out when the fridge door shuts, but we are precluded from drilling a hole in the side of the fridge to check because he says this will instantly stop the fridge from working at all

However, there are also times when he does some science.

Thanks to shpalman, I've come across a new one.

It's hard to know what to make of it.

They report that they went to great efforts at effective blinding.

At a first glance the graphs make interesting reading. The p-values are highly significant.

Those high p-values are odd, though, when they have been derived from non-parametric tests of treatment groups containing 10, 8 and 7 subjects.

"Results
Altogether, 25 volunteers, 6 men and 19 women, with a
mean age of 42.3 years (standard deviation SD = 6.58 years),
all medical doctors, participated in the experiment. 10 were
randomised to receive Natrum muriaticum, 8 to receive Arsenicum
album, and 7 to placebo. They experienced altogether
165 symptoms over the course of 4 days. On average
they reported 6 symptoms when taking Arsenicum album, 5
symptoms when taking Natrum muriaticum, and 11 symptoms
when taking placebo. The results are presented in figure 1,
and a selection of typical symptoms is presented in table 1.
As can be seen, symptoms typical for the respective remedy
were more frequent both in the Natrum muriaticum group
and in the Arsenicum album group, while non-specific symptoms
were more frequent in the placebo group. A non-parametric
Kruskall Wallis analysis showed significant differences
between groups (p = 0.0002), and a pre-planned separate
Mann-Whitney tests confirmed that significantly more specific
symptoms (p < 0.001) were observed in the respective groups
compared with the placebo control group."

Figure 1 can be summarised as below.

Treatment GroupArs AlbNatr MurPlacebo
n8107
No. of Ars Alb Symptoms600
No. of Natr Mur Symptoms04.50
No. of Non-specific Symptoms0010.5
Total Number of Symptoms64.510.5

(Thanks to Here_to_learn I can parse a table properly!)

The phrase, 'too good to be true' does rather spring to mind.

These are "mean" numbers of symptoms, though they then did non-parametric tests. Fig 1 showed markers for those means with 95% confidence intervals, which, of course, necessarily went negative (to approx -2) where the "mean" symptom number was zero. Someone will have to explain to me how you can count objects, get a mean of 0 and have a CI that goes to -2.

Gentlemen (and ladies) start your engines...
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 15th April 2009, 04:15 AM   #2
Yuri Nalyssus
Graduate Poster
 
Yuri Nalyssus's Avatar
 
Join Date: Jun 2004
Location: The Ancient Isle of Avignuon
Posts: 1,316
Originally Posted by Badly Shaved Monkey View Post
Thanks to shpalman, I've come across a new one.

It's hard to know what to make of it.
A Journal called Research in Complementary Medicine edited by Walach, H. with an article in it by Walach, H. saying homeopathy works?

All sounds perfectly pukka and above board to me .

Yuri
__________________
www.rationalvetmed.org/ - because nothing is as good as homeopathy...
Yuri Nalyssus is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 15th April 2009, 04:58 AM   #3
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
Actually, thinking about Fridge Light Bulb Theory a bit more, and like all such things it should be referred to by an acronym to make it sound grander, so FLBT, and, even better, 'flibbet' to real insiders such as ourselves, I realise that if I reverse my metaphor, or make it 'complementary' I more accurately capture the hom's thinking.

So, under 'flibbet', the homs contend that despite all we know of the physics, engineering and design of fridges and the contact switches in the doors, in fact, the light stays on when the door is shut, but that any attempt to test this assumption, for instance by drilling a hole to have a look, will immediately cause the bulb to fail.

There, I think the life has been adequately squashed from that metaphor.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 16th April 2009, 04:28 PM   #4
Graham Jackman
Thinker
 
Join Date: Jul 2007
Posts: 139
It all looks far too neat, with no cross-over in symptoms between the treatments e.g. no-one on "active" treatment showed non-specific symptoms!
I've no interest in paying for the full paper, but we have no idea of what these symptoms were and with an average of 1.65 symptoms per day for each subject over 4 days, I don't imagine these were particularly severe or troublesome
Graham Jackman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 16th April 2009, 04:57 PM   #5
Fnord
Metasyntactic Variable
 
Join Date: Oct 2006
Posts: 6,624
FLBT seems like a special "macro-world" case of the Heisenberg Uncertainty Principle ... at least, as I understand it ("Observing a quantum-level experiment affects the outcome").

Thus, I propose that FLBT be defined as "Attempts to measure the efficacy of a homeopathic substance renders that substance inert and ineffective."
__________________
Belief is the subjective acceptance of a (valid or invalid) concept, opinion, or theory;
Faith is the unreasoned belief in improvable things;
and Knowledge is the reasoned belief in provable things.
Belief itself proves nothing.
Fnord is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 16th April 2009, 06:50 PM   #6
Rolfe
Anti-homeopathy illuminati member
 
Rolfe's Avatar
 
Join Date: Sep 2003
Location: NT 150 511
Posts: 37,868
This is actually a magical property.

I discussed this with an anthropologist a few years ago. He explained that magic must be performed in secret, as otherwise the magic would rebound on the magician. I then suggested that attempts to test magical principles by scientific methods were in effect performing magic out in the open. The corollary of this is that the magic does not want to be revealed, and thus will make very certain it is not revealed. I think I got a footnote citation in his book for that.

Rolfe.
__________________
"The way we vote will depend, ultimately, on whether we are persuaded to hope or to fear." - Aonghas MacNeacail, June 2012.
Rolfe is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 16th April 2009, 09:21 PM   #7
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
Originally Posted by Badly Shaved Monkey View Post
Harald Walach veers erratically between trying to excuse homeopathy from the scrutiny of controlled trials by proposing ever more convoluted reasons why it still works but can't work in a trial. This is the fridge light bulb theory of homeopathic trials- we just have to believe the light goes out when the fridge door shuts, but we are precluded from drilling a hole in the side of the fridge to check because he says this will instantly stop the fridge from working at all

However, there are also times when he does some science.

Thanks to shpalman, I've come across a new one.

It's hard to know what to make of it.
The results don't make sense. Why wouldn't there be non-specific symptoms in all groups?

Quote:
They report that they went to great efforts at effective blinding.
Who judged the symptom diaries and how were they blinded?

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 02:47 AM   #8
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
Originally Posted by fls View Post
The results don't make sense. Why wouldn't there be non-specific symptoms in all groups?



Who judged the symptom diaries and how were they blinded?

Linda
I'm working from my phone so can't paste in the relevant description of the blinding. It sounded good, but could I explain exactly how the data were collated and describe the paper trail in detail? No I could not.

Perhaps Rolfe might extract some (fair use) quotes for us. She has access to the text.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 03:25 AM   #9
Rolfe
Anti-homeopathy illuminati member
 
Rolfe's Avatar
 
Join Date: Sep 2003
Location: NT 150 511
Posts: 37,868
Originally Posted by Badly Shaved Monkey View Post
I'm working from my phone so can't paste in the relevant description of the blinding. It sounded good, but could I explain exactly how the data were collated and describe the paper trail in detail? No I could not.

Perhaps Rolfe might extract some (fair use) quotes for us. She has access to the text.

Mmmm, can I temporarly bow out? I have a week's holiday starting in a few hours, and first I have to get through some work, and then I want to concentrate on clearing and tidying my house, which is what the leave is for. And then I have pressing matters which will be waiting for me when I get back (like two commissioned chapters for a book....). I'm trying not to get caught up in anything detailed online for the next two weeks or so.

Rolfe.
__________________
"The way we vote will depend, ultimately, on whether we are persuaded to hope or to fear." - Aonghas MacNeacail, June 2012.
Rolfe is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 03:31 AM   #10
Mojo
Mostly harmless
 
Mojo's Avatar
 
Join Date: Jul 2004
Location: Nor Flanden
Posts: 25,382
Originally Posted by Fnord View Post
Thus, I propose that FLBT be defined as "Attempts to measure the efficacy of a homeopathic substance renders that substance inert and ineffective."

That doesn't work: homoeopaths and their clients regularly report having observed effects of homoeopathic remedies, so the act of observation evidently doesn't render them inert and ineffective.
__________________
"You got to use your brain." - McKinley Morganfield

"The poor mystic homeopaths feel like petted house-cats thrown at high flood on the breaking ice." - Leon Trotsky
Mojo is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 03:39 AM   #11
Ivor the Engineer
Philosopher
 
Ivor the Engineer's Avatar
 
Join Date: Feb 2006
Location: South Britain, near the middle
Posts: 9,553
Originally Posted by Rolfe View Post
Mmmm, can I temporarly bow out? I have a week's holiday starting in a few hours, and first I have to get through some work, and then I want to concentrate on clearing and tidying my house, which is what the leave is for. And then I have pressing matters which will be waiting for me when I get back (like two commissioned chapters for a book....). I'm trying not to get caught up in anything detailed online for the next two weeks or so.

Rolfe.
You know, sometimes I think you just don't take your online obligations seriously enough.









__________________
My Blog.
Ivor the Engineer is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 04:41 AM   #12
Mikeuk
Student
 
Join Date: Jan 2008
Posts: 34
How does a Homeopath define 'Placebo' -wouldn't that be a lethal cocktail of undiluted chemicals??

Maybe that explains why only 7 in the Placebo group had 11 Symptoms
Mikeuk is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 05:23 AM   #13
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
How much can I quote without getting in trouble?

Quote:
25 out of 59 course participants having given consent were randomised to receive either one of two homeopathic remedies, Natrum muriaticum or Arsenicum album, chosen randomly from a predetermined list of 20 remedies by an independent pharmacist despatching the substances, or an identical placebo... The list of remedies was only known to the study director (HM) and the pharmacist, not to the participants, and not to the study team (HW, RS).

Thus, all participants, the study director (HM) and other personnel involved with the administration of substances were blind as to the precise name of the remedies administered and group allocation. Substances were prepared freshly by Dolisos homeopathic pharmacy... and sent in numbered containers to the study site. The randomisation code was created by an independent researcher (RS) from the random algorithm of RITA (Randomization in Treatment Arms) [Pahlke F, König IR, Ziegler A, Informatik Biometrie Epidemiol Med Biol 2004;35:1–22.]. The code contained three group assignments with random numbers that were used to choose from the containers sent by the pharmacy. The code was kept safely by the study centre. The substances were prepared in CH30... They were then sprinkled over sugar globules, as is the dispensing rule... Placebo was sprinkled with the same amount and batch of alcohol, which is used as the carrier for the homeopathic production process.
It seems as if then all the reported symptoms across all subjects were written down, organized in a "head-to-foot" scheme, and presented to a materia medica expert who used some computer repertorisation programmes to decide which of the reported symptoms were typical of the tested remedies.

Quote:
This procedure yielded our final outcome parameter: the number of symptoms typical for each remedy. This, and the number of untypical symptoms was calculated and submitted to non-parametric statistical testing, using SPSS.
The overall number of all symptoms (per participant, on average) was about 6 in the Arsenicum alb. group, 5 in the Natrum mur. group and about 11 in the Placebo group, but they claim that the symptoms in the two homeopathy groups were specific to those remedies while the symptoms in the placebo group were "non-specific". There's a table giving an example of symptoms reported, but there is no corresponding list of claimed typical repertory symptoms for the remedies.
__________________
--
Opinions expressed in this posting are subject to change without notice
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 05:32 AM   #14
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
Thanks. We also need the paragraph about who did the repertorising.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 05:42 AM   #15
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
Quote:
Finally, a materia medica expert otherwise unconnected with the study was given the list of symptoms, ordered according to the head-to-foot scheme, and the names of the remedies tested. The expert was blind to group allocation, and he was not given the original protocols, but a collated list of symptoms which did not allow him to attribute symptoms to individuals, nor to groups. The expert used a commonly used and commercially available computer repertorisation programme (Synthesis by Archibel, Namur, Belgium) to decide for each symptom, whether it is typical for one of the two remedies tested or not. Note that through this method it was impossible for the materia medica expert to second-guess which volunteer might have produced which series of symptoms. He was strictly bound to make a decision for each individual symptom, irrespective of other symptoms.
Also, from the discussion section:

Quote:
The materia medica expert who determined whether a certain symptom was typical for one of the two remedies or not, was blind as to group allocation, and otherwise not involved with the organisation or conduct of the study. The only way this person could have cheated would have been by having access to the code and coding symptoms, accordingly. This was impossible, as the code existed only in its full version – i.e. allocation of code numbering to remedies – at the research centre and was only fully revealed once the database of symptoms was classified by the materia medica expert and the statistical analysis had been done blindly.
__________________
--
Opinions expressed in this posting are subject to change without notice
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 05:47 AM   #16
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
Thanks

It looks pretty tight. So, how did they achieve zero non-specific symptoms in their remedy groups?? Given what they count as a symptom I can't see how that is even possible.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 05:50 AM   #17
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
I also want to know how a mean symptom count can be 0+/-2
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 05:51 AM   #18
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
Zero plus or minus 2, on average per person in four days: no, I don't know how they get negative symptoms either.
__________________
--
Opinions expressed in this posting are subject to change without notice
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 06:00 AM   #19
Mojo
Mostly harmless
 
Mojo's Avatar
 
Join Date: Jul 2004
Location: Nor Flanden
Posts: 25,382
Originally Posted by Badly Shaved Monkey View Post
It looks pretty tight. So, how did they achieve zero non-specific symptoms in their remedy groups?? Given what they count as a symptom I can't see how that is even possible.

Is there a claim that "proving" symptoms somehow prevent nonspecific symptoms? If not you would expect the same number of nonspecific symptoms in each group, wouldn't you?
__________________
"You got to use your brain." - McKinley Morganfield

"The poor mystic homeopaths feel like petted house-cats thrown at high flood on the breaking ice." - Leon Trotsky
Mojo is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 06:01 AM   #20
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
Originally Posted by shpalman View Post
It seems as if then all the reported symptoms across all subjects were written down, organized in a "head-to-foot" scheme, and presented to a materia medica expert who used some computer repertorisation programmes to decide which of the reported symptoms were typical of the tested remedies.
This part is where we need the details. Was this person blind as to which remedies where given and to whom? Because I have to say that even if they claim she/he was, the results make no sense unless they weren't. Even if you agree that proving is valid, those who took the remedies should have non-specific symptoms in addition to the specific symptoms. The symptom count is similar in the three groups and they differ mostly in their assignment. This suggests that there was not a specific effect of the proving substance, but that there was specific sorting of the effects a posteriori. Also, the small number of symptoms suggests that there must have been considerable overlap with many substances. That the materia medica expert could have narrowed it down to two active substances on the basis of a handful of symptoms seems an impossible task.

Quote:
The overall number of all symptoms (per participant, on average) was about 6 in the Arsenicum alb. group, 5 in the Natrum mur. group and about 11 in the Placebo group, but they claim that the symptoms in the two homeopathy groups were specific to those remedies while the symptoms in the placebo group were "non-specific". There's a table giving an example of symptoms reported, but there is no corresponding list of claimed typical repertory symptoms for the remedies.
A more convincing analysis would have been to give the materia medica expert the information blind and then test whether or not she/he could identify which substances were given to whom.

ETA: Thanks for the info shpalman. I was too slow.

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.

Last edited by fls; 17th April 2009 at 06:06 AM.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 06:12 AM   #21
Mojo
Mostly harmless
 
Mojo's Avatar
 
Join Date: Jul 2004
Location: Nor Flanden
Posts: 25,382
Originally Posted by fls View Post
A more convincing analysis would have been to give the materia medica expert the information blind and then test whether or not she/he could identify which substances were given to whom.

They already tried that; they couldn't:
Quote:
The materia medica expert was not able to determine the correct medicine, either in step 1 (unrestricted choice), or step 2 (restricted choice).

You wouldn't know it from the abstract though.
__________________
"You got to use your brain." - McKinley Morganfield

"The poor mystic homeopaths feel like petted house-cats thrown at high flood on the breaking ice." - Leon Trotsky
Mojo is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 06:13 AM   #22
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
So it's quite simple. The results are most consistent with blinding that has been broken. One only needs to decide whether it should be trusted that it wasn't.

Even if one decides to trust the reporting, it's hard to see how that information can be used to counteract much larger proving trials that show no difference.

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 06:17 AM   #23
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
Originally Posted by Mojo View Post
They already tried that; they couldn't:

You wouldn't know it from the abstract though.
That has always been one of my favourite articles.

That study and other previous studies contradict the results of this study. So even if you are inclined to trust these results, at best the evidence could be considered inconclusive.

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 06:19 AM   #24
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
The materia medica expert was given
Quote:
the list of symptoms, ordered according to the head-to-foot scheme, and the names of the remedies tested.
... so only had to decide whether a given symptom was for Arse. alb. (for example, "After the end of the course unable to remember for a short while where I had parked my car – slight mental disturbance"), Nat. mur. ("Twitching right eye, for 2 minutes"), or neither ("Left ear suddenly free; I had not realised that it was blocked").

You're right that borked blinding is the simplest explanation.
__________________
--
Opinions expressed in this posting are subject to change without notice
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 07:36 AM   #25
Mojo
Mostly harmless
 
Mojo's Avatar
 
Join Date: Jul 2004
Location: Nor Flanden
Posts: 25,382
Here's another one: Can homeopaths detect homeopathic medicines? A pilot study for a randomised, double-blind, placebo controlled investigation of the proving hypothesis

Quote:
In this study a promising trend was observed that symptoms reported by some homeopaths may not be completely attributable to placebo.
If you squint at the results, in the dusk, with the light behind them, there might be something there.

Interestingly, given the method used ("The study design was a double-blinded, crossover trial. It consisted of a 1-week study medication period, a 2-week washout period and a further 1-week on study medication"):
Quote:
There was evidence of an order effect: subjects were much more likely to think they received active Bryonia in the first rather than the second period.

I'd still be interested in knowing whether the three dogs in the Hill et al "trial" got homoeopathy or placebo first.
__________________
"You got to use your brain." - McKinley Morganfield

"The poor mystic homeopaths feel like petted house-cats thrown at high flood on the breaking ice." - Leon Trotsky
Mojo is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 01:53 PM   #26
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
Hmmmm, having made notes to keep track of who did what and who knew what, I notice that the "independent researcher (RS)" who created the randomisation code also
Quote:
did the statistical analysis, wrote parts of the manuscript, participated in discussion of the data and editing of the final manuscript.
So the second author, Rainer Schneider, generated the secret code linking the remedies to the participants and also did the statistical analysis on the results after the expert had collated the symptoms with the remedies but before the blinding had been officially broken.

I'm not accusing anyone of dishonesty, but this strikes me as a possible security hole.
__________________
--
Opinions expressed in this posting are subject to change without notice
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 04:04 PM   #27
Rolfe
Anti-homeopathy illuminati member
 
Rolfe's Avatar
 
Join Date: Sep 2003
Location: NT 150 511
Posts: 37,868
The thing that worries me about this is the difference between unconscious bias and deliberate fraud.

Benveniste didn't try to blind his work at all. The concept simply wasn't in the protocol. And it's the easiest thing in the world to skew a manual white cell count a little bit one way or a little bit the other. If you constantly know which sampe you're counting, it would be easy to do it without realising. That's what Maddox, Sampson and Randi found.

There are situations where an inadvertent unblinding can explain unexpected findings quite adequately. Like the smelly dogs in the herbal remedy paper I mentioned earlier. But in this one, I can't see how an accidental unblinding could cover what seems to have happened. You're talking about someone who knows perfectly well he's not supposed to know which group is which while he's making his decisions about symptoms, and yet he does know. And he goes on deciding nonetheless. You don't do that by accident, even if the fact that you know was due to an accident in the first place.

Rolfe.
__________________
"The way we vote will depend, ultimately, on whether we are persuaded to hope or to fear." - Aonghas MacNeacail, June 2012.
Rolfe is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 04:12 PM   #28
Rolfe
Anti-homeopathy illuminati member
 
Rolfe's Avatar
 
Join Date: Sep 2003
Location: NT 150 511
Posts: 37,868
Another thought, maybe someone else with access to the paper could check it for me.

Three groups. 10 salt, 8 arsenic, 7 placebo. Were these individuals treated as 25 individual people, or as three groups? Did the master prover already know how the patients were grouped before he got going? Because if he did, then he only has to guess a three-way allocation of probability. Which I think might be about a 25% chance anyway, or will someone correct me?

Given that it was a racing certainty that the placebo group would be the smallest one, maybe it was a 50/50 chance. You wouldn't need the blinding to slip much to make it a certainty from there.

I need to read the paper but not right now. Is this idea worth following up?

Rolfe.
__________________
"The way we vote will depend, ultimately, on whether we are persuaded to hope or to fear." - Aonghas MacNeacail, June 2012.
Rolfe is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 17th April 2009, 11:01 PM   #29
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
Two more things have occurred to me.

Firstly, it seems like the blinding was designed to make sure that the materia medica expert would not be able to cheat in the assignment of remedies to participants, since he only received a list of disconnected symptoms. However, I'm not sure exactly who it was who reconnected the symptoms, with their guesses as to which remedies "caused" them, back with the list of participants. But we know that the blinding code, created by RS, "was only fully revealed once the database of symptoms was classified by the materia medica expert and the statistical analysis had been done blindly..." by RS. I think it would have been better if the pharmacist had created the blinding code and not sent it to the study centre until after the statistical analysis.

So there's the possibility that RS could have just changed the which patients certain symptoms were supposed to have come from, once the symptoms were labelled with remedies. I don't know if he could have done this without HM (the lead author, who collected the symptoms from the participants via diaries and interviews) noticing. There's no innocent explanation for how this can have come about that I can think of.

Secondly, there were 7 participants receiving placebo how recorded an average of 11 non-specific symptoms each, so that's about 80 non-specific symptoms out of the reported total of 165. The 10 participants on Nat. mur. had 5 symptoms each so that's about 50; the 8 on Arse. alb. has 6 each so that's also about 50. It adds up to 180 which isn't much over the 165 reported. Given that the materia medica expert knew that what the participants had been given, it seems about right that he would assign the symptoms in these proportions. But then if RS is deliberately reassociating symptoms, now assigned to remedies, with the participants who got those remedies you might indeed get the strange result that all the non-specific symptoms, which you would expect to have been at a similar level for all groups, have been assigned to the placebo group which therefore had double the number of symptoms compared to the groups on homeopathy.

Rolfe, I'm not sure exactly what you mean in your second post. HM, who collected the symptoms, claims to have not known the group allocation or even exactly which pair of remedies out of the possible twenty had been chosen by the pharmacist.

Here are my notes from last evening:

HM: Study director (Heribert Moellinger, first author)
HW: Study designer and overseer, and editor of the journal (Harald Walach)
IP: Independent pharmacist (at Dolisos, Lausanne)
RS: Independent researcher (Rainer Schneider, co-author)
MM: Materia medica expert (Reimund Wagner, acknowledged)

HM produces list of 20 remedies
IP chooses two "at random" to give to selected participants, prepares remedies and placebos and sends them in randomly numbered containers
RS assigns random numbers to participants - "code was kept safely by the study centre"
HM as "proving director" conducts interviews with participants to verify symptoms noted by participants in their diaries
?? collates symptoms across all participants in head-to-foot scheme, sends to MM
MM collates symptoms with the two remedies using software
?? collates symptoms, now labelled according to remedies by MM, with the participants they came from
RS performs statistical analysis of symptom collation
?? breaks blinding
__________________
--
Opinions expressed in this posting are subject to change without notice

Last edited by shpalman; 17th April 2009 at 11:04 PM.
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 02:43 AM   #30
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
How does one perform a statistical analysis before blinding was broken if all you have is a list of symptoms and no way of knowing which symptom goes with which participant or group?

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 03:52 AM   #31
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
Originally Posted by fls View Post
How does one perform a statistical analysis before blinding was broken if all you have is a list of symptoms and no way of knowing which symptom goes with which participant or group?

Linda
They must have broken the blinding before testing because they did pre-specified U-tests after their K-W test, so they had to know which group was which at that point.

But I still don't understand what raw data they used. It seems to be counts of symptoms. So for the Ars Alb test they have 25 separate symptom counts, one for each subject. But we return to the weird fact that for the two non-Ars Alb groups the mean symptom count is zero and somehow they have been able to use this count to calculate a value for 2 SD that means some subjects had a negative number of symptoms. There is no possible list of positive integers that can yield mean=0 and 2SD=2 no matter how skewed, kurtotic or multimodal it might be. This alone suggests that something bloody peculiar is going on.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.

Last edited by Badly Shaved Monkey; 18th April 2009 at 03:56 AM.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 04:55 AM   #32
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
I think I feel a letter to the editor coming on. Just a straightforward comment and request for clarification on the 0+/-2 business. That is nice and specific. All it requires it a brief response from them to show at least one set of the raw data that generated those summary statistics.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 05:04 AM   #33
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
Originally Posted by fls View Post
How does one perform a statistical analysis before blinding was broken if all you have is a list of symptoms and no way of knowing which symptom goes with which participant or group?
The authors were not blinded as to which symptom went with which participant, and after sending the list of symptoms and the names of the remedies to the materia medica expert (who was blind as to which symptoms went with which participant) they knew, in principle, which participants had symptoms typical of which remedies.

Quote:
This procedure yielded our final outcome parameter: the number of symptoms typical for each remedy. This, and the number of untypical symptoms was calculated and submitted to non-parametric statistical testing, using SPSS.
Quote:
... the code existed only in its full version – i.e. allocation of code numbering to remedies – at the research centre and was only fully revealed once the database of symptoms was classified by the materia medica expert and the statistical analysis had been done blindly.
(The only thing that was blinded was the information regarding which participants actually received each remedy.)
__________________
--
Opinions expressed in this posting are subject to change without notice
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 05:14 AM   #34
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
Originally Posted by Badly Shaved Monkey View Post
I think I feel a letter to the editor coming on.
Walach is the editor.

ETA: I hope my blog post won't prejudice matters...
__________________
--
Opinions expressed in this posting are subject to change without notice

Last edited by shpalman; 18th April 2009 at 05:28 AM.
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 05:44 AM   #35
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
Originally Posted by shpalman View Post
The authors were not blinded as to which symptom went with which participant, and after sending the list of symptoms and the names of the remedies to the materia medica expert (who was blind as to which symptoms went with which participant) they knew, in principle, which participants had symptoms typical of which remedies.
HM interviewed the subjects as to their symptoms, not knowing which subject had received which remedy. Somebody takes those symptoms and orders (and presumably codes) them. Each symptom is assigned to one of three treatments, and then someone re-orders them according to subject (e.g. subject 1 had symptoms 16, 23, 72 and 102, three which were coded as Arsenicum and one which was coded as non-specific). Now RS can group the subjects according to how the bulk of their symptoms were labelled (e.g. subject 1 would be grouped with other subjects whose symptoms were mostly labelled Arsenicum), and the table that was given would be consistent with forming groups in that manner (). But prior to breaking blinding, he wouldn't know which subjects would be grouped together based on the treatment they received, and so he wouldn't be able to analyze whether there were significant differences based on treatment received.

The simple answer is that blinding was broken in order to tell him which patients received which treatment. But the point is that it couldn't have happened as described, just like (as BSM points out) an analysis shouldn't yield an average of 0 symptoms with a confidence interval that includes negative numbers if it was performed as described. If we already know that the description has to be wrong on two issues, what else is it wrong about?

This may simply be a misunderstanding, but as it stands, it needs clarification.

And thank you for providing this information for me.

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 06:03 AM   #36
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
I should add that it may be as Rolfe pointed out - that the researchers were given information about which subjects received the same treatment (without knowing what that treatment consisted of). Then it becomes a matter of allocating groups and a one-in-six chance of getting it right.

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 06:12 AM   #37
fls
Penultimate Amazing
 
fls's Avatar
 
Join Date: Jan 2005
Posts: 10,236
Originally Posted by fls View Post
I should add that it may be as Rolfe pointed out - that the researchers were given information about which subjects received the same treatment (without knowing what that treatment consisted of). Then it becomes a matter of allocating groups and a one-in-six chance of getting it right.

Linda
It does say that they were blind as to group allocation and remedies chosen.

So who told the Materia Medica expert to look for symptoms specific to Ars Alb and Natr Mur if none of the researchers had this information?

Linda
__________________
God:a capricious creative or controlling force said to be the subject of a religion.
Evidence is anything that tends to make a proposition more or less true.-Loss Leader
SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine)
Look how nicely I'm not reminding you you're dumb.-Happy Bunny
When I give an example, do not assume I am excluding every other possible example. Thank you.
fls is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 07:02 AM   #38
Badly Shaved Monkey
Anti-homeopathy illuminati member
 
Badly Shaved Monkey's Avatar
 
Join Date: Feb 2004
Location: Outside a banana and far from a razor
Posts: 5,272
Originally Posted by shpalman View Post
Walach is the editor.

ETA: I hope my blog post won't prejudice matters...

Oh, yes!

That's even more funny.
__________________
"i'm frankly surprised homeopathy does as well as placebo" Anonymous homeopath.
"Alas, to wear the mantle of Galileo it is not enough that you be persecuted by an unkind establishment; you must also be right." (Robert Park)
Is the pen is mightier than the sword? Its effectiveness as a weapon is certainly enhanced if it is sharpened properly and poked in the eye of your opponent.
Badly Shaved Monkey is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Old 18th April 2009, 09:30 AM   #39
shpalman
Thinker
 
shpalman's Avatar
 
Join Date: Jun 2007
Posts: 205
Originally Posted by fls View Post
It does say that they were blind as to group allocation and remedies chosen.

So who told the Materia Medica expert to look for symptoms specific to Ars Alb and Natr Mur if none of the researchers had this information?

Linda
Well, that could have been the independent homeopathic pharmacy who did that. But once all the symptoms are written down maybe it doesn't matter so much. The symptoms are going to come back from MM labelled with remedies anyway.
__________________
--
Opinions expressed in this posting are subject to change without notice
shpalman is offline   Quote this post in a PM   Nominate this post for this month's language award Copy a direct link to this post Reply With Quote Back to Top
Reply

JREF Forum » General Topics » Science, Mathematics, Medicine, and Technology

Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -7. The time now is 08:57 PM.
Powered by vBulletin. Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
© 2001-2013, James Randi Educational Foundation. All Rights Reserved.

Disclaimer: Messages posted in the Forum are solely the opinion of their authors.