Homeopathy works after all and here's the proof

[Yuri Nalyssus]

I keep seeing references to this paper by Mathie, 2003 (http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WXX-48HXDX3-8&_user=6619241&_coverDate=04%2F30%2F2003&_alid=955445965&_rdoc=2&_fmt=high&_orig=search&_cdi=7170&_sort=d&_docanchor=&view=f&_ct=3&_acct=C000032518&_version=1&_urlVersion=0&_userid=6619241&md5=7d437ebda2f74a255583d75585b737b4) - The research evidence base for homeopathy: a fresh assessment of the literature, [Homeopathy (2003) 92, 84–91], where it is reported:

"Results. 50 papers report a significant benefit of homeopathy in at least one clinical outcome measure, 41 that fail to discern any inter-group differences, and two that describe an inferior response with homeopathy. Considering the relative number of research articles on the 35 different medical conditions in which such research has been carried out, the weight of evidence currently favours a positive treatment effect in eight: childhood diarrhoea, fibrositis, hayfever, influenza, pain (miscellaneous), sideeffects of radio- or chemotherapy, sprains and upper respiratory tract infection. Based on published research to date, it seems unlikely that homeopathy is efficacious for headache, stroke or warts. Insufficient research prevents conclusions from being drawn about any other medical conditions.
Conclusions. The available research evidence emphasises the need for much more and better-directed research in homeopathy. A fresh agenda of enquiry should consider beyond (but include) the placebo-controlled trial. Each study should adopt research methods and outcome measurements linked to a question addressing the clinical significance of homeopathy’s effects."

Given that interpreting scientific papers is not my strong point :o, could anyone comment on this paper and explain why its conclusions (that homeopathy is effective in certain conditions) seems to fly in the face of the rest of the evidence in more mainstream publications. A lot of homeopaths give this as cast iron evidence that homeopathy works and everyone else is biased against them.

Many thanks, :)

Yuri

[HansMustermann]

Well, the question is whether those 50 papers were properly scientific to start with, i.e., double blind and all, and what was the degree of confidence in their statistic. Otherwise just counting papers doesn't really say much.

To be honest, I've never heard of even a single study which was actually valid in that aspect. Generally they tend to have no control group, for a start.

I'm pretty sure that if the homeopathic gang had even _one_ study which was scientifically acceptable, they'd have had used it as their poster child long ago. Some 50 studies? Why not just quote those then?

Because basically a meta-study counting flawed studies, isn't any more conclusive than those.

[Kuko 4000]

Hans, this is what it reads in the SD link Ivor provided:

Aims.

This review examines the cumulative research from randomised and/or double-blind clinical trials (RCTs) in homeopathy for individual medical conditions reported since 1975, and asks the question: What is the weight of the original evidence from published RCTs that homeopathy has an effect that is statistically significantly different from that in a comparative group?

Method.

Analysis of the 93 substantive RCTs that compare homeopathy either with placebo or another treatment.


Is it just my layman eyes (and brain), but in the Aim part, why didn't they include only randomise AND double blinded clinical trials?

I would also like to know more about the studies that they included.

At the moment, this sounds way too vague for me, but I'm not claiming to be an expert at all. Just thinking aloud here, please correct me if the points I raised are not relevant.

[Mojo]

I suspect that this paper was the foundation for the counts of "positive/inconclusive/negative" RCTs that the British Homeopathic Association/Faculty of Homeopathy put up on their website from time to time. For example here (http://www.emaxhealth.com/60/13121.html) is what looks like a BHA press release from a couple of years after Mathie's 2003 paper (the same figures also cited by Peter Fisher here (http://www.guardian.co.uk/commentisfree/2007/may/23/openletterclosedminds)); here (http://www.britishhomeopathic.org/media_centre/facts_about_homeopathy/the_evidence_base.html) is a BHA page another couple of years later, and here (http://www.britishhomeopathic.org/export/sites/bha_site/research/evidencesummarymay08.pdf) is a FoH document on the BHA's website, bringing it up to the end of 2008. Mathie is variously described as being connected with the BHA and/or FoH (see, for example, the BHA press release about the recent Hill et al. itchy dog paper). Note that all these use the rather odd positive/inconclusive/negative classification of papers (let's face it, an "inconclusive" paper is one that fails to show an effect for homeopathy).

Note that as time progresses, the percentage of "positive" papers decreases, suggesting an improvement in the quality of published trials.

Note also that according to its website (http://www.britishhomeopathic.org/about_us/): The British Homeopathic Association exists to promote homeopathy practised by doctors and other healthcare professionals.


I suspect that the main problem with all of these figures is that they don't pay enough attention to study quality.

Incidentally, if the BHA figures are indeed an extension of Mathie's paper, then since the end of 2002 there have been 45 published RCTs, 10 positive, 27 inconclusive and 8 negative ("negative" being defined in the abstract as "describ[ing] an inferior response with homeopathy").

[Yuri Nalyssus]

I suspect that this paper was the foundation for the counts of "positive/inconclusive/negative" RCTs that the British Homeopathic Association/Faculty of Homeopathy put up on their website from time to time.
I get the impression that this paper is fairly fundamental to the h'paths' claims of anything up to 200 trials in favour of h'pathy, hence my interest.

I suspect that the main problem with all of these figures is that they don't pay enough attention to study quality.
I can't see any description of their exclusion/exclusion criteria for which trials they are considering. Either it's very vague or it's buried in "trial-speak" which I have missed. My recollection with other meta-analyses is that the authors spend some time explaining their criteria in the 'method' section.

Cheers,

Yuri

[Mojo]

I notice that he cites Linde et al. 1997: Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials, stating that it rejected the null hypothesis that "homeopathy has an effect which is not statistically different from that of placebo", but not Linde et al. 1999: Impact of Study Quality on Outcomes in Placebo-Controlled Trials of Homeopathy, which concluded that their earlier paper "at least overestimated the effects of homeopathic treatments".

He does mention that Sterne et al.: Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis. Br Med J 2001; 323: 101–105 (a reanalysis of the first Linde paper) "noted that treatment effects were larger in smaller studies and in those with inadequate blinding of outcome assessment". This is in the context of the section describing the paper's approach to study quality, which states: "This review deliberately does not categorise published trials in homeopathy by their intrinsic scientific quality, for information on this issue is already available."

[fls]

I keep seeing references to this paper by Mathie, 2003 (http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WXX-48HXDX3-8&_user=6619241&_coverDate=04%2F30%2F2003&_alid=955445965&_rdoc=2&_fmt=high&_orig=search&_cdi=7170&_sort=d&_docanchor=&view=f&_ct=3&_acct=C000032518&_version=1&_urlVersion=0&_userid=6619241&md5=7d437ebda2f74a255583d75585b737b4) - The research evidence base for homeopathy: a fresh assessment of the literature, [Homeopathy (2003) 92, 84–91], where it is reported:

"Results. 50 papers report a significant benefit of homeopathy in at least one clinical outcome measure, 41 that fail to discern any inter-group differences, and two that describe an inferior response with homeopathy.

One of the problems is that there is a considerable bias in reporting on these studies - bias is the production of significant findings when none should be obtained. Most of the studies reported as positive were actually negative - that is, a statistically significant difference was not found on the main outcome measure. The key phrase is "at least one clinical outcome measure". What happens is that the homeopaths data dredge. While the main outcome measure is insignificant, they make multiple clinical outcome measures (number of people who had diarrhea on day 1, number of people who had diarrhea on day 2, number of people who reported improvement on day 1, etc.), perform multiple significance tests without taking into considering that this invalidates their measure of significance, and then report the study as 'positive' if one of these many measurements happens to show a (invalid) statistically significant difference. The only research which shows replicable differences in the main outcome measure is the oscillococcinum in influenza research. As I've pointed out previously, this is the only set of homeopathic research which even remotely approaches the standards used in conventional medicine for reporting that the research is "positive".

Considering the relative number of research articles on the 35 different medical conditions in which such research has been carried out, the weight of evidence currently favours a positive treatment effect in eight: childhood diarrhoea, fibrositis, hayfever, influenza, pain (miscellaneous), sideeffects of radio- or chemotherapy, sprains and upper respiratory tract infection.

The studies on childhood diarrhea and hayfever suffer from the problems I mentioned above. The fibrositis studies were actually negative, but reported as positive based on 'adjusting' the negative results until a significant difference in one of multiple measures was obtained. I'm not sure which studies they are using for "pain", but some studies on Rheumatoid arthritis were reported as positive based on an erroneous analysis. Side-effects of radio or chemotherapy includes therapies that are not homeopathy (i.e. they are simply plant-based remedies that are not diluted) like the hay-fever studies (which are based on immunotherapy). I don't remember what the sprains and upper respiratory tract infection studies refer to (the URTI studies may be on Zicam which is also not homeopathic (not diluted)).

Many of these studies are not randomized, placebo-controlled, double-blind studies. It is important to remember that inadequate study design and lower quality studies have the effect of returning a positive result even when no effect is present. The issue of study quality gets treated as though it merely modifies the results, that it represents a quantitative difference. But in reality, it creates results that are simply not present. It really represents a qualitative difference, so much so that studies that are not randomized and placebo-controlled and double-blind shouldn't even be included in this review. As soon as you discard any one of those factors, you will get a positive result regardless of whether effect is present.

Based on published research to date, it seems unlikely that homeopathy is efficacious for headache, stroke or warts. Insufficient research prevents conclusions from being drawn about any other medical conditions.
Conclusions. The available research evidence emphasises the need for much more and better-directed research in homeopathy. A fresh agenda of enquiry should consider beyond (but include) the placebo-controlled trial. Each study should adopt research methods and outcome measurements linked to a question addressing the clinical significance of homeopathy’s effects."

Given that interpreting scientific papers is not my strong point :o, could anyone comment on this paper and explain why its conclusions (that homeopathy is effective in certain conditions) seems to fly in the face of the rest of the evidence in more mainstream publications. A lot of homeopaths give this as cast iron evidence that homeopathy works and everyone else is biased against them.

Many thanks, :)

Yuri

Most homeopaths may simply not know any better. They've learned from their superiors and don't realize that they have been misled and misinformed. Certainly the homeopaths that have shown up here seem completely oblivious to importance of these issues, deriding us for being close-minded instead.

Linda

[Mojo]

Am I misreading it, or is he saying in the passage I quoted above that because information about the effects of study quality on the outcomes of trials is readily available, he is ignoring it in this study?

[Mojo]

While we're on the subject of analyses cited by homoeopaths, here's a more recently published one that has just been waved around on Wikipedia:

Van Wassenhoven: Scientific framework of homeopathy: Evidence-based Homeopathy (http://www.feg.unesp.br/~ojs/index.php/ijhdr/article/view/286) Int J High Dilution Res 2008; 7(23): 72-92, June 2008

[Yuri Nalyssus]

Am I misreading it, or is he saying in the passage I quoted above that because information about the effects of study quality on the outcomes of trials is readily available, he is ignoring it in this study?
Well, that was my interpretation and, because I couldn't believe that even someone with such obvious preconceptions could treat trial methodology with such indifference, was one of the reasons for my question. He seems to be saying that because others have 'dealt' with the question of study quality it's ok for him to ignore it and, presumably, trust readers to interpret appropriately.

I don't get it. :confused:

Yuri

[Mojo]

... trust readers to interpret appropriately.


On the basis that the data relied on is of unknown reliability, presumably.

[Badly Shaved Monkey]

Mojo will correct me if I get the details wrong and I'm on my phone again so can't quickly check, but the whole issue of good studies needing to be placebo-controlled AND randomised AND double-blind was at the heart of a long and tedious dispute with DUllman at Wikipedia where he was trying to force inclusion of a Linde (yes, him) meta-analysis of toxicology studies where Linde basically treated the features of a good study as a lucky dip bag where those three essential features were taken into account with a load of less important features and "high-quality" was defined on a kind of 'two out of three ain't bad' basis. This fails to appreciate that certain features are absolute prerequisites of a good study and failing to satisfy each of PC, R and DB is a reason for instant rejection. What Linde did was to carefully define "high quality" so that some of these sine qua nons become optional, with the, no doubt accidental :), effect of guaranteeing that some apparently positive studies would stick up out of the mud.

But winning that battle was such a struggle within the constraints of Wikipedia's rules that it has sapped my appetite for ever editing there again on anything that will be defended by determined and highly motivated idiots.

[Mojo]

It's here (http://en.wikipedia.org/wiki/Talk:Arsenicum_album/Archive_1), I think.

See the sections headed "The Linde metaanalysis", "Linde meta-analysis on environmental toxicology deleted", "Cazin (1987)", "Cazin and Linde", "The Linde (1994) Meta-analysis and the Cazin article", "How to present these "studies"", etc...

[Badly Shaved Monkey]

OK, anyone keen to wade into the Lake Stupid with DUllman can start here (http://www.sciencebasedmedicine.org/?p=242) and search the page for "Cazin", one of the author's names. There is a link to a Wikipedia page and you can follow the white rabbit onwards from there if you have the time, inclination and no life.

[Badly Shaved Monkey]

Or do what Mojo said.

Thanks, Mojo.

[Badly Shaved Monkey]

As ever, this brings us back to a familiar question are these people knowing frauds or idiots?

I am tending to a third option: they are incompetent true-believers.
They 'know' they are right from personal experience and authority.
They would like scientific approbation and respectability, essentially as window-dressing.
They don't need this for themselves, because they 'know' already.
They want it to promote their activities in the wider world partly because they hanker after that intellectual respectability which they have a sneaking feeling they do not have.
They have neither the skills nor the willingness to apply normal critical standards to their work, so pay no attention to the detail of how to do the scientific method properly.

What they do has the mere semblance of science, but they think they do science and feel fully justified in pushing any apparently positive results they get with as much force as they can muster.

[Pixel42]

What they do has the mere semblance of science, but they think they do science and feel fully justified in pushing any apparently positive results they get with as much force as they can muster.
It's cargo cult science:

http://www.lhup.edu/~DSIMANEK/cargocul.htm

In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they've arranged to imitate things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas--he's the controller--and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land.

Now it behooves me, of course, to tell you what they're missing. But it would be just about as difficult to explain to the South Sea Islanders how they have to arrange things so that they get some wealth in their system. It is not something simple like telling them how to improve the shapes of the earphones. But there is one feature I notice that is generally missing in cargo cult science. That is the idea that we all hope you have learned in studying science in school--we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated.

Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can--if you know anything at all wrong, or possibly wrong--to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.

In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.
I never miss an opportunity to quote Feynman :)

[Mojo]

I am tending to a third option: they are incompetent true-believers.
They 'know' they are right from personal experience and authority.
They would like scientific approbation and respectability, essentially as window-dressing.
They don't need this for themselves, because they 'know' already.
They want it to promote their activities in the wider world partly because they hanker after that intellectual respectability which they have a sneaking feeling they do not have.
They have neither the skills nor the willingness to apply normal critical standards to their work, so pay no attention to the detail of how to do the scientific method properly.

What they do has the mere semblance of science, but they think they do science and feel fully justified in pushing any apparently positive results they get with as much force as they can muster.


There's a comment in Linde & Melchart's 1998 paper Randomized controlled trials of individualized homeopathy: a state-of-the-art review (http://www.ncbi.nlm.nih.gov/pubmed/9884175), under the heading "Problems with randomised clinical trials of homeopathy" on page 384, that is relevant here: The motivation for doing trials seems less to be innovation or self-critical evaluation of performance (which is generally agreed to be the motivation for good research) but rather justification in front of a hostile scientific establishment.


Hence homeopaths do stuff that superficially looks like research, but is in fact intended to arrive at a particular result.

[laca]

What they do has the mere semblance of science, but they think they do science and feel fully justified in pushing any apparently positive results they get with as much force as they can muster.

That is why it's called pseudoscience.

[Yuri Nalyssus]

Hence homeopaths do stuff that superficially looks like research, but is in fact intended to arrive at a particular result.
Agreed, they set out to prove, not to test.

Yuri