Designing a double-blind test for acupuncture

[thaiboxerken]

It seems to me that acupuncturists have a very tough excuse to beat when it comes to double blind studies. They claim that only qualified acupuncturists can make acupuncture work . This makes it hard for bias to be factored out. I mean, if the acupuncturist gives sham treatment to a patient, he could be giving the patient subconscious ques that it is a sham.

This brings up the challenge, how can this bias be factored out?

[crimresearch]

Qualified acupunture doctors claim to be able to make standardized diagnoses, by several methods...

That part of the methodology would be a good place to start..getting several acupuncturists to agree on what the exact Qi imbalance is by having them diagnose a sample of patients, and see how often they agree.

'Patient b has an excess of Yang Qi in the TripleWarmer meridian..Patient C has...'

Overcome that obstacle, and then let's see how to test the curative claims.

[CBL4]

The other thing to do is have to try the "true" accupuncture points vs. random points. The people inserting the needles would not be told which are real and which are fake.

CBL

[crimresearch]

Originally posted by CBL4
The other thing to do is have to try the "true" accupuncture points vs. random points. The people inserting the needles would not be told which are real and which are fake.

CBL
That was what I took to be Thai's point..if they are acupuncturists, they are going to know the real points...
if they aren't real acupuncturists, then they won't be able to properly insert the needles.

[Mojo]

I remember something quite recent about a project using retractable needles which were supposed to appear real to the acupuncturist as well as the patient.

Edited to add: I have my doubts about how convincing this could possibly be...

[Jeff Corey]

What's the point? Which system are you going to use? They all have different diagrams about where your chi flows.
They can stick it.

[Mojo]

Originally posted by Jeff Corey
They can stick it. :D

[Jeff Corey]

Prick.

[crimresearch]

Originally posted by Jeff Corey
What's the point? Which system are you going to use? They all have different diagrams about where your chi flows.
They can stick it.

Not every single acupuncturist uses entirely different meridians and points...There is enough current standardization.

The subjectivity comes more into play with the 'feeling' for 'reedy pulses', and other Qi sensations that can't be externally measured.

And acupuncture does not describe itself as just 'sticking points'...
the needles are eased in until the acupuncturist feels a certain sensation, and then manipulated...so that rules out fake needles or non-acupuncturists in a study.

The question that could be tested, as with polygraphers, is whether or not they can report the same conclusions independently.

If they can't get past that, why try to design a test for efficay?

[Jyera]

Originally posted by crimresearch
Qualified acupunture doctors claim to be able to make standardized diagnoses, by several methods...

That part of the methodology would be a good place to start..getting several acupuncturists to agree on what the exact Qi imbalance is by having them diagnose a sample of patients, and see how often they agree.

'Patient b has an excess of Yang Qi in the TripleWarmer meridian..Patient C has...'

Overcome that obstacle, and then let's see how to test the curative claims. I think crimresearch's suggestion is practical.

Acupuncture is also on the rise in different countries. There are Institute of learning teaching the skill to be an acupuncturist in different countries. If they are not a true scientific and credible system, they are bound to start having conflicting standards.

Bringing their conflict to the fore exposes thier weaknesses to the public.

Eg. Acupuncturist in USA say Patient b there is excess of yang qi in the Triple warmer Meridian. But Acupuncturist in China disagree. Then one of them must be wrong.

Having them resolve to straighten up their act, and to maintain a single standard would be good.

If the effectiveness of this single standard "science" of acupuncture is good, then good for the patients.

If however, despite having a "good", "proven" and "standardised" system of diagnosis and treatment via acupuncture, the effectiveness is poor, then
there is only one single "standard" to defeat. NOT loads of charlatants to expose.

[Ririon]

Originally posted by crimresearch
That was what I took to be Thai's point..if they are acupuncturists, they are going to know the real points...
if they aren't real acupuncturists, then they won't be able to properly insert the needles.

It gets complicated if you want a real doubleblind test of acupuncture. One way of getting close (for the people who see the problem) is to teach two groups of acupuncture students, giving one group the accepted teachings and the other group some other hogwash (placebo). If they could be taught the basics of some simple and not too dangerous acupuncture treatment, you could have acupuncturists giving the treatment without knowing if it was real of placebo.

The trial would have to be quite quick (maybe a weekend) so that the students didn't figure out which group they were in. They would have to be cut off from the rest of the world during the trial and have no prior detailed knowledge of acupuncture (hard to verify 100 %).

This setup is too complex for the Challenge, and would be more at home in a medical journal. Heck, if done properly and with a positive result (for acupuncture) it would go straight into an issue of Nature with some needles on the front cover.

However, acupuncture fails miserably at much weaker attempts at testing. At least i have heard anecdotes of acupuncture failing single blind tests, where the acupuncturist keeps his poker face while placing the needles in the wrong meridian.

I actually use one anecdote to check how much a person knows about the placebo effect and his attitude to acupuncture:

Three people are treated to relax and lower the heart rate. They are all placed in comfortable chairs in quiet rooms.

Person A: Gets acupuncture designed to bring the body to harmony and relaxation and lower the heart rate.
Person B: Gets acupuncture to stimulate the liver or something else unrelated to relaxation, but is told the same as person A.
Person C: Is just told to relax, gets no acupuncture.

What happens to their heart rates?

Well, according to the anecdote, all heart rates drop, A and B drop the same amount, but significantly more than C.

Ririon

[Asolepius]

Originally posted by crimresearch
Qualified acupunture doctors claim to be able to make standardized diagnoses, by several methods...

That part of the methodology would be a good place to start..getting several acupuncturists to agree on what the exact Qi imbalance is by having them diagnose a sample of patients, and see how often they agree.

'Patient b has an excess of Yang Qi in the TripleWarmer meridian..Patient C has...'

Overcome that obstacle, and then let's see how to test the curative claims.
This has already been done - although I can't remember where. They all came up with different diagnoses!

[thaiboxerken]

Ririon, the problem with your "sham the student" protocol is that acupuncturists claim that over 100 hours of training is needed. Of course, in reality, one only needs a bunch of needles and a gullible patient. However, that would be the excuse they use.

[crimresearch]

Of course those needles are now classified as medical instruments by the FDA, which would probably rule out any experiment using un-licensed non-acupuncturists.

[H'ethetheth]

But isn't acupuncture based on ancient Chinese medicine, which was pure speculation to begin with? Apparently they didn't want to cut up dead people because these people would need their entire bodies in the afterlife or something.
So, I guess the question is: Why would you need a double blind test for a medical practise that has as much chance of being right as my great grandma's analysis of the internal workings of a computer and is supposed to work on principles that elude modern medicine?

[drkitten]

Originally posted by H'ethetheth
But isn't acupuncture based on ancient Chinese medicine, which was pure speculation to begin with? Apparently they didn't want to cut up dead people because these people would need their entire bodies in the afterlife or something.
So, I guess the question is: Why would you need a double blind test for a medical practise that has as much chance of being right as my great grandma's analysis of the internal workings of a computer and is supposed to work on principles that elude modern medicine?

Because even a blind squirrel can find a nut once in a while.

[crimresearch]

Originally posted by H'ethetheth
But isn't acupuncture based on ancient Chinese medicine, which was pure speculation to begin with? Apparently they didn't want to cut up dead people because these people would need their entire bodies in the afterlife or something.

That's a new one on me.
They certainly didn't have any objection to cutting up people when they were alive....

So, I guess the question is: Why would you need a double blind test for a medical practise that has as much chance of being right as my great grandma's analysis of the internal workings of a computer and is supposed to work on principles that elude modern medicine?

So that you will be basing your belief that it doesn't work on an objective measure...unlike the anti-vax woos, who don't need no steenkin' proof?

[H'ethetheth]

Originally posted by new drkitten
Because even a blind squirrel can find a nut once in a while. Okay, I guess you're right. They might have found something that empirically works. I withdraw my question.

[thaiboxerken]

No take backs!

[H'ethetheth]

Originally posted by thaiboxerken
No take backs! Damn!

[H'ethetheth]

Originally posted by crimresearch
That's a new one on me.
They certainly didn't have any objection to cutting up people when they were alive... I had thought about that, but this thing about chinese medical practise is obviously just something I heard, and I did recently, though I don't remember where.
Anyway, I wasn't there, and I'm not a historian. So I might as well shut up now...
:bricks:
So that you will be basing your belief that it doesn't work on an objective measure...unlike the anti-vax woos, who don't need no steenkin' proof? You're right, I spoke hastily (and in a slightly intoxicated state).
I've just been informed I'm not allowed to take it back though, sorry mate.

Now about that test. I see where the problem arises, but I don't see a simple solution. I'll think about it, and see what I can come up with.

[CBL4]

Ririon idea of training people to insert needles in the "correct" places and and random places is what I meant. I do not see a need for it to be quick however. It could be repeated weekly for a few months and if neither the acupuncterists or patients had no reason to believe they were using the wrong points, it would continue to be double blind.

You could also have some "real" accupuncturists judge the accuracy of the needle placement via videotape. This would prevent after the fact claims of improper procedures.

CBL

[patnray]

Originally posted by CBL4
The other thing to do is have to try the "true" accupuncture points vs. random points. The people inserting the needles would not be told which are real and which are fake.

CBL

Perhaps the patient could be behind a screen through which the needles are inserted so the accupuncturist doesn't know where on the patient's body the needle was inserted. The practitioner, after making his diagnosis, would request, through a third party, that the patient be positioned so a certain "meridian" is at the screen. Some patients would be instructed to present the requested "meridian" at the screen and others would be instructed to present non-meridian locations.

Could the practitioner tell the difference? Would there be any difference in outcome for the two classes of patient?

[crimresearch]

Folks, acupuncture has never been a dart board....needles aren't randomly stuck...the spots are located by, among other things, feeling for a 'depression in the Qi on the meridian'.

The needles are inserted at certain angles, and depths, again by 'sensing' the patient's Qi and it's resistance to the needle.

And then the needles are manipulated at that spot, depth and angle to 'restore proper Qi flow' according to the ailment diagnosed...

It is so subjective as to be practically untestable in the manner suggested.

What you folks are doing is like saying that you can test whether or not auto mechanics 'works' by throwing screw drivers into a running engine, and seeing if the problem is fixed.

If you don't follow the same procedures as the practitioners, you haven't tested their practices...you have only tested your incorrect version of the practice.

By attemtping to test efficacy, you are peering into the wrong end of the horse...
Move to the other end, and look in the mouth....start by testing the practitioner's ability to *diagnose* with replicable results.

[Ririon]

Originally posted by thaiboxerken
Ririon, the problem with your "sham the student" protocol is that acupuncturists claim that over 100 hours of training is needed. Of course, in reality, one only needs a bunch of needles and a gullible patient. However, that would be the excuse they use.

I guess you have to lock them up for a couple of weeks, then... :)

Anybody have access to 50 eager students, 100 people with the same ailment, as diagnosed by a concensus of acupuncturists and an empty prison wing complete with guards? The patients have to be locked up, too, so they too have no way of finding out if they are given the real thing.

As I said, it gets complicated... :D

Anyway, this is about the level you would have to take your research to if you wanted to get some credible results to support acupuncture. If the effect was very strong, you would need fewer people, but I think we can safely assume based on the research available, that any effect would be very small compared to placebo, so you would probably need even more people than this to get anything statistically significant.

However, to demonstrate that acupuncture is complete BS, you don't need to go through all this trouble.

A single blind study where the acupuncturists know if they are faking (but communication about this to the patient is minimized) is much easier to conduct, but can not be taken credit for if it shows acupuncture to work. As a preliminary study, that could either disprove acupuncture (beyond any reasonable doubt, to use a legal term) or trigger the whole "sham the student"-study.

Ririon

[Asolepius]

This is a fascinating topic, with some insightful posts. As some of you may know from other threads, I was a patient in an acupuncture trial 3-4 years ago. This has now been published (http://www.annals.org/cgi/content/abstract/141/12/911), and is the subject of an acrimonious dialogue between myself and the authors. Essentially, the investigators lied to me about the study design when I enrolled. I still have the patient information sheet - the title of the study is different from that on the published paper. The authors' university is now investigating the study, although I am not holding my breath.

But this discussion highlights the subjective nature of the practice, and the difficulty of blinding it. I like the idea of testing for the diagnostic component. I have had acupuncture 3 times, and each time was told to expect some sort of sensation on insertion - this has been termed `de-chi'. The practitioner is supposed to be able to detect this. Well I never felt a thing, so how did the practitioner place the needles correctly? Perhaps we could test the concordance between what the patient feels and what the practitioner feels?