I acquiesce.
I know that I’m not going to have a lot of control over my selection of blogging material for a given day when I see more than one or two requests for an analysis of an article. So it was, when links like these were showing up in my e-mail:
- Acupuncture May Help Ease Symptoms of COPD
- Acupuncture May Be Worth a Shot for COPD: Small Study Shows Acupuncture May Help People With COPD Breathe Easier
These two news stories refer to a study from Japan by Suzuki et al published online yesterday in the Archives of Internal Medicine entitled A Randomized, Placebo-Controlled Trial of Acupuncture in Patients With Chronic Obstructive Pulmonary Disease (COPD): The COPD-Acupuncture Trial (CAT). It is just what it sounds like: A test of acupuncture on COPD. First, let’s see what these news stories say about it, beginning with U.S. News and World Report:
For patients with chronic obstructive pulmonary disease (COPD), acupuncture may help relieve shortness of breath during activity, Japanese researchers suggest.
COPD is a progressive lung condition that makes it hard to breathe; it is commonly caused by smoking or exposure to other toxins.
“The effects of acupuncture are large,” said Dr. George Lewith, from the University of Southampton in Hampshire, England, co-author of an editorial accompanying the study. “This is particularly remarkable in a condition that seems largely unresponsive to more conventional treatments.”
And WebMD, whose writers should know better:
Exactly how acupuncture improves symptoms of COPD is not fully understood. Researchers speculate that needling the acupuncture points on the rib cage area may help relax muscles involved in breathing.
This makes perfect sense to Tong-Joo Gan, MD. He is a professor of anesthesiology at Duke University Medical Center in Durham, N.C. It also may help reduce anxiety levels, he says. “When you become breathless, your anxiety goes up, so relaxation is another possible explanation for the benefit.”
Acupuncture has been shown to release chemicals that relax the lungs and dilate the airways, he says.
“Clearly it looks like a viable alternative to treat chronic COPD,” Gan says. “The downside is so little and the upside is so huge that acupuncture is well worth a try for those who find it difficult to control their COPD despite medications.”
Wow! If this study is any indication, acupuncture is the greatest thing since sliced bread, at least for COPD. At least, it is if you believe the hype machine that’s revving up to promote this study, just as it does for any seemingly “positive” acupuncture study. But is it right this time? Is this study really good evidence that acupuncture “works” for COPD? Not so fast, there, pardner. The study, despite the breathless descriptions of it popping up in the press yesterday, is–shall we say?–underwhelming.
The study itself is fairly straightforward in that it is a randomized study of patients with chronic obstructive pulmonary disease (i.e., COPD) treated with standard therapy plus either “real” acupuncture or sham acupuncture. In this case, the sham acupuncture consisted of needles that didn’t puncture the skin rather than needling the “wrong” acupuncture points. The device used was a Park sham device, which comprises a needle (real or blunt-tipped placebo) with a guide tube. The blunt needles appear to penetrate the skin but actually telescope back into the tube. The primary endpoint measured was breathlessness as measured by an instrument called the modified Borg scale after a test known as the six-minute walk test. The modified Borg scale measures from 0 (no breathlessness) to 10 (maximal). They also measured lung functions. Acupuncture treatments (sham or “real”) were administered once a week for twelve weeks, and the acupuncture points chosen were as shown below:
After twelve weeks of sham acupuncture or “real” acupuncture, the placebo acupuncture group (PAG) and real acupuncture group (RAG) were compared for various measurements after the six minute walk test. Again, the primary outcome measured was the modified Borg scale, which is a subjective measurement of breathlessness, with a whole bunch of other secondary endpoints. Whenever I see such a large number of endpoints, I wonder about whether any control was made for multiple comparisons, and, as far as I can tell from reading the methodology, there wasn’t. So what did Suzuki et al find? After randomizing 68 patients, the found a significant improvement in the Borg scale after the six minute walk test. They also reported a small improvement in oxygen saturation (86% to 89%) while FEV1 didn’t change. (The significance of FEV1 was discussed in a previous post about acupuncture and asthma.)
Many of the usual caveats with a study of this type apply. First of all, it’s a small study, and it’s very easy to have a false positive in a small study like this. I have a hard time making much of this study without replication or a larger study. Second of all–and this is the biggest flaw in the study, a flaw so large that in my mind it pretty much invalidates the study–the study was only single-blinded. The subjects were blinded to experimental group, but the researchers and acupuncturists were not. There is no good excuse for this lapse, given how many other investigators have successfully carried out double-blinded acupuncture studies. The authors simply state that “we were unable to mask the acupuncture therapists.” Again, other groups have managed to blind the acupuncturists using specially constructed needles; why couldn’t Suzuki et al?
Another thing that drove me crazy about this article was that the authors piled endpoint after endpoint into tables. About half the endpoints appeared to be statistically significantly different, but with wide confidence intervals. For example, adjusted differences between PAG and RAG in three of eight biomedical measures listed in one table (Table 5) and six of eleven physiological measures (Table 6) were not statistically significant. Others that were “statistically significant” appeared not to be particularly impressive. A lot of these measurements, such as pulmonary function tests and the like, can also be influenced by patient effort, which could easily be affected, either intentionally or unintentionally, by how much the investigators measuring ventilatory function encouraged them. In other words, what we have here is a bunch of outcome measures, subjective and objective but potentially influenced by investigators, that are not particularly impressive in a trial that is not double-blind.
Another thing that one has to remember. I discussed this just yesterday, too. For a treatment that does absolutely nothing to the outcome measures being examined, at a statistical significance level of p < 0.05, by random chance alone we would expect about 5% of studies to find an apparent difference between treatment group and control. Of course, it's worse than that, as I've written about many times before. As John Ioannidis has taught me, because no clinical trial is designed and executed flawlessly and because there are always biases and imperfections in any clinical trial, the number of false positive trials for something like homeopathy (which, being water, does absolutely nothing) will actually be considerably higher than 5%. That's why one has to fall back on the totality of the scientific literature filtered through the lens of plausibility as estimated by basic science considerations. For COPD, the plausibility that acupuncture would be expected to have a physiological effect is slim to none. Perhaps it's not as close to "none" as homeopathy is (acupuncture does, after all, involve sticking needles into the body and it's just barely plausible that that might do something), but it's pretty darned low. Thus, filtering this study through the considerations of prior probability, the lack of double blinding, and the lack of controlling for multiple comparisons, and I am profoundly underwhelmed. That doesn't even take into account the fact that I don't see any evidence that the data were analyzed in a strict intent-to-treat analysis, in which all the endpoints were chosen before the study was undertaken and included in its design from the beginning. There were drop-outs in both groups, but in the RAG group three dropped out because they suffered acute exacerbation due to a respiratory infection. In such a small study, that could easily have skewed the results if a strict intent-to-treat analysis weren’t used.
None of this stops the authors from speculating wildly about a “mechanism” by which acupuncture can allegedly improve lung function in COPD:
We therefore speculate that a similar phenomenon is evoked in the accessory respiratory muscles by needling on the acupuncture points on the rib cage. Decreased muscle tone consequently caused the recovery of the muscle strength in the rib cage, resulting in the increased mobility in the rib cage. Relaxation of accessory respiratory muscles may also contribute to rib cage motion. In fact, the present study showed increases in maximum inspiratory mouth pressure, maximum expiratory mouth pressure, and range of motion in the rib cage at the end of acupuncture treatment.
In this study, vital capacity, FVC, percentage of FEV1, and percentage of DLCO significantly increased after acupuncture treatment. These findings suggest that acupuncture treatment might improve DOE and exercise endurance, at least to some extent, through the improvement of pulmonary function. It is not clear why acupuncture improves pulmonary function; however, we speculate that the relaxation of hyperactivated respiratory muscles and the correction of the autonomic tone might cause the beneficial effect on pulmonary function.22 Further investigations are needed to clarify this.
Go back and take a look at the acupuncture points used. Look at how few of them are actually over the ribcage. Is it the least bit plausible that a mere six needles in the ribcage could accomplish this result? I think not. In fact, I think the acupuncture apologists are doing some major contortions reaching for this “explanation.”
Given the inherent implausibility of acupuncture, combined with the large body of evidence that shows that it doesn’t matter where you stick the needles or even if you stick the needles in the skin. You get the same result, which is indistinguishable from placebo effects. For a study to overcome that large body of evidence, it has to be far more compelling than Suzuki et al. In the end, Suzuki et al is not nearly as rigorous as it has been represented and, as a result, not nearly as persuasive as acupuncture apologists would like you to think.
91 replies on “Acupuncture and COPD? Not so fast…”
A talking head from Southampton?
Dammit… Usually when I hear about my alma mater (as I believe the term goes, it’s not really used in the UK) it’s because they’ve done something good, like brought Sir Tim Berners-Lee on board or given honorary doctorates to the Thrust SSC team. Or at the very least becasue they did something entertaining like accidentally blow up one of their buildings.
A talking head from Southampton?
Dammit… Usually when I hear about my alma mater (as I believe the term goes, it’s not really used in the UK) it’s because they’ve done something good, like brought Sir Tim Berners-Lee on board or given honorary doctorates to the Thrust SSC team. Or at the very least becasue they did something entertaining like accidentally blow up one of their buildings.
Department of Understatement:
Also, your choice of the word breathless for describing the hype is inspired, given that COPD is the subject.
So, the acupuncture technicians were not blinded – always good to not blind the one group with a vested interest in a specific outcome of the test.
Researchers speculate that needling the acupuncture points on the rib cage area may help relax muscles involved in breathing. (from the WebMD article)
As you correctly point out, there is some small plausibility that this might actually do something. But they actually need to test this hypothesis, something which this study does not do.
Of interest to me is that they claim to have a placebo for acupuncture. I wasn’t sure at all how you would do this: do you put the needles in at random points rather than the canonical points? The needle that doesn’t go in might do it, but that brings up the question of how you double blind such a study: wouldn’t the placebo givers notice, even if the patient doesn’t?
@Eric Lund
Two ways of doing placebo: non-piercing needles or non-canonical points.
The former is good because you can blind even trained acupuncturists. For the latter, you’d need staff who are taught the technique, but not the points (or are taught the “wrong” points that are being used in the study). There are ways to blind even the practitioners.
@Eric
Actually, they have special needle sets, with sham needles that retract into sheaths and are indistinguishable from the real needles, even to the acupuncturist. This was briefly mentined by ORAC in this post, but he’s described them in more detail in previous posts.
COPD is exactly the type of condition that woo recognises as being most amenable to its ministrations:
it’s chronic, has a fluctuating course, may not be adequately controlled by SBM treatments, compromises life style and has emotional concommitants.
Thus the nets are cast for dis-satisfied and worried COPD patients who may perhaps be frightened about dependence on inhalable meds- as well as their side effects.
I wonder how many people who require meds for a chronic condition are frightened away from using their prescriptions *as directed* because of the anti-pharmaceutical rants emanating from the usual suspects ( Mike, Gary, Joe et al), either directly or indirectly? How much is pharma-phobia spreading like a virus throughout our culture?
I know a fellow, otherwise reasonable, who was diagnosed with asthma at age 50+ : he uses his inhaler as though its contents were aerosolised platinum and he had to pay for it himself.
The George Lewith, homeopath and proponent of alternative medicine?
Are the authors are admitting that meridians are bullshit? They seem to focus on the points directly over the lungs as directly effecting muscular relaxation as opposed to the peripheral “meridian points”.
“Acupuncture may be worth a shot for COPD”
Shouldn’t that read “may be worth a stab”?
OT : however, is the successful mis-education of youth *ever* TRULY OT @ RI?
Age of Autism has a new contributor who should strike terror in the heart of one grad student we all know and love:
she’s the new kid on the block and she’s actually… a kid- 17 year old art student, Natalie Palumbo. Her brother has autism and she’s writing up a storm about it.
Already ( OK, I’ll tread lightly because she *is* after all, a child) she is parroting the propaganda so dearly beloved at AoA: doctors, you see, are “blase” as well as sorely lacking “interest” and “knowledge” about autism, she informs us. I won’t go on.
It’s interesting to me because the woo-meisters I survey often perseverate on the critical importance of ‘educating’ the younger generation who are being ‘mis-informed’ about health and science by government-sponsored schools. Getting their own brand of science to woefully science-deprived kids is high on their list of priorities. Right. *These* guys are talking about deficiencies in educational systems.
As I’ve mentioned previously, I have a special place in my heart for students and I hate seeing them mis-used or mis-guided. Especially by these people.
Denice,
Does young Natalie have a “stage parent” who pushes her, or is this being done of her own volition? I get that a sibling will defend a family member voceiferously, but I’ve seen so many kids ill-used by their zealot parents (on both sides of the socio-political spectrum) and always pisses me off.
How proud her parents must be to have programmed the “vaccinesdidit” mantra in their child. In spite of her words that her brother is “physically perfect”, she maintains the parroted prose she’s been taught. A shame too since she is interested in a very important topic facing autists and their parents which is aging out of the system and having few or no services available to them as adults.
Todd and Kevin: Thanks, but that doesn’t really answer my question. If you have a needle that retracts into a sheath instead of going into the skin, you have to have a mechanical design which allows the needle to retract. I would expect a trained acupuncturist to be able to spot the telltale signs of this mechanism. Of course, I could be overestimating the practitioners, or there could be some feature present on normal acupuncture needles which the retraction mechanism is designed to mimic, but I would expect that eventually somebody who is supposed to think (s)he is using real needles would notice (s)he isn’t using real needles. The alternative blinding option, to have the acupuncturist put the needles in non-canonical locations, suffers from the obvious disadvantage that a real practitioner can’t avoid noticing that (s)he is putting the needles in the “wrong” places.
@Eric
That’s why I said they are a set of needles. The real needles in the set have a sham mechanism that looks just like the real mechanism on the sham needles. The real mechanism, when activated, feels just like inserting a real needle into skin. This has been validated using experienced acupuncturists.
And WebMD, whose writers should know better:
I have seen a small but regular supply of woo-filled (pronounced “waffled” *)articled in WebMD. ‘Tis sad.
* – OK, so I just made that up.
Why not use lack of blinding to increase the stringency of the study? Be interesting to see a study controlled with needles inserted at canonical versus random accupunction points, using technicians trained to insert acupuncture needles but otherwise ignorant of meridians, etc. where the technicians inserting needles into the canonical points had been told up front they were chosen to act as the placebo control and the technicans inserting needles into random points that they were chosen to act as the therapeutic arm of the study.
Does COPD include Cystic Fibrosis, or is it a whole different thing? Just curious, not about the study at ALL, just taxonomy.
The authors of this study may find that drilling holes in their heads helps relieve pressure on their brains. Then again, WebMD writers might find themselves better suited to writing reviews of waffle restaurants. Acupuncture having any effect on COPD patients is about as likely as homeopathy being able to treat lung cancer.
I love the idea, but that study design wouldn’t test the null hypothesis very well. You could not look at the results afterwards and say “These are two groups that are alike except for one independent variable, and therefore the difference in their results is almost surely due to the effect of that independent variable.” But a study with four arms – real needles told they’re the therapeutic arm, real needles told they’re the placebo, sham needles told they’re the therapeutic, sham needles told they’re the placebo – that would provide opportunity for comparison between groups…
Something I’ve always wondered about these sham needles – don’t real acupuncture needles puncture the skin and cause bleeding? How do they keep the users of sham needles from figuring out they’re using sham needles when no blood appears?
@ Pareidolius:
@ Science Mom:
While I’m sure that I would be rebuked for commenting- after all, I am not a parent, so what can I possibly comprehend?
I imagine that she didn’t dream this up on her own.** Because I think it in-appropriate to discuss youngsters ( Jake is an adult and thus, fair game), we should probably focus on parental radical belief systems and their effects on children. Kids long to fit in with peers, I wonder what effect having parents with far-out woo beliefs does for a teenager’s social status, not to mention their science grades at school. Parents’ pseudo-science may throw a wrench into career plans and higher education for many. It encourages laxity in thought and admiration for contrarians. You might say, “Aw, she’ll only study art- it won’t hurt that”. However, I can attest that some who begin in the arts may possibly wind up in other places ( ahem!).
More than one specific casualty, this situation exemplifies natural health/ health freedom attempt to sabotage education which especially concerns me because I suspect that women and girls are prime targets. Kids’ idealism ( in formal operational thought) often predisposes them to seek out perfectionism and utopian visions that eventually will be tempered through hand-to-hand combat with reality. Alt med gurus manipulate their search as much as the skewed parents do.
** It makes you long for the days of pre-requisite teenage rebellion ( which can be very important in the creation of individuality, adult responsibility and being in tune with one’s own historical cohort and their era’s select problems. My own parents encouraged this with an account for travel and airfare- so off I went on walkabout/ studies ).
@ Antaeus:
I don’t think that there is bleeding with normal accupuncture because of the fineness of the needles.
MedPage Today seemed to understand the single-blinding problem too.
“As well, it is possible that the attitude of the acupuncturists might have given patients and investigators some clues about study assignments, they noted. In addition, the study was relatively short and did not have follow-up evaluation.”
Single-blinded tests such as this, especially those that rely on effort-dependent outcomes such as distance walked and PFT results, are subject to the Clever Hans effect. The subjects can easily pick up on the subtle, subconscious direction cues of the examiners. Depending on the examiner’s bias, cues will be given in the direction of the bias.
In Bayesian terms, a claim with a very small prior probability would require unequivocally impressive evidence (extraordinary claims require extraordinary evidence). Small, improperly blinded studies with near equivocal results do not fit this bill.
Two ways of doing placebo: non-piercing needles or non-canonical points.
The former is good because you can blind even trained acupuncturists. For the latter, you’d need staff who are taught the technique, but not the points (or are taught the “wrong” points that are being used in the study). There are ways to blind even the practitioners.
I’ve had acupuncture, and it doesn’t cause bleeding. What I don’t understand is how the sham needles would stay put for the treatment. Acupuncture needles are stuck in the skin and remain there for a while. If the needle part is retracted, wouldn’t the sham needles just fall out?
And let me hasten to say that I didn’t seek out acupuncture treatment – the physical therapist I saw earlier this year for knee problems put in some needles along with his other treatments that actually worked!
Does young Natalie have a “stage parent” who pushes her, or is this being done of her own volition? I get that a sibling will defend a family member voceiferously, but I’ve seen so many kids ill-used by their zealot parents (on both sides of the socio-political spectrum) and always pisses me off.
Rebecca — I believe there is an adhesive, which is also present in the actual needles used in these studies so as to avoid revealing which set is being used.
This study had one primary outcome measure (Borg scale after six-minute walk test) and four secondary outcome measures (six-minute walk distance; lowest oxygen saturation during the walk; forced expiratory volume in 1 second; and quality of life). I literally cannot count how many clinical trials I have read of conventional treatments that had an equal number of primary and secondary outcome measures without discoursing at length on corrections for their gigantic number of comparisons made. “Other outcomes” included a long laundry list of numerical measures of respiratory function, etc. (the “biomedical measures” mentioned above), which cannot be treated as if they and the prespecified primary outcome are part of the same statistical category. Such numbers are of no direct value to patients, but if they hadn’t been collected and supplied, I’m sure the authors would have been criticized for it.
Here are the salient facts about the primary and secondary outcome measures, which most people will find more relevant than the “biomedical” measures. Over the course of the study, the placebo group’s mean Borg scale scores worsened from 4.2 to 4.6 (on a scale of 1 to 10, 10 being worst), while the verum group’s scores improved from 5.5 to 1.9. Improvement in three of the four secondary outcomes was statistically significant. The mean six-minute walk distance decreased from 405.2 to 385.8 m in the placebo group, while it improved from 373.2 to 436.7 m in the verum group. The total score on the respiratory quality of life questionnaire went from 40.8 to 41.1 in the placebo group, while it went from 46.2 to 30.2 in the acupuncture group.
As soon as I saw this publication in Arch. Int. Med., the first thing I said to myself was “Orac will write a blustery article about how acupuncture is Only A Placebo.” Well, if it is, I’ll suggest to anyone I know who has COPD that they ought to try taking a daily sugar pill, ’cause that Placebo is sure powerful stuff.
Yeah, those do look good, but what about the lack of blinding? COPD and asthma are conditions notorious for showing a placebo effect, as both can be affected by stress. I’d love to see a follow-up study which rules out that possible explanation by fully blinding the study.
Jane:
You’re talking about a single-blinded study wherein the only strong positive result came from a subjective measure. That strongly suggests experimenter bias contaminated the research. That’s before we even consider the historical pattern of high-quality, carefully double-blinded studies coming out negative. This isn’t a solid result. Even if the effect were real in this case, this article still would be weak as actual evidence for the real phenomenon.
Small positive results in a few of several objective measures that would not be immune to bias, in an experiment that already shows signs of bias and definitely has greater opportunity for it? That’s not going to help. The frustrating thing is that this didn’t have to be so weak, and a little extra effort would have made this a much stronger paper. If you really want to advance acupuncture, write the authors and encourage them to run a better experiment.
Rebecca and Calli Arcale:
According to this vendor’s site, this particular version of a sham needle set uses a relatively wide disc at the base of the device with adhesive on the bottom. That apparently is enough to keep the assembly attached properly. I think it’s the one used in this study (at least, both are called “Park”), although if these are the good experimental needles then I’m even more puzzled as to how they were “unable to mask the acupuncture therapists”:
http://www.acuprime.com/en/products/placebo-park-sham-research-needle–device/park-sham-device-research-tools/park-sham-placebo-acupuncture-device
Calli Arcale – Never having practiced acupuncture, I don’t know whether it is true that it’s hard to keep experienced acupuncturists from noticing the difference when a needle collapses easily enough that you don’t cause real stimulation of subcutaneous nerve endings. It seems possible that in some of the previous, putatively triple-blinded studies, either (a) the needles were pretty stiff or (b) the acupuncturists did notice but decided to keep their mouths shut about it.
In this study, patients and evaluators were blinded, and patients were asked at the end of the study whether they thought they had gotten real or fake acupuncture. In the placebo group, 4 said “real,” 2 said “placebo,” and 26 said “don’t know”; in the verum group, 2 said “real,” 3 said “placebo,” and 25 said “don’t know.” This implies that blinding held up pretty well. (Caveat: since more than 2 people in the real acupuncture group reported mild needle site pain or bleeding, at least a couple of these folks were either lying or not too bright.)
Anyway, if stress relief could relieve COPD and acupuncture could relieve stress, that still would sound worth trying to me. Most people who are suffering don’t care about the mechanism of relief, just that they are relieved. Sometimes, though, the placebo effect is credited with unlikely powers. If some “spiritual healing” idiot were to claim that he could treat COPD just by waving his hands over someone while directing his positive, loving healing thoughts toward them, we’d guffaw. But attributing this study’s results to the placebo effect requires us to believe that the acupuncturists’ positive thoughts, conveyed to the patients far more subtly and unconsciously IF AT ALL, have equally huge influence over their physiology. (Rather like the time when one of Orac’s minions suggested that a study of tai chi with an attentional placebo control was worthless because the tai chi teacher was probably Asian and that would suffice to make the mindless masses feel better – remember that howler?)
Jane:
“This implies that blinding held up pretty well.”
No, it doesn’t. It implies the patients didn’t generally know what treatment they got. The experimenters knew exactly which treatments they got, and that’s the problem. Don’t dismiss the potential of experimenter bias. It can, and does, influence results significantly.
If all that mattered was what the subject knew or didn’t know, Powerbalance never would have gotten off the ground. I’m not trying to pooh-pooh your broader ideas about placebos and stress with complicated chronic conditions like COPD; ethical debates aside (or not), it’s an interesting question. But even if we assume acupuncturists and magic bracelet salespeople and dowsers are not willful frauds (and it’s pretty clear many, if not most, aren’t), they still are fooling themselves and others. These authors didn’t do a good enough job making sure they weren’t fooling themselves in a field choked with opportunity for self-deception.
I’m just going by what others have said about how studies showed acupuncturists unable to distinguish the fake needles from the real ones, if this particular set was used (which was specifically designed to make it impossible to tell). It is of course possible that some of the acupuncturists and patients were deliberately lying about the results, but if we’re accepting that as a possibility, then I think that only drives home the problem we have here.
“In this study, patients and evaluators were blinded, and patients were asked at the end of the study whether they thought they had gotten real or fake acupuncture.”
But the acupuncturists themselves were not. If, as some have claimed, the real benefit to acupuncture comes from the soothing stress relief of having someone really caring about you, then if the acupuncturist is not blinded, then they will influence the outcome.
I’d still be a little worried about long-term results; symptom relief is a very different thing from control, and asthma and COPD can be deceptive. (That is, you can feel a lot better without actually being better.) This is why salmeterol has a black-box warning; it’s so good at symptom control that some people stop taking their steroids or other controller meds, unaware that they’re not really better.
Another worry I have is that if indeed stress relief is effective at treating COPD (and I think stress relief certainly can’t hurt and should improve quality of life regardless), is acupuncture really a good way to go about that? It does involve actual puncturing of the skin, and acupuncturists are a real mixed bag. There have been adverse events ranging from infection to punctured lungs. (Yeah, I wonder how incompetent an acupuncturist has to be to do that, but it has happened.) Obviously serious adverse effects are rare, but if it’s just a placebo effect, couldn’t there be a safer placebo? We could do sham acupuncture, but then we get the problem of lying to patients, which opens up a whole ‘nother can of worms.
a whole ‘nother can of worms
Not an ideal placebo treatment either.
Oh dear. Herr Doktor, you have just put an earworm (d’oh!) in my head.
Nobody likes me, everybody hates me
Guess I’ll go eat worms!
Long thin slimy ones, short fat juicy ones,
Itty bitty, fuzzy wuzzy worms.
Down goes the first one, down goes the second one,
Oh, how they wiggle and squirm!
Long thin slimy ones, short fat juicy ones,
Itty bitty, fuzzy wuzzy worms.
Up comes the first one, up comes the second one,
Oh, how they wiggle and squirm….
Jaranath – Instead of saying the “experimenters” weren’t blinded, we have to distinguish between the acupuncturists, who weren’t, and the hospital staff who conducted the 6-minute walk tests and collected data on lung function, thorax mobility, etc., who were blinded. The latter group were therefore not able either to lie about how far the verum group walked or how deeply they breathed, or to beam them happy thoughts to encourage them to report less respiratory distress. If the acupuncturists’ positive attitudes were conveyed to the patients powerfully enough that, in separate testing later, they were far more likely to report feeling better, then you would think that they would also be more likely to believe that they were getting an active treatment.
Calli Arcale – Yes, it would be good to know whether the separation between these groups could be maintained for a longer period, although early studies of symptomatic treatments are normally short-term. I doubt that acupuncture will affect lung disease, as opposed to respiratory muscle function, but a treatment that only delays the experience of disability or suffering still has real value to patients. It’s certainly good to know if you still have the condition so that you can consider taking long-term drugs, but we wouldn’t say that a pharma drug that alleviates asthma symptoms shouldn’t be used because people who felt less disabled would take fewer steroids!
Most placebos are not as effective as the acupuncture “Placebo”; there are a number of studies (I’ve cited a few before, and the comments are searchable) in which acupuncture outdoes a pharma drug, which has the placebo effects of a pill. You suggest using sham acupuncture as a “safer placebo,” but in this study, sham acupuncture offered no benefit to patients, so you really would be lying to patients if you told them that sham would do the same thing. Also, in this study there were only minor local adverse reactions – no infections, no punctured lungs. Does any pharma drug offered for relief of COPD symptoms have such a benign safety profile? I will have to check the tiny print next time I see one of those commercials showing elephants sitting on people. If acupuncture were a pill invented in America, it would have no trouble getting approved on safety grounds; indeed, anyone who fussed over its side effects would be screamed down as a paranoid medicine-hater.
Another point to be made here is that Japan was one of the countries that between 1965 and 1995 produced trials “uniformly favorable to acupuncture,” according to the famous article of 1996, “Do certain countries produce only positive results?”
See: http://www.sciencedirect.com/science/article/pii/S0197245697001505
That may have been reported 15 years ago, but I’m aware of no subsequent statements to suggest that researchers in those countries have acknowledged that finding and have endeavored to correct the problem, nor has there been a more recent review showing that things have changed. It remains reasonable, therefore, to summarily dismiss ‘positive’ acupuncture studies from those countries.
“If the acupuncturists’ positive attitudes were conveyed to the patients powerfully enough that, in separate testing later, they were far more likely to report feeling better, then you would think that they would also be more likely to believe that they were getting an active treatment.”
I don’t think that follows. I would have to defer to experts on experimental design and placebos as to how big a difference blinding the evaluators but not the acupuncturists would make (although that seems odd to me…how did that work?) But it doesn’t follow that positive attitude, or whatever bias the therapist may have introduced, had any effect on the patients’ perception of the therapy’s validity. Nor would it need to affect the perception of validity in order to affect the results. A patient’s perception and experience of a treatment and the things peripheral to it is not limited to whether it was real or not.
And why do we even have to bother with trying to explain why this particular mix of blinding and non-blinding is okay? This still could have, SHOULD have, been properly blinded and designed. At the end of the day we’re still left with an iffy study that got its one big hit in its weakest measure, and we didn’t have to be. At most we should be debating how significant the results of this solid study are given its small sample size, and setting up Google alerts to watch for bigger replications.
There has traditionally been a strong tendency to publication bias in Asian countries, and there may still be, but Doctor Atwood’s cheery statement that we can therefore “summarily dismiss” all future heretical papers with Asian authors is disastrous for multiple reasons. First, the fact that he has not had served to him on a silver platter a Western review showing that Japanese scientists are known to publish negative results does not mean that they never do. The cultural practice of publishing negative results is spreading worldwide, except where profit interferes, and I suspect that a little work would turn up negative studies published by Asians. Clearly, Doctor Atwood has no plans to do that work.
Second, what about publications sponsored by Western pharma companies, for which negative results were consistently buried for decades? We ought to take that publication bias into account when evaluating how much trust to place in positive results for antidepressants, for example, but I’m sure Doctor Atwood would be outraged by the suggestion that we could “summarily dismiss” every shred of positive data that have been published. This implies a suspicion of fraud by the authors of all positive studies, since otherwise, the point of publication bias is that you DON’T know whether the unpublished negative data outweigh the positive data, not that you know that they do.
Third, most of the official spokesmen for scientism like to deny that they wish to use Science to impose their own values on the rest of the world or establish Western or American authorities as the sole arbiters of scientific truth. They may say Science demands that traditional practices never be studied, but when accused of prejudice, they rush to say that no, Science is universal, so They can be scientists just like We can, if they can only manage to adopt our practices that are so very superior to theirs. Now here comes Doctor Atwood telling the entire community of Japanese researchers, many of whose expertise far exceeds his own, that he thinks all their publications past and future can be thrown in the garbage can unread because they are Japanese. I suppose I should praise him for his forthrightness.
jane:
Kimball:
Done beating your strawman to a pulp, jane?
Considering the enormous influence the pharmaceutical lobbyists have over the FDA, I think I would trust just about any Asian studies, methodology and treatments over newly approved medications in our own country.
jane:
Actually, we do, and I mentioned it in passing: salmeterol. The FDA mandated that it carry a black box warning for precisely that reason. It is associated with an increased risk of death due to asthma attacks because patients are more likely to stop taking their steroid medications when they should still be taking them.
This is actually why Advair exists; the manufacturer combined it with a steroid, which allows them to drop the black box warning because in that formulation there is no possibility of forgetting the steroid.
Citation needed.
Re. relaxing rib cage muscles: Easy enough to test with an EMG (electromyograph), which shouldn’t interfere with placement of actual or placebo needles.
Re. blinding the acupuncturists: Kevin @ 16: A devious acupuncturist could deliberately make erroneous “guesses” to “support” the idea that the Park placebo needle was indistinguishable from a real one. And if toothpicks “work,” then placebo needles that press upon the skin should also “work.”
One possible solution to this is to use “student” acupuncturists who are sufficiently inexperienced as to be truly unable to distinguish between real and placebo needles. An experienced acupuncturist would first visit the patient and mark the places at which needles are to be inserted (using harmless marking dyes on the exact spots). Then the student acupuncturists would be randomly assigned “real” or “placebo” needles to use on the designated points.
Anteus @ 21: Nice design, someone should do it. Really: the way to kill off quackery is to throw enough research money at it to do tighter studies using methodologies and Ns that everyone (the quacks and skeptics alike) agree *in advance* are acceptable.
Really: post-facto critiques are useful in certain ways, but also give the alties legitimate ground to complain about Monday morning quarterbacks and so on, and then dismiss the criticism with a magical wave of the hand. The gold standard has got to be full agreement by both proponents and skeptics *before* a research design is approved and the study carried out: at that point neither side can complain if they don’t like the outcome. Our side of this debate should have nothing to worry about from this approach.
As for “there isn’t enough money,” sure there is, and truly progressive policies (taxes and other) would claw it back from the casino gambling tables of high-finance monetary Onanism, to where it could actually do some good: not only in quack-hunting but in any other area one might choose to name. Personally I’m partial to non-carbon energy deployment, public transport, and the space program, but the amount of money needed for quack-hunting is small compared to those.
Well, just one more last comment about blinding of acupuncturists. In traditional acupuncture, there’s a sensation called “de qi” (from the study, “tingling, numbness, heaviness”) that is often supposed to be felt if a needle is correctly inserted and rotated at the correct place. (Possible reductionist explanation: this is a marker of adequate nerve stimulation.) In this study, “Perception of de qi … during insertion and/or manipulation was confirmed at every point in the [real acupuncture group]. … Perception of sensation during treatment sessions in [the placebo acupuncture group] included pricking or poking, but no sensation like de qi was reported.”
In other words, the acupuncturists were allowed to ask the patients to inform them of whatever sensations they felt during the treatment, so that, in the verum group, they could provide appropriate treatment by the standards of clinical practice. If an acupuncturist heard one patient say that he felt numbness and tingling during his treatments, while another only mentioned pricking sensations (as one would feel from the sacred Toothpick) no matter how long the needles were manipulated, over 12 weeks he could hardly fail to guess which of them was getting the real needles. This therefore would make it impossible to keep the acupuncturists blinded.
If producing this sensation is a criterion of good practice in Japanese acupuncture, then it might well be impossible to fairly test Japanese acupuncture with blinded acupuncturists.
g724:
One problem with it is that it has nothing to do with either asthma or COPD. Sure, you can hurt your rib cage muscles from coughing and from struggling to breathe with constricted airways, and that’s painful and would be nice to relieve. But it doesn’t do a thing for the constricted airways.
“If producing this sensation is a criterion of good practice in Japanese acupuncture, then it might well be impossible to fairly test Japanese acupuncture with blinded acupuncturists.”
And again, we shouldn’t even have to be speculating about that. We don’t know if that’s what happened or not, and if it’s true then we should be using a different methodology to test it. We have no idea why they weren’t blinded. We should know, but they chose not to tell us. You seem to be assuming the therapists figured out the groups themselves–that they were meant to be blinded but it failed–but we don’t even appear to know that. Rather than strain to explain these uncertainties, we can rule them out or describe their limits in advance.
I also have my doubts such a noticeable difference in patient sensation exists when good studies testing these placebos come up negative, but I don’t know. I wonder if that specific element has been examined before.
Calli Arcale – I wouldn’t say that symptomatic relief “has nothing to do with” any disease. If acupuncture works to improve breathing by muscular effects (such as the 3 cm increase in rib cage expansion seen in the verum group), then indeed that won’t change the underlying airway disease. But people use hundreds of drugs to feel better that don’t cure them, or even slow the progression of chronic diseases.
jaranath – No, I don’t assume that they were meant to be blinded but it failed; I assume that they were meant to be unblinded so that they could ask patients about their sensations and try to make sure that those getting real acupuncture experienced “de qi.” Blinded acupuncturists either would not be able to do that at all, or would fruitlessly and perhaps harmfully struggle to produce the sensation in the placebo patients. I did not note this point on my first quick read of the paper, but it is an adequate explanation that requires no strain.
You say “if [that’s] true then we should be using a different methodology to test it” – but what they were doing indeed was using a “different” methodology (compared to the putative American standard) to test their clinical practice. What you seem to be asking is that they test only something that they will consider to be an inadequate, watered-down version of their practice simply because that version fits better into your vision of the one right way to do a scientific study. But in fact, we do not place such stringent limitations on what kinds of conventional treatments can be developed and scientifically tested. For example, we do not require that studies of stenting vs. bypass chop everyone’s chest open so the patients don’t know whether they’ve had a bypass or not.
No, we don’t: we instead compare outcomes in people who know they’ve had a bypass to outcomes in people who know they’ve had a stent, because we’re comparing two different treatment modalities whose efficacy has already been independently established (in part by using animal models where the placebo arm animals did indeed undergo sham surgery (their chests were opened but no bypass performed) or sham stent implantation (catheterization performed but no stent actually placed).
Why couldn’t they ask about “de qi”? What’s stopping it? I can’t say with absolute certainty short of ordering needlesets and trying it out myself or with an acupuncturist, but I see nothing about the design in that picture that would prevent variable manipulation of a needle in response to patient feedback. It looks to me like you could easily do that with the system. And as I said, I’m skeptical that if there really is a significant, fundamental difference in sensation between real and sham needles, this wouldn’t have shown up in previous tests.
I wasn’t talking about excluding the de qi method from testing, either. I was saying that IF de qi is too prohibitively difficult to devise a placebo for ala sham needles, different testing methods could be used to evaluate it.
And just for fun, as I recall there actually was a famous case of using sham surgery to test the effectiveness of arterial ligation for angina, back in the good ‘ol days of crappy ethical review. 🙂 These days you could probably still do it by piggybacking on other needed invasive procedures.
jane:
My point is that relaxing rib cage muscles is implausible as an explanation for the improvements seen in the study, because there’s no plausible way it would improve airflow. At best, it might relieve some pain, but that pain’s going to come right back. So if the effect measured is real, I don’t think this is what’s responsible.
Also, while I acknowledged relieving pain would have some benefit, I wouldn’t count relaxing rib muscles as symptom relief for asthma or COPD. It would have to provide relief of restricted airflow, which would put it into a category alongside bronchodialators.
JGC – You are overlooking the possibility that different conventional modalities have different placebo effects – or you are saying that it’s impossible to do away with those effects, which is in fact okay by me.
Calli – They reported changes in a bunch of biomedical and respiratory function measures. While it seems plausible to me that breathing more deeply (in a relaxed rather than forced way) would reduce perceived shortness of breath, I don’t know that it’s either necessary or wise to anoint one of many factors as The Mechanism.
jaranath – Maybe some Western studies did not ask about de qi. There are 60 studies on PubMed that mention de qi. One of the earliest of these (J Park et al. Validating a new non-penetrating sham acupuncture device: Two randomised controlled trials. Acupunct. Med. 2002;20:168-174) used the lack of de qi in sham-needle patients as a means of (supposedly) demonstrating that a sham needle would be inactive. Real-needle patients were over 15 times more likely to report feeling de qi. On the other hand, Takakura and Yajima (2008) reported that sensations of de qi for their sham needle were not quite significantly different (p = .17). Maybe that should have made them wonder whether their sham needle was really lacking in activity?
This group and at least one other later reported that *for analgesic effect*, there was no significant correlation between the sensation of de qi and the amount of relief obtained. (N Takakura and H Yajima. Analgesic effect of acupuncture needle penetration: a double-blind crossover study. Open Med. 2009;3:e54-61. That might put paid to the Japanese Never Report Negative Results So They Can’t Be Trusted Theory. But they did report a benefit relative to untreated controls, so I guess they’re still bad scientists.) At least one study, though, has reported that de qi is correlated with pain relief.
It’s possible that the mechanism of action for pain relief is different from the mechanism(s) of action for other traditional uses, and perhaps nonspecific skin stimulation will prove to be effective only for the former. Or perhaps both. So long as there is any question, it’s not plausible to suggest the Toothpick as an inactive control for pain studies.
@ jane:
But when the toothpick does work as well as real acupuncture, that does provide extremely strong evidence (as if more were needed) that acupuncture is pure placebo.
Beamup – Are you asserting that – IF you had not already made up your mind that needling could have no effect on the nervous system – you could still be certain that gouging someone with a toothpick hard enough to make them think their skin was punctured could have no effect on the nervous system?
As a Dr. who sees patients with COPD, I say, “So what?” Even if the accupuncture worked, it only helped during the six minute walk. How long does that affect last? If they repeated the Six minute walk one hour after the first would the effect still be there? What about the next day? If not, then the patient would need accupuncture before every walk from the car to the store. I guess if every patient had his own personal accupuncturist it may be helpful. At least an inhaler does not require its own healthcare coverage.
‘Different placebo effects’? I’m having trouble parsing what you mean by this (I’m hoping you’re not trying to suggest that the efficacy of all conventional modalities isn’t due to placebo effects that somehow magically target specific disease states).
Can you clarify your meaning, and indicate what evidence suggests conventional modalities might have these differing placebo effects?
I think that you are drastically underestimating the capacity of the placebo effect to produce misleading results, in defiance of the empirical evidence we have that tells us just how deceiving the results can be. Have you ever heard of Clever Hans? Probably you have, but in case not, I’ll go over it for you.
Clever Hans was a horse who reportedly could do mathematics. Ask Hans a question like “What is 2+2?” and Hans would tap his hoof four times. Ask “What is 4+6?” and Hans would tap his hoof ten times. It was considered proof of the amazing genius of this horse… until it was discovered that Hans could not answer when he was blindfolded. Or when none of the humans around Hans knew what question Hans had been asked, and therefore the correct answer.
What was actually happening was not that Hans could do math; what happened was that Hans had learned to react to signals given off by the humans around him. When they asked him a question, they expected Hans to start tapping his hoof; Hans learned to respond to those signals by, indeed, tapping his hoof. When he had tapped his hoof the right number of times, the people sent a different signal; they expected him to stop tapping, and he learned to respond by stopping his tapping.
That is an example of expectations being communicated subtly, and unconsciously – and quite successfully! – by people who had no idea they were doing it, to an animal that (while clearly quite smart for its kind) does not begin to approach the intelligence of a human. If you are telling us it staggers the imagination to conceive that a human being who is aware of whether he performed an “effective” treatment or a “placebo” treatment might subtly and unconsciously communicate to the patient his expectations about what results that treatment will have, I’m afraid the only thing that tells us is the limited nature of your imagination.
No, I do not. Frankly, that sounds so bizarre that I would have to see the actual comment in question before believing that your description of it is correct in all relevant aspects.
Sorry, should have read “trying to suggest that the efficacy of all conventional modalities is due to placebo effects”.
jane:
I’m not saying it is The Mechanism, just that I don’t see a plausible way for it to be; I was responding to another poster who had suggested it as a possibility, not attempting to say it was the be-all and end-all.
Perceived shortness of breath…. I guess it’s possible it might reduce that, but again, the study appeared to show changes in objective measures, not just perceptions, so we need to look further. (Especially since perceived shortness of breath is especially deceptive when it comes to asthma and COPD.) Either the results are misleading, or there is something more going on than just stimulation of rib muscles — that was my point.
Brian34OSU:
I keep thinking that as well. I am an asthmatic, personally. It’s quite possible for an asthmatic to delude themselves that they’re better for six minutes, and I would expect the same to be true of COPD patients. How does it compare to bronchodialators?
Another thought just occurred to me: time and cost. If it takes ten minutes of acupuncture to achieve this effect, it’s worthless. Bronchodialators work almost immediately. And then there is cost; I can’t imagine an acupuncture session costing less than, oh, $40. My albuterol rescue inhaler costs about that — but contains 200 metered 90 mcg doses. So even if this study is entirely accurate, it’s faster, cheaper, and more effective to use albuterol.
Another thought occurred to me — how long does it take to perform the acupuncture? Is it possible that the acupuncturists were taking more time when they were doing it for real than when they did the placebo one? If so, perhaps the effect is explained at least partially by patients getting a chance to rest. You have to stay still during acupuncture, after all.
Those are some head-scratching results, Jane. I haven’t the faintest idea what to make of them as described. I admit too much laziness to try getting and reading that much literature… At least it’s good to know that researchers are aware of the method.
Antaeus: I’m sure Jane could elaborate, but it sounded to me like a reference to a practitioner appearing Asian affecting results. I don’t know who said what, but FWIW I have actually heard comments made elsewhere that if one seeks acupuncture, one should select an Asian practitioner. Even if I hadn’t heard that, I could easily imagine people making that assumption.
Brian @ 47 and Calli @ 56 and 60:
Even if EMG results had no correlation to relief of objectively-measurable primary symptoms of illness, and even if Albuterol costs pennies a dose compared to acupuncture, to my mind there are interesting “pure science” issues here that deserve to be examined critically.
That is not much different to wanting to understand the mechanism of action for a drug that is no longer prescribed because there are better drugs available now. It may be “academic” but it’s still interesting.
So here we have a study that apparently reports some improvement in a couple of subjective measures, and we also have a claim by Jane that “de qi” is a reliable subjective indicator of an effect.
OK, now those items raise interesting questions: what’s the mechanism? As someone who deals with electrical and electronic systems all the time (PBX engineer), the first thing that comes to mind is some kind of interference with transmission of nerve signals. There’s no need to supernaturalize that any more than to supernaturalize an herbal remedy that appears to have some kind of active ingredient that actually does something.
Useful technique: ask yourself “what would change if I was wrong?” For example, imagine a scenario where there is a robust report of some kind of symptomatic relief or palliative relief, and this was supported across multiple studies with a total N that was acceptable. Imagine that the results were strong enough to persuade skeptics that the effect was real. So now everyone ends up agreeing, for example, “OK, acupuncture really does produce measurable pain relief for 20% of the population,” and asking “So what?, we have excellent medications that produce equal or better relief for 88% of the population.” The answer to “so what?” is, it would be interesting to know the mechanism involved, and as a clinical matter it’s useful to have additional types of treatments available for whatever small number of people don’t respond to the preferred treatment.
We need to get over our fear of being gloated at by people we dislike, and really stick to the objective stance here. IF acupuncture is ever shown to have any value at all, we should be willing to acknowledge it and incorporate it into SBM for whatever it’s worth, even as a minor treatment modality for people who can’t take whatever the preferred treatment is. And then we should move on without getting a grump about it.
As for where I’m coming from on this:
All other factors equal I don’t like needles. I wince slightly for vaccines but sometimes I wince when I vote too, and both of those are civic duties. I wince a bit more for ordinary veinous blood draws, and a bit more than that for IV lines in the back of the hand. I get downright scared when I hear the words “phlebotomist” and “arterial blood draw” with my name in the same paragraph, and remind myself to “practice the Buddhist art of detachment from suffering” when hiding from the phlebotomist is not an option.
There’s no way I’d volunteer for acupuncture even if I “believed” that those needles weren’t going to hurt: they’re *needles* dammit, and not only that, they get *twirled*! But Keatsian negative capability requires the willingness to consider what might happen IF acupuncture turned out to actually do something, and on balance the answer to that is, it might be useful for a small handful of patients, and it shouldn’t be a big deal. In any case the opportunity to learn something new about the functioning of the human organism is always interesting, whether the results come in positive or negative.
I really liked what you had to say aboutAcupuncture and COPD? Not so fast . I found that http://tinyurl.com/bu853dg really helped me with b vitamin intake using natra boost….
I’m also someone who works with electronic systems, signal processing, etc, and I think before we work out the mechanism, we need to better characterize the effect. Otherwise, we’re putting the cart ahead of the horse. It’s fine to speculate, but we need to be careful that this speculation doesn’t inform characterization of the effect — otherwise, we will find what we seek whether it is there or not. I have seen the same thing with computer systems, where an effect was seen, a mechanism was postulated, they investigated and found what appeared to confirm the mechanism, and then later we realized the effect was false; it was a consequence of the testing equipment and not actually part of the system. Long story short, it ended up being a subtle debounce effect in a physical switch that behaved differently in different test environments. That problem report was open for ages. If we hadn’t found what looked like confirmation of the first theory right away, though, we would probably have figured it out sooner. We hadn’t even contemplated the equipment, since it had all been proofed by our customer, we were performing to spec, and, critically, we thought we knew what was going on. But we hadn’t adequately characterized the effect in the first place.
So we need to first be sure an effect actually exists; this study doesn’t exist in a vacuum, and other studies haven’t been as favorable. Studies can return positive results purely by chance, so it needs to be replicated. And then it needs to be extended; if we want to learn more about the functioning of the human organism this way, we need more data to get a larger picture of what is happening. I think if we spend too much time thinking about mechanism at this stage, we are putting the cart ahead of the horse. I will be less concerned about that once the study is replicated.
But I will still feel that it’s irresponsible for this to be proposed as a treatment for COPD or asthma on the basis of this trial. If it’s inferior to existing treatments and costlier, then it should not be preferred. Certainly you can’t carry your acupuncturist around with you like you can your rescue inhaler. 😉
A couple of you seem to have begun to believe that this effect works, and this study was organized, in such a way that the patient must leap from the acupuncture table to the six-minute walk test. Not so. If that were the case, the first eleven weeks of treatment would have been superfluous. You don’t have to have ten minutes of acupuncture immediately before you walk anywhere; after a course of acupuncture, you walk better when separately tested. If this is because of a placebo effect (or as Calli interprets that, you are “deluding yourself that you’re better”), well, most people would happily be “deluded” if it means they can get through the grocery store without sucking wind.
JGC – There is an entire book about the evidence for varying levels of placebo effects: Meaning, Medicine and the “Placebo Effect”, by Daniel Moerman. This is not a controversial concept among those who have looked into the literature. For example, in most Western cultures a placebo injection provides more pain relief than a placebo pill, because people assume that an injection is a more potent treatment. It is reasonable to suspect that major surgery is viewed the same way. To return it to the field of acupuncture, there are studies in which acupuncture outdoes pharma drugs. If the acupuncture is entirely Placebo, it beats the placebo plus mechanistic effects of the drug, and therefore is a much more powerful placebo.
Then I retract my comparisons to albuterol. Comparisons to steroids would be more apt, though this makes the results much less impressive.
But if it *is* just delusion, then these people are putting themselves at risk, because it is entirely possible with both asthma and COPD to *feel* better without actually improving airflow. I’m not saying it *is* just delusion, just that we need to be very careful about that possibility, because it is very dangerous in these diseases.
It is certainly true that there are varying levels of placebo responses, though. I wonder sometimes whether this explains the perceived effectiveness of chelation therapy in nonstandard applications, such as athersclerosis and autism.
@Jane,
“There has traditionally been a strong tendency to publication bias in Asian countries, and there may still be,…”
There are many possible reasons for entire countries publishing exclusively positive results of acupuncture trials; publication bias is but one. We don’t actually know the reasons for the findings that I mentioned.
“Doctor Atwood’s cheery statement that we can therefore “summarily dismiss” all future heretical papers with Asian authors is disastrous for multiple reasons.”
I did not write that we can summarily dismiss all future papers with Asian authors. Read my comment again.
“the fact that he has not had served to him on a silver platter a Western review showing that Japanese scientists are known to publish negative results does not mean that they never do.”
What I had served to me, and I served to you, was a review (“Western” is irrelevant) showing that between 1966 and 1995, acupuncture reports from Japan that met reasonable study design criteria were exclusively positive. It is certainly possible that some of those reports would have been favorable to acupuncture even in the absence of whatever the biases were that poisoned the collection as a whole, but we have no way of knowing which those might have been, and we continue to have no way to know. That’s why summary dismissal remains reasonable.
Iâm well aware that Japanese scientists publish negative results of trials of other methods. What we don’t know is whether the collection of acupuncture reports from Japan since 1995 has included enough negative examples to convince us that the previous, longstanding bias, whatever its basis might have been, is no longer in effect.
You are correct that I have âno plans to do that work.â So what? The Vickers review bears repeating, and I hope someone does it. Itâs not what I do, but that hardly disqualifies me or anyone else from acting upon the current evidence until new evidence convinces us otherwise. Those who have the greatest stake in convincing us otherwise are Japanese scientists themselves, of course. It is their credibility that is undermined, not by me, but by the pertinent history of bias among some of their peers.
The remainder of your comment is a mishmash of ad hominem and tu quoque arguments (antidepressant trials? We’re talking about acupuncture), culminating with the implication that my comment somehow suggested that “Western” or “American authorities” are superior to “the entire community of Japanese researchers.” My comment did nothing of the sort. Please attempt to argue points on their own merit.
I’m not sure why it makes the results less impressive; I would say more impressive, since it means acupuncture has not just an immediate effect but a lasting one.
Feeling better doesn’t put people at risk; ceasing to take long-term medication might. You’ve said before that you suffer from asthma, so I’m sure you don’t mean to suggest that asthmatics should avoid anything that relieves their symptoms simply because it might reduce their motivation to take long-term medication. I can’t imagine arguing, as a parallel example, that a drug that relieved angina would be bad because people who are having chest pain are more “compliant” with statins. If there is evidence that long-term prophylactic use of a drug offers a net statistical benefit even when it does not improve current well-being, educating patients to that effect is part of a doctor’s basic job description.
Doctor Atwood is skilled in the rhetorical style of scientism, I admit, but when people are actually trying to make intellectual progress, it does not suffice to scream “tu quoque” when the implications of one’s starting position are pointed out. We know that Asian acupuncture researchers used, as a general rule, not to seek publication for studies with negative results. We know, too, that that was a typical practice among other Asian researchers, not just people working on acupuncture. We also know that it was the profit-motivated practice among Pharma-funded Western researchers.
Now, Doctor Atwood declares that we can automatically discard every single positive publication on acupuncture by Asians. He is very anxious, if incoherently so, to make it clear that he does not wish to apply that to Asians who publish positive results on methods accepted in American allopathy? But why not, logically? And should it apply to positive results reported by Western antidepressant manufacturers, who at the time period of the review mentioned were just as unwilling to publish negative results as the Asian acupuncturists and for more sinister reasons? Why, that’s “tu quoque” – which is usually just an overeducated snot’s way of saying “Don’t be ridiculous!” But why not? If the application of Doctor Atwood’s principles depends so totally on who he is judging and what he thinks their beliefs are, then whatever his principles are, they’re not rational.
Um, no. No in a big way. No, in fact, on such a level that it really brings into question whether you ever do any sort of research at all.
“Tu quoque” does not mean, by any stretch of the imagination, “Don’t be ridiculous!” It means “What you have said, pointing out the flaws in System B, does not actually have anything to do with the subject under discussion, which is System A. You are changing the subject.” Note that this has nothing to do with anyone or anything being “ridiculous.” Note that it has nothing to do with whether the criticism of System B is true or false. It’s still changing the subject.
Now what strikes me is that you could have spent five seconds with Google and discovered what tu quoque actually meant. Instead you claimed something completely false. I suspect that tells us a lot about how well your defense of acupuncture is researched. (Of course, the alternative is that you actually knew what tu quoque meant, but chose to pretend you didn’t, so you could respond sneeringly with “over-educated snot!” to a straw man criticism instead of addressing the valid criticism that was actually made. I can’t say that reflects any better on you.)
Ah, Jane. You have a flare for misrepresentation. I did not write “by Asians.” I wrote “from Japan.” I could also have written “from China” or “from Taiwan,” if the context had called for it, but I did not write “by Asians.” I trust that readers will recognize the significance of the distinction.
We do not “know that Asian acupuncture researchers used, as a general rule, not to seek publication for studies with negative results.” What we know, simply, is that for at least 30 years, acupuncture reports from certain countries were uniformly ‘positive.’ We don’t know why that was: we don’t know, for example, whether there were any “studies with negative results.” As previously stated, there are many possible reasons for the finding, but it is the finding itself that casts doubt on the entire project.
Contrary to another of your misrepresentations, I do not “judge” the authors of those reports or pretend to know “what their beliefs are.” Merely knowing that every report was favorable to acupuncture, whatever the reason, is sufficient grounds for dismissing them all, even acknowledging that some may have been legitimate. The point is that we have no way to decide which to take seriously. That conclusion is rational, but you are free to disagree with it. If so, please offer a rational argument.
By the misnomer “American allopathy,” may I assume that you were referring to modern medicine? That is a world-wide phenomenon and has nothing to do with the non-existent “allopathy.”
Regarding antidepressant trials and tu quoque, let’s assume that every trial funded by antidepressant manufacturers was favorable. Would that validate acupuncture reports from Japan, or would it merely damn the antidepressant reports? The latter is obviously the case: two wrongs don’t make a right (which is one way of explaining tu quoque). I do not give a free pass to antidepressant trials, although they’ve not been uniformly favorable. I’m well aware of the bias in that literature and have written about it–which I mention not to suggest that your bringing it up was relevant to the issue at hand, but to demonstrate that my views are consistent and not based on the ethnic or nationalistic prejudices that you seem to imagine:
http://www.sciencebasedmedicine.org/index.php/annals-of-questionable-evidence-a-new-study-reveals-substantial-publication-bias-in-trials-of-anti-depressants/
It may be tempting to ascribe irrational or prejudicial motivations to what others write, but your arguments will be taken more seriously if you resist stating as much. Go with the evidence: no more, no less.
I was actually diagnosed with an autoimmune disorder, lichen planopilaris, and after a year of seeing no improvement with the medications I was prescribed by my dermatologist, I began seeing an acupuncturist but still continued my treatments with the dermatologist and I saw a huge improvement. I told my dermatologist about the acupuncture and he said that it does actually help with lichen planopilaris. I eventually had to stop the acupuncture because it was too expensive and the insurance doesn’t cover it, but my dermatologist had wanted me to continue. After my entire experience, I am a believer in acupuncture.
@ Rob: You would have to provide us with more details…including the location on your body of your lichen planopilaris and including…why you *believe in* acupuncture.
(Citations please, from reputable sources, about successful treatment of your disorder with acupuncture)
Of course, your medical insurance refuses to pay for any more acupuncture *treatments*, Rob. I’m willing to venture an educated guess, that your insurance carrier would pay to test you for the hepatitis C virus:
http://www.ccjm.org/content/76/10/599.full
Scroll down the article under “Porphyria cutanea tarda-lichen planus”, see figure 4.
Doctor Atwood is skilled in the rhetorical style of scientism
What in the name of feck is “the rhetorical style of scientism”? Does it mean “sounding smarter than Jane”? That is the conclusion I derive from her later complaint that he is “an overeducated snot”.
it does not suffice to scream “tu quoque”
Is Kimball Atwood using “the rhetorical style of scientism”, or is he “screaming”? I wish Jane would make up her mind.
Jane comment 65,
Is it just me or did this study not say how long after the 50 minutes accupuncture session they performed the 6 minute walk? I read the methods and reuslts section twice and do not see where they said this. It is like reporting results of a drug but saying how soon after taking it you may expect relief or more important to my comment, how long the effect should be expected to last. What is the half life of the accupuncture? If they did the test immediately after the accupunture testing, then yes, the previous eleven weeks would be superfluous. If you tell me they did the six minute walk immediately before the accupunture session you could say the effect lasted one week. But what you cannot say based on this study is that 12 weeks of accupunture lasts longer than 1 week. They could easily have done a “dose duration” study and found how long it takes for this effect to go away. Just do the six minute walk every week after stopping treatment and see how long it lasts.
But they didnt. If they did the test immediately after the session you could not say anything more than accupunture increases the six minute walk time immediately after the session, and thus you would need an accupuncturist to help you go grocery shopping.
I’m not a acupucture practitioner, but I don’t think (correct me if I am wrong?) any classically trained acupuncturist would ever needle ONLY those points as part of a treatment protocol–for COPD or anything else. Is it possible that the study takes the few selected points used seriously out of context, thereby subjecting to scientific scrutiny only PART of a possible treatment protocol?
Oh lookie! Spambots already.
Simply desire to say your article is as astounding. The clarity in your post is just cool and i can assume you are an expert on this subject. Well with your permission allow me to grab your feed to keep up to date with forthcoming post. Thanks a million and please continue the gratifying work.
I was always doubtful about acupuncture until we tried it on my old dog, who could walk ok, but had problems with backing up and turning. Someone suggested it had helped his dog who had a bad hip and was too old for a hip replacement. The first time we took him, he didn’t like the needles going in one bit, but soon he relaxed. On the way home, it was obvious something was happening. He seemed to be able to turn around and move better. By the third time we took him, he practically was dragging me into the vet’s office, and would almost instantly go to sleep as soon as the needles went in. He gained weight back to his lifetime weigh of 75 pounds and stayed there, even after it was discovered he was loaded with cancer. The day he was diagnosed, he got the needles and went on a 2.5 mile walk. We finally had to put him down about 10 days later. I give AP the credit for making his last months pretty good ones.
Dave F, just a reminder:
The plural of anecdote is not data.
And my sympathies in losing your dog.
(and could the folks at Nat Geo do something about the spam!)
One of the things that I would imagine is difficult in creating a double-blind experiment is hiding the fact that acupuncture hurts. I’ve had treatments — having sought them out in desperation — and no one was more surprised than me when the treatments were successful. Darn my embarrassing weak-mindedness!
Oh, and incidentally, Chris, the plural of “anecdote” is “qualitative data”.
The writer or this article sounds very biased against acupuncture. I take a homeopathic remedy for my asthma, Lycopus Virginicus, and it is very effective. IN fact since I started taking it I use my rescue inhaler rarely. Placebo?
I don’t think so , I tried several other remedies first, why would this one be a placebo?
The writer of this article is biased against ‘treatments’ with no plausible mechanism and no evidence of their efficacy. Why aren’t you biased against such ‘treatments’ as well?
Do you think you are competent to evaluate whether or not any change in your experience is due to placebo effect?
(hint: nobody is)
Yes, complete placebo. It’s just plain water. The variability of placebo effects is very interesting, so nobody can say with confidence why this particular one seemed to work.
But that homeopathy is complete placebo, and utter fraud, is not in question. Acupuncture isn’t quite as bad, but still thoroughly demonstrated to be pure placebo.
Question: what type of asthma do you have? Asthma has many triggers. Stress can be a trigger for some people; for them, placebos can seem very potent. Also, many cases of asthma wax and wane over time; are you sure it did not just go into remission? Please keep your rescue inhaler handy just in case. (Helpful hint: a US Army study found that even in bad storage conditions, i.e. a forward operating base in Afghanistan, those are good for much longer than the sell-by date on the package.)
I have exercise-induced asthma. One of the great things about that is that it’s amenable to treatment by exercise. You have to be gentle about ramping up, but once you get there, it can very much improve your asthma control. I also rarely use my rescue inhaler, and haven’t used a controller medication in over a year. It’s possible I’ve just been lucky, but I’m going to stick to the exercise regimen anyway, because it’s good for me in other ways as well.
Yes, placebo. All you’re taking is water, after all.
So, let me get this straight: a chiropractor pops in to complain about how acupuncture is being treated in an article and, as evidence, offers an anecdote about using homeopathy for asthma?
Narad: it’s the Triple Crown of woo! And unfortunately the three most mainstream, and “legitimate” forms of woo. My mother just told me last night her friend swears acupuncture helped her stop smoking. I asked her to find out what else her friend used, before jumping to conclusions.
I haven’t looked into it too much yet but the British skeptic boards are discussing a court decision today that will make it almost impossible for homeopathy to continue in its present form in the UK. Maybe Orac can check that out in coming days.
Mr. Calhoun:
Sorry I missed that, I was a bit busy that week.
Which is not a very good form of data either, and that statement is still false. For qualitative data to be any good there has to more than an N=1, and it has to have a consistency in its collection. Not third party stories about a dog and how it felt.
It wouldn’t surprise me in the least if acupuncture helped some people quit. There’s a significant top-down psychological component in play. (E.g., PMID 16237578.)