I know that I’m not going to have a lot of control over my selection of blogging material for a given day when I see more than one or two requests for an analysis of an article. So it was, when links like these were showing up in my e-mail:
- Acupuncture May Help Ease Symptoms of COPD
- Acupuncture May Be Worth a Shot for COPD: Small Study Shows Acupuncture May Help People With COPD Breathe Easier
These two news stories refer to a study from Japan by Suzuki et al published online yesterday in the Archives of Internal Medicine entitled A Randomized, Placebo-Controlled Trial of Acupuncture in Patients With Chronic Obstructive Pulmonary Disease (COPD): The COPD-Acupuncture Trial (CAT). It is just what it sounds like: A test of acupuncture on COPD. First, let’s see what these news stories say about it, beginning with U.S. News and World Report:
For patients with chronic obstructive pulmonary disease (COPD), acupuncture may help relieve shortness of breath during activity, Japanese researchers suggest.
COPD is a progressive lung condition that makes it hard to breathe; it is commonly caused by smoking or exposure to other toxins.
“The effects of acupuncture are large,” said Dr. George Lewith, from the University of Southampton in Hampshire, England, co-author of an editorial accompanying the study. “This is particularly remarkable in a condition that seems largely unresponsive to more conventional treatments.”
And WebMD, whose writers should know better:
Exactly how acupuncture improves symptoms of COPD is not fully understood. Researchers speculate that needling the acupuncture points on the rib cage area may help relax muscles involved in breathing.
This makes perfect sense to Tong-Joo Gan, MD. He is a professor of anesthesiology at Duke University Medical Center in Durham, N.C. It also may help reduce anxiety levels, he says. “When you become breathless, your anxiety goes up, so relaxation is another possible explanation for the benefit.”
Acupuncture has been shown to release chemicals that relax the lungs and dilate the airways, he says.
“Clearly it looks like a viable alternative to treat chronic COPD,” Gan says. “The downside is so little and the upside is so huge that acupuncture is well worth a try for those who find it difficult to control their COPD despite medications.”
Wow! If this study is any indication, acupuncture is the greatest thing since sliced bread, at least for COPD. At least, it is if you believe the hype machine that’s revving up to promote this study, just as it does for any seemingly “positive” acupuncture study. But is it right this time? Is this study really good evidence that acupuncture “works” for COPD? Not so fast, there, pardner. The study, despite the breathless descriptions of it popping up in the press yesterday, is–shall we say?–underwhelming.
The study itself is fairly straightforward in that it is a randomized study of patients with chronic obstructive pulmonary disease (i.e., COPD) treated with standard therapy plus either “real” acupuncture or sham acupuncture. In this case, the sham acupuncture consisted of needles that didn’t puncture the skin rather than needling the “wrong” acupuncture points. The device used was a Park sham device, which comprises a needle (real or blunt-tipped placebo) with a guide tube. The blunt needles appear to penetrate the skin but actually telescope back into the tube. The primary endpoint measured was breathlessness as measured by an instrument called the modified Borg scale after a test known as the six-minute walk test. The modified Borg scale measures from 0 (no breathlessness) to 10 (maximal). They also measured lung functions. Acupuncture treatments (sham or “real”) were administered once a week for twelve weeks, and the acupuncture points chosen were as shown below:
After twelve weeks of sham acupuncture or “real” acupuncture, the placebo acupuncture group (PAG) and real acupuncture group (RAG) were compared for various measurements after the six minute walk test. Again, the primary outcome measured was the modified Borg scale, which is a subjective measurement of breathlessness, with a whole bunch of other secondary endpoints. Whenever I see such a large number of endpoints, I wonder about whether any control was made for multiple comparisons, and, as far as I can tell from reading the methodology, there wasn’t. So what did Suzuki et al find? After randomizing 68 patients, the found a significant improvement in the Borg scale after the six minute walk test. They also reported a small improvement in oxygen saturation (86% to 89%) while FEV1 didn’t change. (The significance of FEV1 was discussed in a previous post about acupuncture and asthma.)
Many of the usual caveats with a study of this type apply. First of all, it’s a small study, and it’s very easy to have a false positive in a small study like this. I have a hard time making much of this study without replication or a larger study. Second of all–and this is the biggest flaw in the study, a flaw so large that in my mind it pretty much invalidates the study–the study was only single-blinded. The subjects were blinded to experimental group, but the researchers and acupuncturists were not. There is no good excuse for this lapse, given how many other investigators have successfully carried out double-blinded acupuncture studies. The authors simply state that “we were unable to mask the acupuncture therapists.” Again, other groups have managed to blind the acupuncturists using specially constructed needles; why couldn’t Suzuki et al?
Another thing that drove me crazy about this article was that the authors piled endpoint after endpoint into tables. About half the endpoints appeared to be statistically significantly different, but with wide confidence intervals. For example, adjusted differences between PAG and RAG in three of eight biomedical measures listed in one table (Table 5) and six of eleven physiological measures (Table 6) were not statistically significant. Others that were “statistically significant” appeared not to be particularly impressive. A lot of these measurements, such as pulmonary function tests and the like, can also be influenced by patient effort, which could easily be affected, either intentionally or unintentionally, by how much the investigators measuring ventilatory function encouraged them. In other words, what we have here is a bunch of outcome measures, subjective and objective but potentially influenced by investigators, that are not particularly impressive in a trial that is not double-blind.
Another thing that one has to remember. I discussed this just yesterday, too. For a treatment that does absolutely nothing to the outcome measures being examined, at a statistical significance level of p < 0.05, by random chance alone we would expect about 5% of studies to find an apparent difference between treatment group and control. Of course, it's worse than that, as I've written about many times before. As John Ioannidis has taught me, because no clinical trial is designed and executed flawlessly and because there are always biases and imperfections in any clinical trial, the number of false positive trials for something like homeopathy (which, being water, does absolutely nothing) will actually be considerably higher than 5%. That's why one has to fall back on the totality of the scientific literature filtered through the lens of plausibility as estimated by basic science considerations. For COPD, the plausibility that acupuncture would be expected to have a physiological effect is slim to none. Perhaps it's not as close to "none" as homeopathy is (acupuncture does, after all, involve sticking needles into the body and it's just barely plausible that that might do something), but it's pretty darned low. Thus, filtering this study through the considerations of prior probability, the lack of double blinding, and the lack of controlling for multiple comparisons, and I am profoundly underwhelmed. That doesn't even take into account the fact that I don't see any evidence that the data were analyzed in a strict intent-to-treat analysis, in which all the endpoints were chosen before the study was undertaken and included in its design from the beginning. There were drop-outs in both groups, but in the RAG group three dropped out because they suffered acute exacerbation due to a respiratory infection. In such a small study, that could easily have skewed the results if a strict intent-to-treat analysis weren’t used.
None of this stops the authors from speculating wildly about a “mechanism” by which acupuncture can allegedly improve lung function in COPD:
We therefore speculate that a similar phenomenon is evoked in the accessory respiratory muscles by needling on the acupuncture points on the rib cage. Decreased muscle tone consequently caused the recovery of the muscle strength in the rib cage, resulting in the increased mobility in the rib cage. Relaxation of accessory respiratory muscles may also contribute to rib cage motion. In fact, the present study showed increases in maximum inspiratory mouth pressure, maximum expiratory mouth pressure, and range of motion in the rib cage at the end of acupuncture treatment.
In this study, vital capacity, FVC, percentage of FEV1, and percentage of DLCO significantly increased after acupuncture treatment. These findings suggest that acupuncture treatment might improve DOE and exercise endurance, at least to some extent, through the improvement of pulmonary function. It is not clear why acupuncture improves pulmonary function; however, we speculate that the relaxation of hyperactivated respiratory muscles and the correction of the autonomic tone might cause the beneficial effect on pulmonary function.22 Further investigations are needed to clarify this.
Go back and take a look at the acupuncture points used. Look at how few of them are actually over the ribcage. Is it the least bit plausible that a mere six needles in the ribcage could accomplish this result? I think not. In fact, I think the acupuncture apologists are doing some major contortions reaching for this “explanation.”
Given the inherent implausibility of acupuncture, combined with the large body of evidence that shows that it doesn’t matter where you stick the needles or even if you stick the needles in the skin. You get the same result, which is indistinguishable from placebo effects. For a study to overcome that large body of evidence, it has to be far more compelling than Suzuki et al. In the end, Suzuki et al is not nearly as rigorous as it has been represented and, as a result, not nearly as persuasive as acupuncture apologists would like you to think.