Another acupuncture study misinterpreted

ResearchBlogging.orgI have to hand it to acupuncture mavens. They are persistent. Despite numerous studies failing to find any evidence that acupuncture is anything more than an elaborate placebo whose effects, such as they are, derive from nonspecifice mechanisms having nothing to do with meridians, qi, or “unblocking” qi. Moreover, consistent with the contention that acupuncture is no more than an elaborate placebo, various forms of “sham” acupuncture (needles that appear to insert but don’t or acupuncture in the “wrong” locations, for example) produce results indistinguishable from “real” acupuncture.

That record won’t change with the latest acupuncture study for low back pain that was published on Monday and is making the rounds through the media. Let’s start with a news report on Medpage Today:

WHEELING, W.Va., May 11 — Acupuncture was more effective than conventional treatment for relieving lower back pain in a randomized trial, but performed no better than poking patients gently with toothpicks.

The editors of Medpage Today should really know better than to publish nonsense like this. They shouldn’t have allowed a story about this particular study to start out by saying that acupuncture was bound to be “more effective” than conventional treatment because this study showed nothing of the sort, for reasons that I’ll discuss later. Actually, I bet that astute regular readers here will be able to identify immediately exactly why such a conclusion is unjustified when I describe how the study was done. If not, I promise I’ll make it painfully clear to you by the end. I’ll also feel like I’m repeating myself because I’ve mentioned this very same defect in acupuncture studies. In any case, the study did show that “real” acupuncture was no better than sham acupuncture.

Back to the study. It appeared in Archives of Internal Medicine and was published by a team of investigators led by Dr. Daniel Cherkin of the Center for Health Studies in Seattle; investigators from Northern California Kaiser Permanente, Cancer Research and Biostatistics in Seattle; Department of Family Medicine, Oregon Health and Science University; and (of course) the National Center for Complementary and Alternative Medicine, National Institutes of Health. Yes. NCCAM had a hand in this study. Is anyone surprised? In any case, the study was entitled A Randomized Trial Comparing Acupuncture, Simulated Acupuncture, and Usual Care for Chronic Low Back Pain (Arch Intern Med. 2009;169(9):858-866).

The study was a four-arm study that enrolled 638 adults with chronic mechanical low back pain. The group, which I quote from the study, were:

  1. Individualized acupuncture. This was the treatment prescribed by the diagnostician at the beginning of each visit. It could include any acupuncture points that could be needled with the participant lying prone. There were no constraints on number of needles, depth of insertion, or needle manipulation. Treatments averaged 10.8 needles (range, 5-20) retained for 18 minutes (range, 15-20 minutes). Seventy-four distinct points were used, half on the “Bladder meridian” that includes points on the back and legs.
  2. Standardized accupuncture. We used a standardized acupuncture prescription considered effective by experts for chronic low back pain.15 This included 8 acupuncture points commonly used for chronic low back pain (Du 3, Bladder 23-bilateral, low back ashi point, Bladder 40-bilateral, Kidney 3-bilateral) on the low back and lower leg.14 All acupuncture points were needled for 20 minutes, with stimulation by twirling the needles at 10 minutes and again just prior to needle removal. Therapists manipulated the needles to elicit “de qi,” which they perceive as a biomechanical response in tissue as it tightens around the inserted needle and constricts its movement.
  3. Simulated acupuncture. We developed a simulated acupuncture technique using a toothpick in a needle guide tube, which was found to be a credible acupuncture treatment by acupuncture-naïve patients with back pain.Simulating insertion involved holding the skin taut around each acupuncture point and placing a standard acupuncture needle guide tube containing a toothpick against the skin. The acupuncturist tapped the toothpick gently, twisting it slightly to simulate an acupuncture needle grabbing the skin, and then quickly withdrew the toothpick and guide tube while keeping his or her fingers against the skin for a few additional seconds to imitate the process of inserting the needle to the proper depth. All acupuncture points were stimulated with toothpicks at 10 minutes (ie, the acupuncturist touched each acupuncture point with the tip of a toothpick without the guide tube and rotated the toothpick clockwise and then counterclockwise less than 30°) and again at 20 minutes just before they were “removed.” To simulate withdrawal of the needle, the acupuncturist tightly stretched the skin around each acupuncture point, pressed a cotton ball firmly on the stretched skin, then momentarily touched the skin with a toothpick (without the guide tube) and quickly pulled the toothpick away using the same hand movements as in regular needle withdrawal. The acupuncturists simulated insertion and removal of needles at the 8 acupuncture points used in the standardized treatment.
  4. Usual care. Participants in the usual care group received no study-related care–just the care, if any, they and their physicians chose (mostly medications, primary care, and physical therapy visits). All participants received a self-care book with information on managing flare-ups, exercise, and lifestyle modifications.

I included a full description of each straight from the study because I think it’s important. In fact, with one glaring error, this is actually a very good experimental design for this sort of clinical trial. I’m not going to say what that glaring flaw is right now, as I want you to try to figure it out. Don’t worry, it will be revealed soon enough. But first I want to reveal the reported results of the study.

Back pain was assessed in each patient upon entry into the study using a standard, well-validated questionnaire for back pain, the modified Roland-Morris Disability Questionnaire (RMDQ), and then again at 8 weeks, 26 weeks, and 52 weeks. Strengths of the study included a reasonably number of subjects, a good placebo control group in terms of the simulated acupuncture, and a comparison of “individualized” versus “standardized” acupuncture. I mention this latter aspect of the trial because one complaint that woo-meisters often make is that clinical trials are too “artificial” and do not adequately reflect their “individualized” practice. This trial provided the opportunity to put that claim to the test.

So what was the result? Check out the graphs below, which show two measures of relief of back paint as assessed at the various time points:

i-f45f2faf1dfaffacdf00bd388d960609-ioi80225f3.gif

The bottom line is that there was no significant difference between any of the three acupuncture groups when compared to each other. Individualized acupuncture was no different from standardized acupuncture, and neither were any different than sham acupuncture. True, all of them were reported as better than “usual treatment,” but can you tell what the problem was? Sure, I knew you could. I’ve mentioned it before enough times when looking at other acupuncture studies. While it’s true that the three acupuncture groups were fairly well blinded, as far as studies like this go, there was no blinding at all between the “usual treatment” group and any of the acupuncture groups. The patients in the acupuncture groups all knew that they were assigned to an acupuncture group. They didn’t know which acupuncture group they were in (which is good experimental design), but they knew they were getting acupuncture. Similarly, the patients in the “usual care” group all knew they were assigned to a non-acupuncture group. They knew they weren’t getting acupuncture. That makes any comparison between the two virtually worthless, other than as an estimate of what the placebo effect was during the study.

In fact, the very fact that there was a difference between all the acupuncture groups and the “usual treatment” group is best described as being due to the placebo effect. The reason? There was no restriction on “usual care” among the acupuncture groups. Not just the “usual treatment” group used standard medical treatments. All four groups were allowed to continue using their standard conventional treatments for low back pain, and all four groups did. Consequently, the three acupuncture groups received their acupuncture in addition to whatever nonsteroidal anti-inflammatory drugs they were taking, the same care regimen that the “usual care” group was getting. Given such an experimental design, I would have been shocked if the acupuncture groups didn’t show superiority to the “usual treatment” arm. Such a result was exactly expected based on the expected placebo effect from being in one of the acupuncture arms of the study.

That doesn’t stop the authors from incorrectly concluding about their trial and other recent acupuncture trials:

Collectively, these recent trials provide strong and consistent evidence that real acupuncture needling using the Chinese meridian system is no more effective for chronic back pain than various purported forms of sham acupuncture. However, both real and sham acupuncture appear superior to usual care.

Particularly irritating are news reports like this, with reporters saying that acupuncture “works” and the question is how:

I do, however, express some amusement at the discomfiture of acupuncture boosters in trying to explain why there was no difference at all between any of the three acupuncture groups and being forced to admit that “patient beliefs” (code word among CAM boosters for nonspecific placebo effects) are the reason for their observed results showing more pain relief among the acupuncture groups than among the “usual care” group. True, the discomfiture caused by this study is not as amusing as that caused by a study from about a year ago, which forced acupuncturists to try to explain why the study’s sham acupuncture group had a stronger response than than the “true” acupuncture group. The logical contortions the authors made in order to try to justify that result as somehow being anything other than yet another large study showing acupuncture to be nothing more than a placebo were truly something to behold.

In any case, a more correct way to have done this trial would have been to leave out the “usual care” group altogether because it couldn’t be blinded. Mixing blinded groups with unblinded groups always mucks up a study design, and the sham acupuncture group was a more than adequate control group for this study, if the question to be answered was whether individualized acupuncture or “standardized” acupuncture provided better pain relief than a placebo. Indeed, I can’t help but think that the investigators must have known when they included that group that there would be a placebo response and that in the event they didn’t find what they wanted to find it was likely that the acupuncture groups would appear to do better than the “usual treatment” group. If the reason for that group was to estimate the degree of placebo response, that would be justifiable, but the wildly inaccurate misinterpretation of the observed placebo effect as meaning that acupuncture “did better” than standard care shows the peril of including such a group in studies such as these. Not only did the investigators wildly misinterpret the meaning of the difference observed between the “usual care” group and the three acupuncture groups, but the woo brigade has latched onto this difference as though it really did indicate that acupuncture is “better” than conventional therapy for low back pain. Witness Mike Adams’ lackey over at NaturalNews.com, where S. L. Baker crows Acupuncture Beats Western Medicine for Treating Low Back Pain:

The results of the largest randomized back pain trial of its kind shows acupuncture clearly helps people with chronic low back pain more than standard medical care. But the results of the SPINE (Stimulating Points to Investigate Needling Efficacy) study, just published in the May 11, 2009 Archives of Internal Medicine, has some researchers scratching their heads over the remarkable findings. The reason the study’s results are so intriguing? Not just one but three different forms of acupuncture beat out western medicine in helping relieve low back pain.

Uh, no. What it shows is that there is no difference between “individualized” acupuncture, a standardized acupuncture regimen, or a placebo control sham acupuncture treatment. That’s all it shows. Contrary to the way this study is being spun, it is yet another in a long line of negative acupuncture studies, showing that acupuncture for back pain is no better than placebo. Indeed, it is about as resoundingly negative a study as I can imagine, showing clearly that acupuncture has no specific effects that can be attributed to anything that makes a procedure considered to be “acupuncture”: individualization of needling, needling location according to traditional Chinese medicine precepts, or even bothering to put the needles into the skin at all. None of it matters at all. Indeed, the result showing that all three acupuncture groups produced results indistinguishable from each other is very strong evidence that none of it matters and that it’s all placebo effect. Unfortunately, the NaturalNews.com article is not a Mike Adams flight of fancy; it more or less quotes authors of the study, who speculated that the reason for their results is that the sham acupuncture group was the equivalent of “acupressure” and therefore nothing more than another form of acupuncture. Never mind that, were that truly the case, the “usual care” group would have been the real control group, making their experimental design unblinded with respect to the control group and therefore utterly worthless.

NCCAM strikes again, and your taxpayer dollars are funding it all. Acupuncture is based on a prescientific philosophy of disease, but apparently that doesn’t stop credulous 21st century investigators from wasting taxpayer money producing yet another study showing that it does not work.

REFERENCE:

Daniel C. Cherkin, Karen J. Sherman, Andrew L. Avins, Janet H. Erro, Laura Ichikawa, William E. Barlow, Kristin Delaney, Rene Hawkes, Luisa Hamilton,, Alice Pressman, Partap S. Khalsa, & Richard A. Deyo (2009). A Randomized Trial Comparing Acupuncture, Simulated Acupuncture, and Usual Care for Chronic Low Back Pain Arch Intern Med, 169 (9), 858-866