Clinical trials Complementary and alternative medicine Integrative medicine Medicine Skepticism/critical thinking

A bait-and-switch study of acupuncture in stable chronic angina

This week, JAMA Internal Medicine published a clinical trial purporting to find that acupuncture helps stable angina. Here’s a hint: It doesn’t. It’s a bait-and-switch study that used “electroacupuncture” instead of acupuncture with poor blinding and lack of consideration of prior plausibility.

Acupuncture is theatrical placebo, nothing more. I feel obligated to say that right at the outset in most posts that I do on acupuncture for the simple reason that it’s true. If there’s one thing that’s clear from the mass of acupuncture studies out there, it’s that it doesn’t matter where you stick the needles and it doesn’t even matter if you stick the needles in. The effect is the same: nonspecific and indistinguishable from placebo. This background was in my mind when I saw this Tweet about a new study testing acupuncture versus chronic stable angina:

Oh, lovely, I thought. Another acupuncture study, but this time showing up in a pretty prestigious journal JAMA Internal Medicine. That being said, you might want to know what Dr. Mandrola meant with his little quip about, “In sham-controlled trials then it’s Acupuncture 1 PCI 0.” In this case PCI stands for percutaneous coronary intervention, a.k.a. angioplasty. Basically, what he’s referring to is the paucity of evidence for angioplasty in the setting of stable angina, a condition in which angina is brought on by exertion and is, yes, stable; i.e., not worsening and brought on by a predictable amount of effort. ORBITA was a randomized, sham procedure controlled trial of PCI in stable angina that was basically negative; PCI did not improve exercise tolerance or frequency of angina. He was being sarcastic, of course. Now, don’t get me wrong. PCI is effective in certain clinical situations where cardiac revascularization is urgently required. However, there’s just no high quality evidence that it’s more effective for stable angina than medication and lifestyle modification.

Of course, as Ben Goldacre says, just because there are problems in the aviation industry doesn’t mean that flying carpets work, and similarly just because we still have procedures in medicine with a weak evidence base does not mean that magic like acupuncture works. Yet this study tries to claim that it does. Let’s take a look at this study.

On the surface, this looks like a rigorous randomized sham-controlled clinical trial. Basically 404 patients with chronic stable angina, diagnosed according to the classification criteria of the American College of Cardiology and the American Heart Association that included men or women between 35 and 80 years, presence of angina for more than 3 months with attacks occurring at least twice weekly at baseline, and no significant change in the frequency, extent, nature, and inducing and alleviation factors of angina attacks at baseline. Patients were excluded who had any of the following conditions: previous myocardial infarction; severe heart failure; valvular heart disease; severe arrhythmias; atrial fibrillation; primary cardiomyopathy; psychiatric, allergic, or blood disorders; poorly controlled or uncontrolled blood pressure or blood glucose; other severe primary disease not effectively controlled; heart disease treated with acupuncture within the previous 3 months; pregnancy or lactation; or involvement in other clinical trials. The subjects, after enrollment, were randomized into four groups:

  1. Acupuncture on acupoints in the “disease-affected meridian” (DAM)
  2. Acupuncture on a “non-affected meridian” (NAM)
  3. Sham acupuncture
  4. Waitlist control group (no acupuncture)

Here is a description of the specific acupuncture interventions:

Acupuncture was performed by licensed acupuncturists with more than 3 years’ experience. Participants in all groups except the WL group received 12 sessions of acupuncture treatment (3 times weekly for 4 weeks); each session lasted 30 minutes. We chose bilateral acupoints PC6 and HT5 in the DAM group and bilateral acupoints LU9 and LU6 in the NAM group (eFigure in Supplement 2) according to traditional Chinese medicine theory based on review of the literature28,29 and consensus meetings with clinical experts. The uses of acupoints other than those prescribed were not allowed.

For those of you not familiar with “traditional Chinese medicine” (TCM) theory, here’s a picture of the exact points used:

So, basically, the DAM group was treated with acupuncture at what was, according to TCM “theory,” the “correct” two acupuncture points; the NAM group two different acupuncture points on the same arm; and the sham acupuncture (SA) group, two “sham” points. Of course, it needs to be stated right here that what makes the DAM points “correct” and the NAM group “incorrect” is a “theory” based on prescientific mysticism and “flows” of qi (life energy) through meridians, pathways for the flow of qi that have never been shown to exist. I know that certain aficionados of “complementary and alternative medicine” (CAM) and “integrative medicine” have tried desperately to find anatomic correlates to acupuncture meridians. (Unfortunately, one of those people is now the director of the National Center for Complementary and Integrative Health.) As for what determines what is an “acupoint” and what isn’t, it’s the same prescientific mystical mumbo-jumbo. Remember, as I said initially, there’s a mass of literature out there that is clear in its conclusion that it doesn’t matter where the needles are stuck.

There’s also a bait-and-switch going on here. It’s one that so many acupuncture studies pull, and it’s this. This study examines “electroacupuncture.” In this case:

In addition, auxiliary acupuncture needles were inserted 2 mm lateral to each acupoint to a depth of 2 mm without manual stimulation. This method could ensure the electrical stimulation working on the local points. We used a HANS acupoint nerve stimulator (Model LH 200A; HANS Therapy Co) after needle insertion. The stimulation frequency was 2 Hz; intensity varied from 0.1 to 2.0 mA until patients felt comfortable. Two fixed sham acupoints were used in the SA group as in a previous study.30 Acupuncture needles were inserted at bilateral sham points for 30 minutes but without achieving a deqi sensation. The parameters of acupuncture needles and electrical stimulation were identical to those in the DAM and NAM groups. Patients in the WL group did not receive acupuncture but were instructed to schedule 12 sessions of acupuncture treatment free of charge after the 16-week study had concluded.

So right off the bat here you can see that this isn’t really true acupuncture. There are extra “auxiliary” needles, and there’s electrical current. I don’t know about you, but my history courses never taught me about the existence of equipment capable of generating electrical current two or three thousand years ago, which is when acupuncture is claimed to have been developed by ancient Chinese physicians. Of course, those of us who read this blog know that this history is simply not true; it’s revisionist history. In actuality, acupuncture and TCM were promoted by Chairman Mao Zedong. Acupuncture hundreds or thousands of years ago (or even 100 years ago) doesn’t resemble acupuncture now, and, in fact, what is now considered acupuncture, is only really less than 100 years old. Indeed, the filiform needles currently used in acupuncture are a recent “innovation.” In the late 1920s and early 1930s the pediatrician Cheng Dan’an (承淡安, 1899-1957) resurrected and rehabilitated the vanishing trade of acupuncture by moving the needling points away from the blood vessels. (Acupuncture had fallen into disrepute and even been banned before then.) He illustrated his revisions by painting the new pathways onto the skin of individuals and then photographing them. He also replaced the coarse medieval tools with the fine needles we associate with acupuncture today. It is the reproduction of these new pathways in modern acupuncture textbooks, and the new use of fine needles that modern acupuncture advocates mistake for ancient history. (To his credit, Da’nan eliminated the use of astrology to guide needle positioning.) The addition of electricity to acupuncture is just another evidence-free modification of what in ancient times resembled the “Western” treatment of bloodletting than anything else.

So, what’s being tested in this study isn’t acupuncture. It’s the use of electricity deceptively rebranded as acupuncture. As Steve Novella notes, electrical stimulation is a treatment unto itself, and “electroacupuncture” is just a deceptive way to rebrand electrical therapy as acupuncture in a way that the two can’t be separated, hence my calling this study a “bait-and-switch.” Thus endeth the history lesson. Let’s move on to results:

Wow. That looks pretty impressive, doesn’t it? DAM reduced angina attacks by more than 50% compared to the no treatment control and by nearly 40% compared to the NAM and SA groups. As Edzard Ernst notes, the results of the trial look too good to be true, the separation of groups too clean and exactly what most people would expect. Actually, not quite. There’s very little difference between the curves for the waitlist (WL) no treatment control and the SA or NAM groups. I might have missed it, but I didn’t see any statistical analysis reported whether there was a difference between the WL group and the SA or NAM groups. I question whether there was a difference, and if there wasn’t that, as Novella also noted, flies in the face of what we expect even if electroacupuncture were effective for angina, which would have been for there to be a significant placebo effect for the SA and NAM groups and for the DAM group to outperform them. It’s all very fishy and suggests that blinding was not adequate, particularly given that the primary outcome was subjective. Cardiac monitoring, such as a Holter monitor, could have been used to look for evidence of acute cardiac ischemia during these angina attacks, to see if the subjective symptoms correlate with objective measures of decreased blood flow to the heart.

Of course, there’s another issue here, one that’s critical for any randomized, placebo- or sham-controlled clinical trial, and that’s double blinding. It turns out that double-blinding is something this trial didn’t have:

Patients in the 3 acupuncture groups were treated in a single treatment room and blinded to which acupuncture method they would receive. The outcome assessors, data collectors, and statisticians were blinded to group allocations during the study.

Note who wasn’t blinded. That’s right, the acupuncturists providing the treatment were not blinded. Similarly, there was no attempt to determine the adequacy of blinding after the study by asking the patients the DAM, NAM, and SA groups which group they thought they were in. It’s known that the interaction with an acupuncturist has a great effect on the magnitude of placebo effects. That’s why the practitioners need to be blinded. I’m sure that the authors of the study would argue that they couldn’t do that because of the design of the study because the acupuncturists would know which were real acupuncture points and which were not. This has not been a huge problem for other acupuncture studies, or the investigators could have used any of a number of sham needles and just stuck with the “correct” acupuncture points.

There are a couple of other fishy things about this trial. For example, the study was concluded four years ago. Why the delay in publication. Another issue is that acupuncture was used as an adjunctive therapy; no effort to see if another reason could account for the result, such as patients in the acupuncture group being more adherent to their medication regimen.

Then there’s a more general reason to be suspicious of this study, two actually. First, as Edzard Ernst notes:

The authors claim they have no conflicts of interest. This may well be true as far as financial conflicts of interest are concerned, but I have long argued that, in SCAM, ideological conflicts are much more powerful than financial ones. If we look at some of the authors’ affiliations, we get a glimpse of this possibility:

  • Acupuncture and Tuina School, Chengdu University o fTraditional Chinese Medicine, Chengdu, Sichuan, China
  • Department of Acupuncture, Hospital of Chengdu University of Traditional Chinese Medicine, Chengdu, Sichuan, China
  • Acupuncture and Tuina School, Hunan University of Traditional Chinese Medicine, Changsha, Hunan, China
  • Acupuncture and Tuina School, Guiyang University of Traditional Chinese Medicine, Guiyang, Guizhou, China
  • Acupuncture and Tuina School, Shaanxi University of Chinese Medicine, Xianyang, Shaanxi, China
  • Acupuncture and Tuina School, Yunnan Provincial Hospital of Traditional Chinese Medicine, Kunming, Yunnan, China

Indeed. I’ve argued this same thing in different contexts. There’s another general reason too, which both Steve Novella and Edzard Ernst noted, and that’s that the study came from China. Why is this relevant? Multiple analyses have shown that virtually no studies of TCM, especially acupuncture coming out of China, are negative, which strongly suggests bias. Indeed, a 2014 review found that, of 847 reported randomized clinical trials of acupuncture in Chinese journals, 99.8% of these reported positive results, which is remarkable and would be suspicious even if acupuncture worked. Also, data fabrication and scientific fraud are rife in China.

In the end, no one study can demonstrate efficacy of a treatment, particularly one like acupuncture, except under rare circumstances in which a treatment is so massively effective, that the effect size is so massive that it couldn’t possibly have come about due to flaws in the trial design, failure of blinding, and systematic bias. (Such trials are rare.) However, there’s more than that reason to doubt that this trial shows much of anything. Certainly, at the very least, it doesn’t show that acupuncture works for chronic angina for the simple reason that it never tested acupuncture; it tested electrical current. Other reasons include the poor blinding, subjective outcomes, and the overall problem with acupuncture studies that come out of China. Clearly the JAMA Internal Medicine peer reviewers were unaware of these meta-problems with acupuncture studies in China, and seemed not to be troubled by all the other problems with the study. Add to that the lack of basic biological plausibility, and this study tells us basically nothing. Certainly, it doesn’t demonstrate that mystical magical meridians exist through which a mystical, magical life force energy flow, a flow that’s unblocked somehow by sticking needles in these points, with healing effect.

Both clinical trial design and scientific plausibility matter. This study has neither. It’s a “bait-and-switch,” nothing more.

By Orac

Orac is the nom de blog of a humble surgeon/scientist who has an ego just big enough to delude himself that someone, somewhere might actually give a rodent's posterior about his copious verbal meanderings, but just barely small enough to admit to himself that few probably will. That surgeon is otherwise known as David Gorski.

That this particular surgeon has chosen his nom de blog based on a rather cranky and arrogant computer shaped like a clear box of blinking lights that he originally encountered when he became a fan of a 35 year old British SF television show whose special effects were renowned for their BBC/Doctor Who-style low budget look, but whose stories nonetheless resulted in some of the best, most innovative science fiction ever televised, should tell you nearly all that you need to know about Orac. (That, and the length of the preceding sentence.)

DISCLAIMER:: The various written meanderings here are the opinions of Orac and Orac alone, written on his own time. They should never be construed as representing the opinions of any other person or entity, especially Orac's cancer center, department of surgery, medical school, or university. Also note that Orac is nonpartisan; he is more than willing to criticize the statements of anyone, regardless of of political leanings, if that anyone advocates pseudoscience or quackery. Finally, medical commentary is not to be construed in any way as medical advice.

To contact Orac: [email protected]

17 replies on “A bait-and-switch study of acupuncture in stable chronic angina”

It is very difficult to conceive why, and how, this report came to be published in JAMA, and 4 years after the event.

As an erstwhile neurophysiologist, with limited knowledge of angina, I question whether the number of attacks was a suitable parameter, or whether it should have been the number and intensity. I accept that intensity would have been a subjective measurement, but, if it is relevant, I believe that some effort could have been made to measure changes in intensity between attacks.

“men or women between 35 and 80 years, presence of angina for more than 3 months with attacks occurring at least twice weekly at baseline”

It was stated that the subjects were randomised into 4 groups. But, i note 3 variables, age, sex and number of attacks at base line, within the subject population. I believe that valid comparisons (i.e. apples v apples) could only be made if randomisation was based on at least 2 strata (age and sex), but possibly on the 3 strata. Stratified random samples.ensure that the experimental data reflects the influence of the experimental variable, without effects from the population variables.

The graph was amateurish crap. Firstly, they seem to have determined base line numbers by averaging the number of attacks reported in each group, and then continued to measure outcomes by that method. Even with stratified groups this would be a rather crude set of data. A more meaningful statistic would be the % change in the number of attacks for each stratum relative to its base line. And, from the raw data the significance of the changes should be rigorously tested with appropriate statistical tools.

But, my concerns seem rather trivial when considered in the light of Orca’s comprehensive appraisal of the farcical circumstances and methodologies of the mysterious paper that managed to surface in JAMA contrary to the high standards expected of that journal.

Seems to me the details of the farcical methods are rather trivial in light of the fact that an allegedly “pretty prestigious journal” looked past them and published the thing anyway. Why?, one wonders. Orac says “clearly the peer reviewers were unaware of these meta-problems…” This reminds me of the kinds of “he just doesn’t understand” breaks journalists are always giving Trump, Look, I’m just a modestly informed lay person, but after I followed the link, skimmed the actual paper, even I could see a few honking bright red flags. Or perhaps, to steal a joke from an SBM commenter, Red flags:

Funding/Support: This trial was supported by grants… from the National Natural Science Foundation of China and… from the State Key Program for Basic Research of China.

Additional Contributions: W…The 5 clinical centers included the Hospital of Chengdu University of Traditional Chinese Medicine, the First Affiliated Hospital of Hunan University of Traditional Chinese Medicine, Yunnan Province Hospital of Traditional Chinese Medicine, the First Affiliated Hospital of Guiyang University of Traditional Chinese Medicine, and the Affiliated Hospital of Shaanxi University of Chinese Medicine.

So what we appear to have here is vertically integrated acupuncture apparatus from research to treatment institution to practitioner all funded and/or operated by one arm or the Chinese government or another.

Which, again begs the question ‘why?’. According to Orac, Mao promoted acupuncture to placate the masses after WWII when the government didn’t have the resources to provide conventional coverage in many areas of the country. That seems unlikely the case today, as China’s a world economic power. Just speculation on my part, but maybe China is trying to build some kind of TCM export thing to the West as a means of gaining profit, another chip in trade wars, etc. etc.

Whatever, it sure seems the Chinese government has a very big investment in promoting the legitimacy of acupuncture, not just locally but globally, including or maybe especially the good ol’ US of A. We are talking about the Journal of the AMERICAN Medical Association. If the pharmas can use their money in power to influence scientific research and publication, certainly the government of China has the juice to do the same. Now this kind of thing is probably subtle, and nothing as blatant as out-and-out bribery. But that’s how corruption often works. It creeps like kudzu, with a tiny little compromise here, a wee bit of go-along-to-get-along, in pursuit of some more “important’ goal, maybe even framed within “the greater good”.

If reviewers were just clueless or lax, that wouldn’t explain why they always seem to be clueless and lax in the same direction on the same issues. Maybe there isn’t any immediate carrot held out for the Journal. the editor(s) who chose the reviewers, or the reviewers themselves… Maybe it’s just some kind of ideology now shared in certain quarters. But if you examine how that kind of ‘common sense’ gets to be ‘common sense’, there’s almost always some significant movement of money and/or power working underneath to shift the sands.

This trial was supported by grants… from the National Natural Science Foundation of China and… from the State Key Program for Basic Research of China.

There’s nothing unusual about funding from these agencies, as opposed to what was being funded here.

The explanation is simple. It’s both ideological, a long standing view that TCM is a “national treasure” and a source of national pride, coupled with the financial. Acupuncture is a major part of TCM and TCM is a huge industry in China, with a major chunk of its pharmaceutical industry devoted to a profitable business of selling TCM nostrums, herbal remedies, etc. You can’t haves TCM without acupuncture; so China promotes acupuncture. It’s busily promoting TCM at home, having recently passed laws requiring TCM in state hospitals, and to the world through its promotion in the US and its having used its muscle to get The Who to integrate TCM diagnoses into ICD=11.

Here are some of my past posts on the topic: It’s also suppressing criticism of TCM companies rather harshly.

An honest researcher would have had that graph’s y-axis go all the way to zero, instead of 4. That’s visually dishonest and the editors shouldn’t have allowed it

Why not just also have an electrical stimulation group only as well for this study? Oh, wait, if just passing a current through skin alone had the same effect as jamming painful needles into people, that would negate acupuncture.

Based on the paper, this deqi thing these acupuncture quacks seek is painful:

Disposable stainless steel needles were used in acupuncture treatments. Insertion was followed by stimulation performed by lifting and thrusting the needle combined with twirling and rotating the needle sheath to produce the sensation known as deqi (sensation of soreness, numbness, distention, or radiating, which is considered to indicate effective needling).

First, I hope the disposable stainless steel needles were also sterile.

Second, I had a relative last year getting physical therapy on a sore knee. Apparently the physical therapist had already been doing cupping (ruh-roh, Shaggy sure sign of quackery) and I got called because something called “dry needling” was being suggested–which basically is this deqi crap but even deeper way into a muscle before you start twisting it around for the “no pain go gain” premise of so-called treatment (I spoke directly with the physical therapist and this is what they described). Not surprisingly I found no good studies supporting dry needling. I convinced this relative to go see a PT who was much more science-based in their approach, working off the biomechanics of muscle groups to determine a successful rehab plan.

“… acupuncture needles were inserted 2 mm lateral to each acupoint …”

Really!? Does an accupoint have a little red X to mark the spot so precisely that it is possible to resolve a 2 mm distance from the point? Think of all the savings in fiddly imaging that could be achieved if only real medical procedures involving sticking-in of assorted pointy things could be accomplished with such accuracy by merely eyeballing the spot.

I don’t understand that graph at all. To me the labeling of the X axis is just bizarre. What is the meaning of a discrete point for a time in weeks of 1 to 4? Then we have the Y axis as frequency as “No.” Number per what? And apparently good things come to he who waits – not major good things, but improvement nonetheless.

Why, why, why is this garbage ever published? Does editor mean anything anymore?

On a related note, I got a stent in 2000 (before ORBITA I assume). I had stable angina but when angiogram revealed a 98% blocked LDA, they did the stent. Presto! no more angina! I am glad I got the stent instead of medication–my lifestyle was ideal BTW, in fact it was the bike riding that induced the angina. I was told that I was a heart attack waiting to happen, but I wonder if that would be the case today?

Why did this get published, indeed. In the last few years I’ve seen too much of this kind of nonsense. Using somewhat standard methodology does not validate nor excuse what’s essentially junk. This is not science; it’s using the mechanisms of science to assert the viability of utter nonsense. As the adage goes, garbage in/garbage out. This is a reliable tactic of people doing junk science: use somewhat standard if subtly flawed methodology to fool peer reviewers, and you just might get away with it. That journal is now off my list of credible sources of information.

Orac is being unfair to Chinese science.

Many terrific papers out of China reveal the benefits of acupuncture, but the Western Pharma Stooges keep getting them retracted.

And then there’s the recent groundbreaking article showing how acupuncture on parents treats their children through quantum entanglement – also retracted!

Shameful, it is.

Get Upton Sinclair on the phone! He needs to update that “when his salary depends on it” business.

A commentor over on SBM also noted that of the 404 patients in this study, only 17 were dropped for non-compliance, in a study that involved 16 weeks of in-patient treatments and all those home logs. That’s more than a 95% compliance rate! I can’t think of anything that gets a 95% compliance rate, including things like insulin.
It’s a bit fishy to me.

The other thing is: are they actually recommending a 30 minute acupuncture/TENS treatment three times a week for stable angina? Imagine the cost and time involved, when you can simply apply a patch every day to prevent angina and take a tablet as required to treat the angina.

But, but, but … patches and tablets are products of Big Pharma and hence to be despised and reviled. Oxycontin comes in pills and fentanyl comes in patches and look at how terrible and without any legitimate use they are!!1!

If TENS done at home with simple surface electrodes had demonstrable positive effects, I think it could wide acceptance. Of course for some people, the attention of having someone do something for them a few times a week is probably very pleasing – better to go to a quack who tries to do something than a real physician who says there is nothing to be done or prescribes a medication that is safe and effective, but doesn’t come with hand holding and head patting. That seems to be at the very core of the success of so much of “alternative medicine.”

From the quack’s point of view, three visits a week is healthy part of a guaranteed income plan.

Orac writes,

Acupuncture is theatrical placebo, nothing more.

MJD says,

Recently filed a U.S. patent application for a anticancer skin-cream wherein acupuncture is described in the body of the application as a means to enhance medicament absorption. In this application, acupuncture is not a theatrical placebo.

Q. When is the phrase “nothing more” meaningful.

A. In the absence of “something more.”

I always thought that nitroglycerin is a proper angina medication. Why not compare acupuncture with it ?

Excellent question. I think the answer is, use of nitroglycerin (or any long acting nitrate like isosorbide mononitrate/dinitrate) would prove acupuncture completely ineffective, which is the opposite of what the authors wanted to prove.

Comments are closed.