Categories
Clinical trials Complementary and alternative medicine Integrative medicine Medicine Pseudoscience Quackery

Surprise! Surprise! (Well, actually, no.) Acupuncture does not work for in vitro fertilization

Australian researchers have carried out another randomized clinical trial on acupuncture for in vitro fertilization. Unsurprisingly, it’s completely negative. Also unsurprisingly, acupuncturists are not happy and are furiously making excuses.

I didn’t think I’d be writing about acupuncture again so soon after my discussion of a howlingly bad acupuncture meta-analysis, but here we are. Of course, as I note from time to time, I do tened to write about acupuncture a lot because it is the most “respectable” form of quackery. After all, Harvard Medical School even teaches a large online course in acupuncture worth over 300 continuing medical education credits, a course I first noted nearly a decade ago and wrote about again last year. It doesn’t matter that rigorous science shows that acupuncture is no more than a theatrical placebo. It seems to be the “gateway woo” that leads many physicians into the “integration” of quackery with medicine in the specialty known as “integrative medicine.” (Another gateway woo is functional medicine.) Once that woo is accepted, all manner of pseudoscience, quackery, and outright nonsense seem to follow into academic medical centers, up to and including the finding of new “organs” to “explain” how acupuncture “works.” Other times, it leads the National Center for Complementary and Integrative Health (NCCIH) to try to use it as a potential answer to the opioid addiction crisis. it leads to studies trying to demonstrate that acupuncture improves pregnancy rates after in vitro fertilization (IVF).

Yes, earlier this week, JAMA published the results of a randomized clinical trial of acupuncture on IVF success rates. Amazingly, this was, unlike most acupuncture trials, large and well designed. Not surprisingly, it was negative. Before I discuss the study itself, let me just discuss the biological plausibility of acupuncture as an intervention to improve the success rates of IVF. Basically, there is none, as far as science has been able to tell. Certainly, science has not been able to detect the “life energy” or “life force” that supposedly travels down the various acupuncture meridians and whose flow sticking thin needles into the skin is supposed to “unblock” to healing effect. Indeed, no anatomic correlate has ever been convincingly demonstrated for acupuncture meridians.

But, say acupuncture advocates, you don’t have to accept the vitalistic prescientific view in traditional Chinese medicine as to how acupuncture works. It works through adenosine! It stimulates nerves! it does other things! We can mix it up, too. After all, it involves sticking needles into the body; so surely it isn’t so implausible that it does…something. Maybe. But what? After all, remember that acupuncture enthusiasts claim that it’s good for almost anything that ails you, for conditions including back pain, insomnia, weight loss, menopausal hot flashes, battlefield injuries, Parkinson’s disease, headache, gastroesophageal reflux, and, well, IVF and many other conditions with unrelated physiology.

Indeed, this is not the first time that I’ve written about acupuncture for IVF. For instance, ten years ago, a dubious meta-analysis didn’t convince me. Neither did another meta-analysis from 2013. This brings us to reinventing the crappy wheel, 2018 edition, in the form of this study. It’s an Australian study, from the NICM Health Research Institute, Flinders University, University of New South Wale, University of South Australia, Greenslopes Private Hospital, and Western Sydney University.

It’s also a pretty big study with reasonably good design. It’s biggest flaw is that it was single blind such that study participants, fertility specialists and nurses, and the analysts were blinded to group allocation. However, the clinical trial coordinator and acupuncturists were not blinded to group allocation. Still, it might not have been so critical in a study like this, because pregnancy after IVF is a “hard” outcome. IVF either results in pregnancy and a live birth, or it doesn’t. As for the rest of the study design, it’s a randomized, sham acupuncture placebo-controlled trial. Subjects were recruited from 16 IVF centers in Australia and New Zealand. Inclusion criteria included women aged 18 to 42 years undergoing IVF or intracytoplasmic sperm injection (ICSI) (IVF in which a sperm is injected into the egg). Exclusion criteria included women undergoing a frozen embryo transfer, previous randomization to the study, planning preimplantation genetic diagnosis, or receiving donor eggs. One change was made to the protocol and was approved by all ethics committees. Initially, only women with recurrent IVF failure and unsuccessful clinical pregnancies from 4 IVF centers were included. However, these women turned out to be high users of acupuncture. So, in June 2013, following a review by the data and safety monitoring committee, a decision was made to revise the inclusion criteria to increase eligibility to the trial by removing the requirement that women must have previous failed IVF cycles and poor treatment outcomes, and to increase the number of recruitment sites.

The protocol, it turns out, was developed by the Delphi method. (Lovely.) Women were recruited at the time of their decision to undergo IVF or ICSI, and randomization occurred before the the beginning of the follicle stimulation protocol. IVF or ICSI treatment protocol was determined by each subject’s treating clinician. After randomization, women underwent either acupuncture or sham acupuncture treatment beginning between days 6 and 8 of ovarian stimulation and two treatments were given on the day of embryo transfer. In this particular study, non-penetrating sham needles were placed in non-acupuncture points.

Reading the article and the protocol, I was amazed at how much handwaving was necessary to produce a facsimile of biological plausibility. From Supplement 1:

Mechanisms underlying acupuncture have been widely researched in relation to pain. Acupuncture analgesia research suggests acupuncture triggers a sequence of events involving the release of endogenous opioid like substances, for example β-endorphin, and enkephalin that modulate pain signals. Further, imaging technology studies show the hypothalamus-limbic system plays a role with acupuncture analgesia by modulating sensory and emotional aspects of pain processing. Mechanisms underlying other health conditions may involve modulation of the autonomic nervous system, as seen in patients with irritable bowel syndrome and major depression. Electroacupuncture has been shown to release ß-endorphin a neurotransmitter and neuro hormone via two systems. One system involves the hypothalamus and neuronal network which results in an inhibitory effect on the vasomotor centre resulting in decreased sympathetic tone. Secondly, ß-endorphin released into the blood may exert an effect on both the hypothalamicpituitary axis and hypothalamic-pituitary-gonadal axis and may have an effect on gonadotropinreleasing hormone (GnRH) and pituitary gonadotrophin secretion. Acupuncture may exert a sympatho-inhibitory effect reducing uterine artery impedance and increase uterine and ovarian blood flow. Improved blood flow in women undergoing the down regulation phase of IVF from acupuncture has been reported26. This improved blood flow could hypothetically improve the growth and thickness of the endometrium and endometrial receptivity. To date these findings have not been confirmed in controlled studies.

This is, as I said, major handwaving. It’s what I like to refer to as “woo babble,” which is like Star Trek technobabble, only with woo and science-y sounding sentences that sound impressive to lay people but are clearly nothing more than wild speculation.

But lets get to the results. Among 848 women randomized, 24 withdrew consent, leaving 824 women. Of these 809 had data available on live birth outcomes, 405 receiving acupuncture and 404 receiving sham control acupuncture, whose live birth outcome rates were 18.3% and 17.8%, respectively. The difference was not statistically significant. A clinical pregnancy was achieved in 25.7% of the acupuncture group and 21.7% in the sham control group, which was also not a statistically significant difference.

In other words, acupuncture doesn’t work to improve IVF.

Not that the authors didn’t try really hard to salvage some result from their data. They did some post hoc analyses. They did some exploratory analyses. They didn’t find much of anything. Here’s where I ask you to guess what the authors said in their conclusion to excuse their lack of any sort of positive effect due to acupuncture. Yes, they admit that they found no difference between the acupuncture and sham groups and admitted that their results support a recent guideline from the American Society for Reproductive Medicine and 2 high-quality meta-analyses that found that acupuncture compared with a sham control performed around the time of ovarian stimulation and embryo transfer does not improve live birth rates in IVF.

But:

The study addressed whether acupuncture was more efficacious than a placebo and specifically examined whether any effect was mediated through the point-specific needle insertion. To do this, a noninsertive needle was used. A recent meta-analysis examining placebo devices as effective control methods in acupuncture clinical trials found that these devices are not necessarily inert control interventions. These findings therefore may be explained by potential activity from the sham control. To determine whether any treatment effect in clinical outcomes arises from point-specific needle insertion or from other nonspecific effects, such as the placebo or time and attention from a therapist, would require the inclusion of a no-adjuvant treatment group in the trial design. A usual care–alone group was not included in the RCT due to findings from a previous pilot study that women would decline randomization to this design. Therefore, it remains unclear whether there are nonspecific effects from acupuncture or a sham control.

Yep. They’re saying that the sham control had an effect, and that could be why there was no difference between the acupuncture and sham acupuncture groups. Of course, if they had included a regular treatment group and there had been no difference between the two acupuncture groups but even a slight improvement in one of the two groups over the regular treatment group, the authors would be saying that acupuncture works. Heads, I win, tales you lose. Of course, as I’ve said many times before, it doesn’t matter where you stick the needles. It doesn’t even matter if the needles are inserted. There is no specific effect attributable to acupuncture. It’s a theatrical placebo.

At least the authors stopped there with that bit of speculation, which was actually fairly restrained for believers in acupuncture whose large multicenter acupuncture study had been negative. What surprised me, however, is that I found myself a little bit disappointed by the commentary on Medscape by F. Perry Wilson MD, MSCE on its Methods Man blog. He goes beyond even the authors’ speculation. First, here he is on biological plausibility:

As always, we should start with biologic plausibility. Is it biologically plausible that acupuncture should increase the live birth rate in women undergoing IVF? Well, it depends what you mean by acupuncture. If, by acupuncture, you mean the insertion of needles in very specific locations to alter the flow of an immeasurable energy force within the body, then no, there is no biologic plausibility there.

OK, so far, there’s nothing objectionable here—or even anything that I wouldn’t have written myself. However, notice the framing (“it depends what you mean by acupuncture”). Now read what comes later:

But acupuncture may not be totally without physiologic effect. Perhaps the local stimulation of nerve endings can release endorphins or promote blood flow (though these claims are actually a bit controversial). But of course, if that’s the case, sham acupuncture (where the needles are placed randomly) should work just as well as “real” acupuncture for IVF.

And indeed, that is what was found.

Yes, it’s the same speculative excuse invoked by the authors. He builds on this:

The use of a rigorous sham control here demonstrates what we know deep down – there is nothing magical about acupuncture.

This is not to say it is without value. As a clinical researcher, when I look at an acupuncture study the first thing I think of is “co-intervention bias.” Acupuncture is so much more than the sticking of needles into specific locations. It’s a quiet room, soft music, a compassionate therapist, and human touch.

These are all things that might be really beneficial.

Maybe. Or maybe not. They probably wouldn’t hurt, at least. Look, I agree that there’s nothing magic about acupuncture. I also used to think there might be something to it. After close to 15 years of delving into the nitty gritty of acupuncture studies, I’ve become convinced that acupuncture really is nothing more than a theatrical placebo.

Here’s the problem. You can’t say that about acupuncture based on this study. You just can’t. Why? Because there’s no regular care control group to compare the acupuncture and sham acupuncture to! Only if there had been such a group and it had had lower live birth rates than the acupuncture groups would such speculation be (barely) justifiable on a scientific basis, given the extreme lack of biological plausibility of acupuncture as a treatment to increase IVF success rates. What we have here is a resoundingly negative study.

Maybe I shouldn’t be too hard on Dr. Wilson, though. Amusingly, even his mild commentary provoked pushback from an acupuncturist:

That’s because no one can make excuses for negative studies like acupuncturists. Check out the comments after Dr. Wilson’s article. There are several to choose from. Some point to other, less rigorous studies. Others claim that acupuncture done outside of the TCM diagnosis of infertility is pointless. Never mind that TCM diagnosis methods were based on prescientific mystical beliefs. Two question the skill of the acupuncturists used. For instance, here’s Geoffrey Levins:

My comment is in line with Mr. Levens, above. Who did the “real” acupuncture and how were they trained? Was each patient subject evaluated as an individual? Or was everyone given the identical treatment protocol? Additionally it is written clearly in the Chinese medicine literature that acupuncture can create a strong potential for miscarriage, if used incorrectly. If it was doing nothing much, as proposed by the author here, then why the increase in miscarriages with the “real” vs the “sham” or perhaps the “sham” had a preventive effect on miscarriage? Yes, many many unanswered questions.

And Geoffrey Levins:

“Rigorous sham acupuncture”. I don’t have access to article but how rigorous was the “real” acupuncture? After practicing as a professional for 25+ years and having hundreds at least treatments myself, I know that there is a very wide variety of skill levels between various practitioners. Also, there several, very different from each other, approaches to acupuncture for example Traditional Chinese vs Japanese vs Korean, etc. Lots of unanswered questions so far as I am concerned.

How much money does the Tooth Fairy leave per tooth? How soon after the tooth is put under the pillow does the Tooth Fairy come? Is there a lifetime limit to what the Tooth Fairy will give? Yes, Levins is asking Tooth Fairy research questions, because this study was Tooth Fairy science. As for the skill of acupuncturists, how does that matter when there is no scientific basis to acupuncture and it doesn’t matter where you stick the needles?

What really bothers me is how unethical this study was. There was no plausible scientific basis for it. Existing evidence for IVF and acupuncture was, by and large, negative. Worse, this was a decent-sized study that likely cost millions of dollars to carry out, money that could have been used to study real medical questions whose answers might actually lead to improvement in patient care and improved outcomes.

By Orac

Orac is the nom de blog of a humble surgeon/scientist who has an ego just big enough to delude himself that someone, somewhere might actually give a rodent's posterior about his copious verbal meanderings, but just barely small enough to admit to himself that few probably will. That surgeon is otherwise known as David Gorski.

That this particular surgeon has chosen his nom de blog based on a rather cranky and arrogant computer shaped like a clear box of blinking lights that he originally encountered when he became a fan of a 35 year old British SF television show whose special effects were renowned for their BBC/Doctor Who-style low budget look, but whose stories nonetheless resulted in some of the best, most innovative science fiction ever televised, should tell you nearly all that you need to know about Orac. (That, and the length of the preceding sentence.)

DISCLAIMER:: The various written meanderings here are the opinions of Orac and Orac alone, written on his own time. They should never be construed as representing the opinions of any other person or entity, especially Orac's cancer center, department of surgery, medical school, or university. Also note that Orac is nonpartisan; he is more than willing to criticize the statements of anyone, regardless of of political leanings, if that anyone advocates pseudoscience or quackery. Finally, medical commentary is not to be construed in any way as medical advice.

To contact Orac: [email protected]

29 replies on “Surprise! Surprise! (Well, actually, no.) Acupuncture does not work for in vitro fertilization”

Well this is the absolute minimum, they also received donations from acupuncture fertility clinics (which they did not declare – https://frankvanderkooy.com/2016/10/28/the-nicm-and-their-undeclared-conflict-of-interest-an-example-of-scientific-misconduct/ ) and I guess more tax payer money from the University hosting them. In total, yes, it will be well over a million.

But doesn’t the 18% live birth rate seem a bit low? According to IVF Australia’s website women between the ages 30-34.9 have a live birth rate of just above 35%, and for women between the ages 35-39.9 just above 25%. In this study the median age was 35.5 so it does indeed seem to be on the low side, although I am no IVF expert.

https://www.ivf.com.au/ivf-success-rates

If acupuncture requires neither accuracy nor puncture, what do we have left? Is that like therapeutic touch which doesn’t require touching and isn’t therapeutic?

Next up: We will test the efficacy of me waving my arms at them from 1000 miles away! (I have a friend who unfortunately believes that Long Distance Reiki works.)

I have to admit I tried acupuncture on about the 6th and 7th unsuccessful IVF cycles. The cycles were still unsuccessful but I did find it very relaxing (fell asleep actually). Didn’t use it on the successful 12th cycle though. Massage would probably have had the same effect.

I am already using this study to confound accupuncturists. From a surface plausibility point of view, this one peeps slightly above the horizon with a bit of handwaving included. There is this idea that having women more relaxed might increase IVF success (To be honest I haven’t checked the evidence base on that, but lots of people mention it) and accupuncture could help relaxation. So I point out that even when everything is stacked in favour of accupuncture, it doesn’t do anything.

The claim that the sham might have been an active treatment is in fact a cop out. Even if true, it couldn’t save accupuncture.

Oh fun days.

The problem with their methodology is obvious. You don’t stick needles into the woman, you stick them into the EMBRYO! Then the magic of needle-power will ensure a successful impregnation, just like putting a dagger under the bed wards off evil spirits that can harm the fetus.

On a different blog someone made the comment that in the ANZCTR trial registry the control group is described as ” Study group 2: The sham acupuncture group will use the Park Sham device which has a guide tube making it impossible for the subject to see whether the needle has penetrated the skin or not. The sham needle its self is able to slide into the handle, which increases the appearance that the skin has been penetrated. Needle penetration does not occur. The sham needle will be placed away from acupuncture points. The sham acupuncture device is an effective masking device for blinding in randomised controlled trials of acupuncture. Treatment will be administered on day 6-8 and twice on the day of an embryo trasnfer. Study Group 3: Standard care. Standard care involves use of the usual pharmacological and surgical protocols used during the IVF cycle. This is a non randomised arm and women will contribute data only. The trial co-ordinator will make contact with subjects at the assessment time.” https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=336609

The results of study group 3: IVF alone is not even being mentioned. Any ideas why?

“Initially, only women with recurrent IVF failure and unsuccessful clinical pregnancies from 4 IVF centers were included. However, these women turned out to be high users of acupuncture.”

Ergo acupuncture prevents pregnancy and is a useful method of contraception.

Acupuncture does whatever you want, it helps with back pain, insomnia, weight loss, menopausal hot flashes, battlefield injuries, Parkinson’s disease, headache, gastroesophageal reflux, IVF, it can cut a tin can without dulling, repair this boat we cut in half, remove blood and grape juice stains from this white carpet, and it really, really works.

Since I last took stats >25 years ago, I’m curious. Why is a 4% difference considered not statistically relevant in this case?

clinical pregnancy was achieved in 25.7% of the acupuncture group and 21.7% in the sham

I know that acupuncture is a crock of shit, but 4% seems to be a fair bit – even if solely placebo. Sorry for what may be a stupid question, but I learned long ago, the only stupid question is the one not asked.

Statistically relevant? What does that even mean?

The difference didn’t achieve statistical significant. I didn’t go into how it was calculated. You can be sure that if there were a way to calculate it and get it below 0.05, the authors would’ve used it.

Sorry I misspoke. I did mean significance and not relevance.

I guess my question should have been why is a 4% difference (~16 pregnancies difference out of ~800 total) not considered significant? Is there a higher level needed in medicine for it to be considered significant?

In Poisson statistics, the standard deviation is equal to the square root of the number of events. To get to p < 0.05, you need two standard deviations. The square root of 800 is about 28.3, so you would need 57 events to reach the significance threshold, which is about 7%.

Since there are two groups being compared here, the actual computation is not quite like this, but you find the same thing: a 4% difference for the numbers involved here is not significant, and not even close.

Thank you Eric. That clears it up for me.

I’ve spent my career in a world where 4% one way means abject failure, another way success. The way you’ve described it makes it more clear for me.

True. That is a small sample. Would it remain statistically insignificant if that was increased by a factor of 10? So 40 flips, 30 tails, and 10 heads or even 25 tails and 15 heads.

But isn’t the more relevant statistic in this study the rate of live births? 18.3% vs 17.8% isn’t statistically significant and doesn’t feel clinically significant (a 0.5% difference in 809 women is less than one birth).

<

blockquote>After practicing as a professional for 25+ years and having hundreds at least treatments myself, I know that there is a very wide variety of skill levels between various practitioners. Also, there several, very different from each other, approaches to acupuncture for example Traditional Chinese vs Japanese vs Korean, etc. Lots of unanswered questions so far as I am concerned.

Damn it! I hate not being able to edit replies. It should be:

After practicing as a professional for 25+ years and having hundreds at least treatments myself, I know that there is a very wide variety of skill levels between various practitioners. Also, there several, very different from each other, approaches to acupuncture for example Traditional Chinese vs Japanese vs Korean, etc. Lots of unanswered questions so far as I am concerned.

Levin has totally devastated all critics with his No True Acupuncture ™ argument. All must hang their head in shame and leave the internet.

So these authors are seriously arguing that sham acupuncture is not a placebo? There is no other way for them to draw the conclusion they draw here. The point of the sham acupuncture protocol is that it is a method which even an otherwise reasonable acupuncture fanatic would agree that it is a placebo, and the study demonstrated that real acupuncture produced results not significantly different from that placebo.

It takes a great deal to convince people acupuncture/ woo doesn’t work if they have had ( what they think are) “good results”…

Some one I know ( NOT me) believed that acupuncture helped him to recover more quickly from a sports injury years ago which was repeated twice more: he is generally not woo-entranced but he even paid for me to try it ( in a very atmospheric environment complete with recorded flute music and incense presided over by an elderly Chinese woman straight out of central casting) the results of which I related here at RI, i,e, Not much.

At any rate, he went back last week because of another injury that has persistently bothered him for several months AND…
he said it didn’t work. He even said this to other people he knows.

Sceptics CAN get through to others if they persist as I did.

-btw- I WISH it did work: it would make life easier and rehabilitation cheaper

If acupuncture cannot be defined in a way which allows for a form of sham acupuncture with which to make an unambiguous and meaningful comparison, then acupuncture is beyond the boundaries of empirical science.

Acupuncture studies are generally a combination of magic – we have no scientific theory of how it works but loads of “theories” of how it might work, and we know it does work – and sham studies in which sham acupuncture maybe isn’t sham acupuncture – except, of course, when acupuncture outperforms sham acupuncture, when we really know that sham acupuncture really is sham acupuncture.

So an undisputed form of sham acupuncture is any form which fails to match an undisputed form of acupuncture, and there are gazzillions of them, in a trial. Otherwise the sham is maybe one of the gazzillions of forms of acupuncture.

Can’t fail, won’t fail.

“I don’t have access to article but how rigorous was the “real” acupuncture? After practicing as a professional for 25+ years and having hundreds at least treatments myself, I know that there is a very wide variety of skill levels between various practitioners.”

The classic battle cry of the disappointed wooist.

It can’t be that the premise is laughable and the evidence convincingly negative. The reason it didn’t work is the same reason other alt med practices turn out to be worthless: “They did it wrong!”*

*or They sabotaged the trial.

Comments are closed.