I hate to do this to Bora again. I really do. I’m also getting tired of blogging all these crappy acupuncture studies. I really am. However, sometimes a skeptic’s gotta do what a skeptic’s gotta do, and this is one of those times.
As you may recall, a mere week ago I was disturbed to have discovered the publication of a truly horrifically bad acupuncture study in PLoS ONE. It had all the hallmarks of quackademic medicine: an implausible hypothesis, trying to correlate mystical concepts of meridians and qi to anatomy and failing miserably, and dubious statistical modeling. That PLoS ONE actually published this tripe shows me that, for all its claims of being scientifically rigorous, PLoS ONE has a serious problem when it comes to so-called “complementary and alternative medicine” (sCAM or, if you’re a fan, CAM). In fact, PLoS ONE has as its tagline “accelerating the publication of peer-reviewed science.” Sadly, the acupuncture study published there a month or two ago made me think that going a bit slower wouldn’t necessarily be such a bad thing.
So does this latest atrocity, which comes from the University of New South Wales, and the Prince of Wales Hospital and is entitled The Brain Effects of Laser Acupuncture in Healthy Individuals: An fMRI Investigation. I’ll give Im Quah-Smith and crew some credit. Unlike a recent acupuncture study from a month ago, where the authors explicitly tried to hide that they were not actually doing acupuncture at all but rather were running electrical current thorough the needles, at least Smith et al admit up front that they are using “laser acupuncture.” Of course, “laser acupuncture” is no more acupuncture than is “electroacpuncture”–unless, of course, the Chinese were more techologically advanced than anyone had previously suspected, having developed electricity and having understood quantum mechanics a couple of millennia before we backward “Western” scientists did. Either that, or the first incarnation of Deepak Chopra was alive and active back then.
Yes, it’s just that ridiculous. Before I even discuss the study, I just want to reiterate and reemphasize that “laser” acupuncture is not acupuncture. It is lasers zapping the skin. True, “laser acupuncture” is a much cooler term than “lasers zapping the skin,” but that does not make it any more accurate or correct. What is it with woo-meisters and their propensity to take modern technology, graft it onto treatment based on prescientific notions of disease and how the body works, and then “rebrand” it (to put it kindly) into something that sounds cool but goes woo. That’s all the “electroacupuncture” study was done, and it’s how this “laser acupuncture” study was done.
Smith et al begin:
Despite the remarkable developments in Western Medicine in modern times, public interest in Traditional, Complementary and Alternative Medicine (TCAM), such as acupuncture, remains high , . This may be because TCAM is perceived as holistic and relatively free of adverse effects. However, these treatments sit uncomfortably alongside scientific medicine because of strikingly different explanatory systems and the empirical tests applied by each discipline. In order to bridge the gulf between high public acceptability and the lack of empirical evidence for many of these treatments, it is important to reconcile them with modern scientific concepts. Our focus here is on laser acupuncture, and we address the question whether laser acupuncture produces brain effects that are biologically plausible.
Here we go again with the classic argumentum ad populum. Notice that the argument is not that acupuncture works and is scientifically interesting for what it can teach us about the body. Notice that the argument is also not that studies of CAM should be scientifically rigorous because science-based medicine is the best medicine and, if acupuncture can’t stand up to rigorous science, we shouldn’t use it. Rather, the reason to apply science to acupuncture, according to Smith et al, is to “bride the gulf between high public acceptability and the lack of empirical evidence for many of these treatments. Also notice that Smith et al have set the bar for evidence incredibly low. Heck, they’ve set it so low that I wonder why they bothered at all. Basically, they are asking whether laser acupuncture causes measurable effects in the brain. Seriously. That’s all this study tests. One might as well state the hypothesis as: Shining lasers on the skin will cause changes in the brain. It’s such an obvious hypothesis that it’s virtually trivial, as this passage slows:
In this study, we examined the blood oxygen level dependant (BOLD) functional magnetic resonance imaging (fMRI) response to laser simulation on the above mentioned acupoints CV14, LR14, LR8 and HT7. We chose laser acupuncture as it permits blinding of application because of the lack of a local sensation at low intensity, together with the previously mentioned increases in practical usage and limited understanding of its mechanisms. We reasoned that if laser acupuncture is altering a person’s mental state a neurological effect should be observable. Further, if the effect differs dependent on the site of stimulation, then the neural locus of the activity should also differ.
Well, duh. Why wouldn’t the effects in the brain differ dependent on the site of stimulation? This is Neuroscience 101, people. In fact, this is so simple that even a surgeon like me can understand, no need to call in a neuroscientist or a neurologist like Steve Novella. In fact, to me this would be the sort of laboratory exercise that Neuroscience 101 students would do if it weren’t for the fact that functional MRI is too complex and expensive. The hypothesis, however, is neither complex nor challenging. So what did Smith et al do?
Basically, they took 10 students (7 men and 3 women) who had no history of depression or psychiatric disorder and had never undergone acupuncture treatments. They then called upon the magical mystical arts of acupuncture to choose the following acupuncture points:
Why were these acupuncture points chosen? Who knows? The mysticism that is traditional Chinese medicine appears to claim that these acupuncture points have relevance for depression, as well as on a small randomized trial of “laser acupuncture” for depression. One problem that lept out at me right away was that this study was only single-blinded. The subjects were blinded to whether they were receiving “real” or “sham” laser acupuncture, the sham laser acupuncture consisting of the laser being turned off. Four acupuncture points were tested against one sham acupuncture point as follows, alternating between “real” laser acupuncture and placebo laser acupuncture (no laser) at each acupuncture point. fMRI images were acquired and duly analyzed. Lots of MRI time was wasted along with subjects time, and one of the simplest and least revealing tables of results I’ve ever seen. Particularly suspicous to me is that the results were presented in a single table of results and numeric values. No actual images were presented, which would allow readers to get a much better feel for how significant and convincing the reported results are.
Another deficiency in this study is the lack of something that is absolutely critical for any study that involves any form of imaging, be it fMRI or even just simple chest X-rays. What is that something? Blinding. That’s what. For any imaging study, it’s very important that the radiologists evaluating the images be blinded to the experimenatal group. Even though the images were quantified using computer software, there is still a huge subjective component in analyzing and interpreting fMRI images and identifying the various anatomic areas being examined. Add this to the apparent lack of blinding of the actual acupuncturists doing the “laser acupuncture” and the investigators doing the computer prestidigitation on the fMRI images.
After all that effort, here’s what the investigators concluded, after bragging that their study was the “first fMRI study to examine the effects of laser stimulation of a suite of acupoints found to be efficacious in a clinical condition (depression)”:
The main finding of our study was that each acupoint or control point resulted in a different pattern of brain activity when contrasted against all the other acupoints or control point. The acupoints we investigated in this study were those that have been used in our previous treatment study for depression . This finding suggests that although these acupoints are all used in the treatment of depression, the neural locus of this effect differs depending upon the site stimulated.
Well, duh, I repeat. Well, duh. Shine laser light on different parts of the body, and you get different reactions in the brain. Who’da thunk it?
Not surprisingly, as nearly all quackademic medical articles do, this study takes utterly uninteresting results and then uses the differences observed in what parts of the brain light up depending upon what acupuncture point has laser light shined on it as evidence that acupuncture–excuse me, laser acupuncture–“works” for depression. Indeed, they even make it explicit. Because shining a laser light on the four acupuncture points cause the fMRI of the brain to light up in areas that are also involved in depression, that must mean that acupunture using these points is a valid treatment for depression. Personally, I see this as confusing correlation with causation combined with the Texas sharpshooter fallacy.
Finally, it’s interesting to note who the investigators are. A quick Google search on the first author, Dr Im Quah-Smith, revealed this gem, which declares that “Dr Im Quah-Smith fervently believes acupuncture can relieve depression. And now she’s putting that faith to the test.” I also found this video:
Now there’s some serious quackademic medicine for a serious journal like PLoS ONE.
Having read this study, I found it to be a steaming, stinking pile of fetid dingo kidneys, even by the usual low, low, low standards of typical CAM studies. My guess for what happened is this: The authors knew this study didn’t meet the standards for a real neuroscience journal. Neither did they want to relegate it to the scientific ghetto of the CAM literature, where editorial standards are so low that they’re subterranean and attempts at actual science are intermingled with the purest woo, like homeopathy and reiki. So what to do? Well, PLoS ONE bills itself as a journal where only the science counts, where the reviewers don’t make judgments on the import of the science being presented but rather only assure that it is sound, boasting a 60-70% acceptance rate for manuscripts. I can see how PLoS ONE might be a tempting place for a quactitioner to drop off her latest attempt at quackademic medicine. Unfortunately, somehow these latest bits of tooth fairy science on acupuncture belie the scientific rigor that PLoS ONE claims for itself. Seeing these articles in PLoS ONE make me even more certain than I was before that I no longer wish to submit any manuscripts to PLoS ONE.
After all, if I’m going to risk having my work placed along side a quackademic article, I should at least aim high–higher than PLoS ONE. The New England Journal of Medicine, perhaps?
Quah-Smith, I., Sachdev, P., Wen, W., Chen, X., & Williams, M. (2010). The Brain Effects of Laser Acupuncture in Healthy Individuals: An fMRI Investigation PLoS ONE, 5 (9) DOI: 10.1371/journal.pone.0012619