The justification for NCCAM: “What can be done to generate a better placebo?”

It’s probably an understatement to say that I’ve been critical of the National Center for Complementary and Alternative Medicine (NCCAM). Indeed, I consider it not only to be a boondoggle that wastes the taxpayers’ money funding pseudoscience, but a key promoter of quackery. Worse, its promotion of highly implausible (one might even say magical) modalities gives these modalities a patina of scientific respectability that they do not deserve, especially given that, even under the most favorable conditions possible, they routinely fail to demonstrate any efficacy above and beyond that of a placebo control. Indeed, as I’ve emphasized time and time again, often such therapies appear to show efficacy in smaller pilot studies, which are not as rigorously controlled, but fail to show any efficacy above that of a placebo in larger, better-designed randomized trials. Indeed, acupuncture is the prototypical example, as is homeopathy.

An article from Monday in the Washington Post leads me to believe that the mainstream media may finally be catching on. It isn’t perfect, but it’s better than any straight reporting I’ve seen on the issue in a long time:

The impending national discussion about broadening access to health care, improving medical practice and saving money is giving a group of scientists an opening to make a once-unthinkable proposal: Shut down the National Center for Complementary and Alternative Medicine at the National Institutes of Health.

The notion that the world’s best-known medical research agency sponsors studies of homeopathy, acupuncture, therapeutic touch and herbal medicine has always rankled many scientists. That the idea for its creation 17 years ago came from a U.S. senator newly converted to alternative medicine’s promise didn’t help.

Although NCCAM has a comparatively minuscule budget and although it is a “center” rather than an “institute,” making it officially second-class in the NIH pantheon, the principle is what mattered. But as NIH’s budget has flattened in recent years, better use for NCCAM’s money has also become an issue.

It’s rare to find an article in a mainstream news source begin with such a blunt statement of what we critics have been saying about NCCAM for years. Dr. Wally Sampson, for instance, is as far as I know the first who openly called for the defunding of NCCAM, and he did this back in 2002. These calls have recently gained new force given how CAM advocates are currently trying very hard to tie the legitimization of CAM to whatever health care reform legislation President Obama tries to get through the Congress, camouflaging it under the cloak of legitimate treatment modalities such as “diet and exercise,” “health maintenance,” and “prevention.”

The reporter of this story (David Brown) in a pleasant surprise, appears to have understood most of the issues involved. He quoted Steven Salzberg, a genome researcher and computational biologist at the University of Maryland, who started the infamous (in a good way) “defund NCCAM” initiative before President Obama ever took office using the Obama Transition Team’s Change.gov website. He quoted Steve Novella:

Critics of alternative medicine say the vast majority of studies of homeopathy, acupuncture, therapeutic touch and other treatments based on unconventional understandings of physiology and disease have shown little or no effect. Further, they argue that the field’s more-plausible interventions — such as diet, relaxation, yoga and botanical remedies — can be studied just as well in other parts of NIH, where they would need to compete head-to-head with conventional research projects.

The critics say that alternative medicine (also known as “complementary” and “integrative” medicine, and disparagingly labeled “woo” by opponents) doesn’t need or deserve its own home at NIH.

Which is the point that I’ve been making all along. The NIH funded studies of diet and exercise as means of maintaining health and preventing disease perfectly well long before NCCAM was a misbegotten dream in Tom Harkin’s fevered little mind. It could do so again. Herbal medicines, another of the more plausible treatments from the grab bag of unrelated and mostly implausible remedies that make up CAM (or, as it’s being called more recently, “integrative medicine”) are a similar case. The study of natural products is nothing more than the time-honored subdiscipline of pharmacology known as pharmacognosy. Indeed, it was through the study of natural products that a large percentage of our currently used drugs were discovered (aspirin, taxol, and numerous others), and there is no reason to think that such studies could not be funded just as well by other Institutes and Centers within the NIH. After all, that’s how it was before the misbegotten Center of Woo known as NCCAM came into being. Indeed, pharmacognosy would arguably be strengthened because pharmacognosy would no longer be tainted by its association with all the woo that goes along with CAM. Truly, NCCAM is a victim of its own history, and one of the major complaints I have against it is that, by lumping together the plausible (diet, lifestyle, exercise, herbal medicines) with the implausible (energy healing, homeopathy), from a scientific standpoint it taints the plausible modalities with the brush of woo.

Unfortunately, all is not sweetness and light, as Mr. Brown succumbed to two instances of the journalistic “balance” of the kind that drives me crazy. As much as I’d like to give him a pass on a little bit of the old Respectful Insolence, I can’t. You wouldn’t respect me anymore if I didn’t. Here’s number one:

At the same time, it’s difficult to determine the clinical implications of some of the positive studies.

For example, reiki — but not sham treatment — blunted the rise in heart rate, but not the rise in blood pressure, in rats put under stress by loud noise. Therapeutic touch, a different modality, increased the growth of normal bone cells in culture dishes, but decreased the growth of bone cancer cells.

I’ve dealt with the infamous “reiki for rats” and “therapeutic touch for bone cancer” studies before. Suffice it to say that neither study represented “positive” studies of energy healing.

Then, true to the usual tenets of journalism, Mr. Brown couldn’t resist bringing in the naturopath:

Although the overall effect of therapies such as homeopathy and acupuncture may be small, individual response can be large. The route to the placebo effect — if that’s what it mostly is — also varies in method and efficiency.

“What can be done to generate a better placebo? Why isn’t that an interesting and valid area of investigation?” said Calabrese, who was on NCCAM’s advisory council from 2004 to 2007. “Here we have a totally harmless intervention that seems to get a better result in some people than others. Why wouldn’t you want to study that?”

This is an astounding quote, so astounding that the fact that Mr. Brown got Calabrese to admit this very basic fact about CAM greatly blunts my disappointment that he was forced by journalistic convention to quote “the other side.” Consider: Calabrese just flat out admits that CAM therapy is placebo therapy! Of course the placebo effect is a very interesting physiological phenomenon. It’s very complex, and it’s true that it is variable, but it’s not as variable as Calabrese makes it sound. More importantly, though, there is a serious ethical dilemma in trying to maximize placebo therapy effects due to CAM.

It requires in essence lying to the patient through representing ineffective therapies to the patient as something they are not, namely effective.

Which brings me to something that disappointed me greatly, namely fellow ScienceBlogger Janet Stemwedel’s take on the whole NCCAM issue. Unfortunately, her discussion of NCCAM, sparked by a post by blog bud PalMD, reveals her unfamiliarity with the issues involved. Then she had to compound the problem by posting a followup listing various studies funded by NCCAM. I don’t want to be too hard on her, but unfortunately she did lay down at least one real howler.

After making a point that I entirely agree with, namely that the ethics of human studies are paramount in not just human studies in general but in human studies in CAM, unfortunately Janet went off the rails a bit. That she relied primarily on what is on the NCCAM website to make her point shows that she lacks a deeper understanding of background history of NCCAM, a history that one will not find anywhere on the NCCAM website; it is always perilous to rely solely on a website that is there to put NCCAM’s best foot forward. One wonders if Janet would rely on the website of, say, Merck or Pfizer as her sole source of information regarding the influence of the pharmaceutical industry. On the one hand, it’s hard to blame Janet too much for that, given that she has apparently only recently examined the issue, but her relying pretty much solely on the propaganda on the NCCAM website (and what is on government websites but propaganda designed to make the government agency for which the website was designed look as good as possible?) reveals a failure of skepticism on this issue, which is why I generously suggest that she search “NCCAM” on my blog and on Science-Based Medicine for a taste of what she’s missing, including the $30 million boondoggle of an unethical trial known as the TACT trial of chelation therapy. In essence, Janet either misunderstands or mischaracterizes the concept of prior plausibility in wondering why it’s not a good idea to study many of the magical therapies that fall under the CAM rubric:

Going forward with a clinical trial that we have good reason to think would be harmful to the human subjects crosses the line ethically. I think there’s also a feeling that plausibility ought to play a role in the evaluation of whether a clinical trial is ethical. Plausibility is tricky – it’s a question about whether a particular kind of intervention could possibly be effective given the current state of our understanding (of human biology, of this sort of intervention, etc.). Clearly, the current state of our understanding is incomplete and subject to revision; this is why scientists are still involved in conducting research.

Does “X could plausibly treat condition A” mean that there must be some clear mechanism by which X might act to cure or improve condition A? There have been compounds whose efficacy was widely accepted before we had anything like a detailed understanding of the mechanisms by which they worked (think aspirin). Does “X could plausibly treat condition A” mean that there already exists a body of empirical data demonstrating its efficacy? If this were the standard, no compound that wasn’t already being used by a significant number of people could ever make it to clinical trials.

That is not what critics of NCCAM mean when they criticize studies of implausible remedies based on a lack prior plausibility. Really, it’s not. It’s a straw man argument to say that what critics of CAM are arguing is that we have to have a detailed understanding of the mechanisms by which a drug works before we have sufficient prior plausibility to justify a trial. The example of aspirin is, in fact, not a good one to argue this point. The reason, of course, is that, although scientists may not have necessarily known exactly how aspirin works, whatever its mechanism may be was (and is) entirely consistent with our scientific understanding that aspirin is a molecule that must be able to interact with proteins in cells in some way in order to produce the effect that it does. Scientists may not have known the details, but those could be worked out and there was no reason to suspect that they involved head-on collisions with what we understand of science. Such is not true with many CAM modalities, such as “energy healing techniques,” that postulate the existence of not just a “life energy” (qi) that no scientist has ever been able to detect and that, based on the characteristics attributed to it would require the overthrow of many fundamental principles of our current understanding of physics, but also of the ability of human “healers” to manipulate that undetectable energy for healing effect. Another problem is that scientists definitely knew that aspirin had an effect over and above that of a placebo–strongly so. Consequently, even if they didn’t know how it worked, they did know that it did work. No such statement can credibly be made of most CAM therapies, whose effects, even when reported, struggle to rise above random noise (another reason why for most the evidence is most consistent with placebo effect). So not only does the vast majority of CAM other than diet/exercise and herbal remedies consist of highly implausible treatments, but these treatments have no compelling evidence for their efficacy. Finally, as John Ioannidis showed, combining low prior probability with weak effects is a recipe for numerous false positive trials.

Janet also makes an observation that she doesn’t quite attach the correct conclusion to:

It’s worth noting, though, that at least some of the “alternative” treatments that turn up in NCCAM studies involve stuff a good number of people do everyday that they might not consider a medical treatment – drinking cranberry juice or herb tea, eating tofu, getting a massage, meditating, doing yoga. At least from my vantage point in the land of veggie restaurants and yoga studios, I don’t imagine that these “alternative treatments” are feared to be harmful in themselves. Rather, they are probably viewed as “risky” to the extent that they may replace medical interventions that are well-grounded in empirical evidence – or to the extent that funding to study them in clinical trials might waste funds that would be better directed to studying interventions judged more likely to be effective.

Yes! That’s exactly what CAM promoters want you to think! They want you to link CAM with all sorts of activities that are not inherently “alternative, such as diet and exercise. As I’ve said many times before, such activities are the Trojan horse, within which CAM advocates hide all the real woo, like homeopathy, reiki, and even shamanistic healing practices. Worse, in the case of NCCAM, studies of highly improbable therapies, such as chelation therapy for cardiovascular disease and the Gonzalez protocol for pancreatic cancer, end up with no publications providing guidance for physicians or patients. Indeed, in the case of the Gonzalez protocol, arguably negative results have been suppressed.

Unfortunately, Janet compounds her mistake in her second post, where she in essence lists a bunch of trials from NCCAM and then concludes:

To my eye, some of these trials have studied pretty improbable treatments (like distant healing). However, if these treatments are the kind of things actual patients are seeking out and using, I have to believe that they don’t seem improbable to everyone.

It hurt me to read that last sentence. My simple retort is that ghosts, alien abduction, and various other forms of paranormal phenomena seem very plausible to someone. Bigfoot seems plausible to someone, as does the Loch Ness Monster. That some credulous person finds an idea plausible is not a reason to assign the idea sufficient plausibility to spend millions of dollars investigating it! Quite frankly, I don’t give a rodent’s hindquarters if some woo or other is plausible to “someone.” I care only whether it’s scientifically plausible, and, for instance, when it comes to distant healing under current scientific understanding there is no plausibility. I concede that there will be gray areas in this argument, in which reasonable people can disagree with whether an idea is scientifically plausible or not, but, by and large, for “energy healing” modalities, homeopathy, and most other CAM modalities other than diet/exercise/herbal remedies, it ain’t even close.

After that, Janet gets more reasonable:

This raises an ethical conundrum. If people who think treatment X is plausible are faced with results from a carefully designed and conducted clinical trial, will they revise their view of whether X is a plausible treatment? Would such a revision be a big enough benefit to justify conducting what seems at the outset to be a pretty implausible clinical trial? And, in the absence of such concrete scientific findings, what options do you have to convince the X-seeker about the implausibility of X as a treatment?

What Janet does not understand is that this issue goes deeper than that. It is, quite literally, a clash between two opposite world views over the very rules that will determine the scientific method. Science-based medicine emphasizes repeatable, observable phenomena and testing new therapies yielded by scientific investigation on patients in carefully controlled, blinded, randomized trials. There may be weaknesses in this approach, the main one of which is applying population-level data to individuals (which can be sometimes tricky indeed), but these shortcomings pale in comparison to the difficulties posed by the methodology CAM advocates, namely anecdote-based studies, where personal experience trumps science.

Unfortunately, the two world views are completely incompatible, and the reason NCCAM is failing is because of this incompatibility. I’ll give Janet an example that is relevant. There are numerous–and I do mean numerous–studies that have failed to find even a whiff of a correlation between vaccination and autism. Yet Jenny McCarthy is still out there pushing the myth. Generation Rescue is still out there raising money for “biomedical research” (i.e., quackery) to treat “vaccine injury.” The DAN! doctors are still out there selling quackery in the form of chelation therapy, supplements, diets, and many other dubious therapies. Thousands upon thousands of parents are still terrified of vaccinating, and Age of Autism continues to defend Andrew Wakefield, author of the MMR scare in the U.K. a decade ago, against well-documented charges of undisclosed conflicts of interest and falsifying data. Nothing has changed.

Just like CAM.

There are numerous studies that have failed to find an effect for homeopathy, acupuncture, reiki, and numerous other modalities greater than that of placebo. Yet homeopathy thrives, and reiki is invading my old hospital. That’s because data don’t matter to CAM believers. Heck, there are even still people out there going to Mexico for laetrile, even though randomized clinical trials in the 1980s showed that it doesn’t work.

Data don’t matter to true believers.

That’s not to say that we should never study this stuff. It does mean that in the case of the truly implausible therapies, such as homeopathy, distant healing, or various other “energy healing” modalities, we can safely dismiss them on the basis of current scientific knowledge alone, as accepting them would require a rewriting of our physics and chemistry textbooks. In the absence of truly compelling evidence that makes us doubt our current understanding of physics and chemistry at a truly fundamental level, there is no more reason to consider them than there is to consider whether fairies exist. Show me a fairy, and I’ll reconsider my opinion on their existence. Until, then, I can safely conclude that they do not exist. In the case of modalities like homeopathy or acupuncture, combining prior knowledge alone plus the clinical trials that have been done thus far would provide sufficient evidence to reject them as having any efficacy above that of a placebo, although the clinical trials that have already been done are merely icing on the cake of an argument from lack of prior plausibility.

In fact, according to the Helsinki Declaration, it is unethical to study such wildly implausible therapies, which fail to meet this requirement for human subjects research:

Medical research involving human subjects must conform to generally accepted scientific principles, be based on a thorough knowledge of the scientific literature, other relevant sources of information, and adequate laboratory and, as appropriate, animal experimentation. The welfare of animals used for research must be respected.

(Emphasis mine.)

Testing modalities that go against very well-established tenets of science–indeed, very basic, well-founded scientific theories–without extraordinary evidence supporting such modalities, evidence sufficient to call the scientific theories that preclude these modalities into doubt, is unethical. Moreover, many–if not most–of these modalities have not been adequately tested in preclinical models to show the possibility of efficacy, the “reiki for rats” and “therapeutic touch for bone cancer” studies notwithstanding. For example, in a proposed clinical trial of chelation therapy for autism, the study in essence proposed to skip any cell culture and animal research altogether and go straight for a human trial, which is profoundly unethical.

Whatever good NCCAM might have been capable of, it’s long gone. Woo-friendly legislators like Tom Harkin are displeased with it because it doesn’t deliver up the positive results supporting CAM that they want. It’s embedded itself into the very fabric of medical education by funding “educational” and “training” fellowships in CAM, truly a case of putting the heart before the horse. Moreover, there is nothing that NCCAM does that couldn’t be done as well or better either in or funded by the other existing Centers and Institutes of the NIH, and the things that it does do better are things that shouldn’t be done at all on an ethical basis. After all, it’s not ethical to promote a therapy before it’s been proven safe and effective, but that’s exactly what NCCAM does through its funding of CAM fellowship training programs.

It’s time for the abomination that is NCCAM to be buried once and for all.