…is one of my favorite surgeon-bloggers Sid Schwab when he discusses what happens when naturopaths actually subject their “healing art” to scientific examination. In the wake of the rather–shall we say–vigorous discussion that ensued after my post last week about the recently passed law in the State of Minnesota to license and regulate naturopaths, today I can’t resist taking a look at the widely publicized study of St. John’s Wort for Attention-Deficit Hyperactivity Disorder (ADHD) that by coincidence was published in JAMA last week and that Sid discusses.
What he’s discussing is a study done by Bastyr University, that stonghold of naturopathy “at the heart of natural medicine education” in the Pacific Northwest, in which St. John’s Wort was tested in children with ADHD. The results? What do you think? It was a negative study (direct link to study here). Basically, in a randomized, double-blind, placebo-controlled trial 54 children between the ages of 6 and 17 were randomized to St. John’s Wort or placebo and treated for eight weeks after a placebo run-in phase of one week, and participants who were less than 80% adherent as assessed by pill count or who had a dramatic placebo response during the run-in phase were excluded from the trial prior to randomization. At the end of the treatment period, there was no detectable difference between the groups in various measures of ADHD. Studies don’t get much more negative than this, and I do applaud the naturopaths at Bastyr for producing compelling evidence suggesting that one of their favorite herbal medicines, St. John’s Wort, just plain doesn’t work for ADHD.
Of course, the authors had to try to find a silver lining in this dark cloud of a study:
This trial was designed as a single-agent clinical trial, so the results pertain only to the use of H perforatum in isolation for the treatment of ADHD. It is possible that H perforatum may work synergistically with other botanicals, vitamins, minerals, or supplements. In addition, independent testing at the beginning of the trial confirmed that the product was standardized to 0.3% hypericin. Initially in the study of H perforatum, focus was on the constituent hypericin, a napthodianthrone, which was believed to work as a monoamine oxidase inhibitor, but it was not found to reach levels in the blood that would be physiologically active. More recently, the attention has turned to hyperforin, a phloroglucinol derivative, which is believed to be responsible for the reuptake inhibition of serotonin, dopamine, and norepinephrine. The product used for this study was not one of the newly marketed “high-hyperforin” products that range from 3% to 5% hyperforin. The product used in this trial was tested for hypericin and hyperforin content at the end of the trial and contained only 0.13% hypericin and 0.14% hyperforin. Hyperforin is a very unstable constituent that quickly oxidizes and then becomes inactive, which is likely what happened to the product used in this clinical trial. The majority of H perforatum products on the market are at risk of oxidation due to their delivery as 2-part capsules. It is possible that a product standardized to at least 3% hyperforin could benefit children with ADHD symptoms if it were delivered in a method that limits oxidation.
Of course, few indeed are the agents that perform synergistically with other agents without showing at least a modicum of activity on their own. It’s possible that St. John’s Wort may work this way, but unlikely given the results of this trial. It’s also not unreasonable to wonder if a higher dose might have worked. (After all, that’s what they’re basically saying with all that business about the hyperforin and hypericin.) Fair enough; that’s what is said after many “conventional” drugs fail to show efficacy in RCTs. Of course, no increase in dose is without a cost. Increase the dose, and you increase the chance of side effects and toxicity. You can’t escape that. Also, if either of these two constituents of St. John’s Wort are indeed active against ADHD, that begs the question: Why not just isolate and purify the postulated active compounds from the plant and test them under highly controlled dosages and conditions? What possible advantage is there to using an impure mix derived from a plant compared to the pure active compound, if either of these compounds are indeed active? That’s just what those “unimaginative” pharmacologists and natural products chemists have been doing for decades, if not centuries, and it works pretty darned well. Think digoxin, taxol, and a host of other plant-derived pharmaceuticals.
In fact, the only weaknesses of this study were the above and somewhat of a mismatch in the percentage of boys in each group. The trial wasn’t the largest, either, but it was certainly well-designed enough and large enough to have been powered to detect an effect equivalent to that of currently used drugs for ADHD, as described by the authors:
Sample size calculations were performed to determine the number of participants needed to detect effect sizes similar to those that have been reported in recent ADHD medication trials.
After all, isn’t that what naturopaths are claiming anyway, that herbal medications can do as well or better than pharmaceuticals for various conditions? That’s exactly what this trial was designed to test, and St. John’s Wort failed to demonstrate an effect at all, much less an effect equivalent to that of Ritalin or other drugs used for ADHD. If there is indeed an treatment effect against ADHD attributable to St. John’s Wort that this study was underpowered to detect, it is almost certainly much smaller than the effect of good, old-fashioned Ritalin. “Naturally” (sorry, couldn’t resist), there was an accompanying editorial by a doctor named Eugenia Chan. As is the case with many of these editorials about so-called “complementary and alternative medicine” (CAM), particular when apparently written by an apparnetly woo-friendly physician, there is a lot of hand-waving about why this trial was negative. There was also a bit of the tu quoque fallacy thrown in for good measure:
Most studies of CAM efficacy, both in adults and children, are generally of poor quality; this is true for many conventional medical practices as well. In mainstream medicine, the concepts of outcomes research, evidence-based medicine, and the need for rigorous methods and quality standards for reporting clinical trials have emerged just in the past few decades. Only very recently have these standards been extended to RCTs of nonpharmacologic treatments.
All true, but it’s also all pretty much irrelevant to the study at hand. Even if the rest of “conventional” medicine were truly not based on high quality evidence (an exaggeration that, aside from the infiltration of CAM into medical academia, bears less and less resemblance to reality these days), Dr. Chan is simply muddying the waters by shouting “you, too” back at scientific medicine, just because an “alternative” medical treatment has failed the test of scientific medicine in this trial. Unfortunately for her, two wrongs don’t make a right, and the point that I emphasize again and again is that–finally–medicine is moving away from what I like to call tradition- or dogma-based medicine to seriously trying to rigorously systematize science- and evidence-based medicine, examining scientifically treatments both new and old to discern which ones work (and in what situations) and which ones don’t. This is a very good thing, and long overdue. Sadly. although Dr. Chan plays lip service to the concept of EBM and the importance of RCTs, she still can’t help but bring out the usual attacks on RCTs as inadequate for addressing CAM therapies, complete with the obligatory appeal to other ways of knowing:
While the RCT should remain the gold standard for providing evidence of treatment efficacy for both CAM and conventional therapies, many interventions pose unique challenges to rigorous RCT methodology. For example, randomizing participants may be difficult or impossible when the therapy to be evaluated relies on participants’ belief in the treatment or relationship with the practitioner. Use of placebo and blinding may be difficult in therapies such as acupuncture, yoga, psychotherapy, or surgery, although techniques such as placebo needles that do not actually enter the skin have been developed. Even when a plausible placebo can be used, placebo and expectation effects can be very large in both CAM and conventional interventions and may be part of the mechanism of treatment efficacy.
Researchers may have difficulty defining precise and measurable outcomes for therapies for which the main effect is subjective (eg, general well-being, level of energy) or dependent on the skill of the practitioner (eg, therapeutic touch, psychotherapy). Some CAM therapies (eg, homeopathic remedies, traditional Chinese medicine) are tailored for an individual’s specific needs and may not be able to be studied at a conventional “active ingredient” or “dose” level. Moreover, individuals often use a variety of CAM modalities simultaneously (eg, meditation, aromatherapy, herbs) or adjunctively with conventional therapies, and secular trends in lifestyle values such as diet, exercise, yoga, and massage may attenuate or magnify treatment effects.
Need I remind Dr. Chan that “conventional” therapies are often used in combinations too and therefore must control for the same things? Or that there is an increasing move towards “personalized” medicine in “conventional” medicine as well? The difference between “personalized” medicine in CAM and science-based medicine, of course, is that choices of treatments in science-based medicine are to the extent possible based on measurable, objective standards, such as gene expression, genomic profiling, and metabolic measurements, among other criteria. The “individualization” of CAM treatments, on the other hand, tends to be based on fairy dust such as “life energy” or “auras” or other pseudoscientific measurements of questionable accuracy and relevance from dubious labs. Naturally, the CAM-friendly don’t see it that way, and neither, apparently, does Dr. Chan, who uses another favorite gambit of the CAM crowd:
Some of these challenges could be addressed through alternative study designs; for example, multiple N of 1 trials could be used to test the effectiveness of highly individualized CAM therapies, and preference RCTs can empirically test the effects of patient preferences on outcomes. Placebo and expectation effects can be addressed by excluding high placebo responders, as in the Hypericum trial, or by experimentally manipulating such effects as its own variable (eg, as graded dose-responses). “Attribute by treatment interaction” research methods can evaluate the extent to which interactions between individual patient and treatment characteristics, as opposed to the treatment’s overall effect across a group of patients, explain variability in outcomes.
At its heart, calling for “multiple N of 1 trials” is really nothing more more than a call to use poor quality anecdotal evidence instead of high quality evidence from RCTs to study CAM, a strategy that, because of the placebo effect, is guaranteed to produce “positive” results. (Of course, that is the very intent of such calls.) As usual, this call is cleverly buried under a barrage of verbiage truly deserving of being included in Dr. Kimball Atwood’s Weekly Waluation of the Weasel Words of Woo (although, admittedly, not quite as dramatic as this example). Dr. Chan then tops it all off with a spectacular example of the false equivalence fallacy:
Ultimately, increased attention to and emphasis on deepening a rigorous evidence base for all health care practices will benefit patients and families. The time for bad science, whether in conventional or unconventional medicine, is past.
Actually, I’d go further than that and say that there never was a time for bad science, and I’m very happy that medicine has been becoming more and more rigorously scientific during the two decades since I graduated from medical school. The vast majority of CAM modalities, which with the exception of herbal medicines, are at their heart bad science, because they are nearly all based on either pre-scientific notions of disease (vitalism being the most common one) or on religious or “spiritual” ideas (the concept of the life force qi, which is a combination of vitalism and religion). Some, like homeopathy, are based on concepts found in magic rather than science (for example, the Law of Similars). Yet CAM advocates keep telling us that we must “integrate” pseudoscience with science-based medicine to produce so-called “integrative medicine.” How on earth can that be good science? Worse, pseudoscience abounds in naturopathy, including homeopathy, electrodiagnostic testing, chelation therapy for all manner of illnesses besides acute heavy metal poisoning, galvanic therapy, “glandular” therapy (in reality this is probably nothing more than “live cell therapy,” which usually involves grinding up animal glands and administering them), and hydrotherapy, among many, many others.
Perhaps what’s more disturbing is the alliance with the University of Washington that was necessary for Bastyr University naturopaths to carry out this study, as Wally Sampson pointed out:
In reality, N’paths hold no fund of knowledge not available to MDs and other medical practitioners. At the same time, N’paths warp that knowledge’s application to fit ideological frames containing life forces, anti-technology, anti-public health measures and irrational fears of progress. Worse, N’paths believe a mass of unproved and false information that they claim as their exclusive area of expertise and which they apply on the basis of hunches and feelings. A N’path supplies no additional positive substance to healing the sick or to the science of medicine. In this sense, UW is an enabler of quackery, even if the short run product for the university could be algebraically positive. The overriding social consequence of enabling quackery, which is taught to Npathy students, is to spread a system of pseudo- and anti-science throughout the society.
One must grant the UW some degree of understanding for taking on this alliance, out of ignorance, innocence, and misplaced beneficence, but the establishing of precedent is an act they will one day regret and will find progressively more difficult to reverse. Bastyr’s existence and UW’s acceptance materialized in a social environment lacking an overarching theory of scientific social beneficence and a sense that something is wrong with anti-science.
Even though this study was completely negative, Bastyr gained far more from this collaboration than the University of Washington ever could. This sort of collaboration serves the purposes of Bastyr well in that it allows them to insinuate their woo into respected medical schools like the University of Washington and gives them an undeserved patina of respectability. After all, if the real medical doctors at such a prestigious institution are willing to collaborate with Bastyr naturopaths, there must be something to naturopathy, right?
Wrong. Aside from the potential that some herbal medicines might have as drugs, there ain’t. Naturopathy is nothing if not one-stop shopping for woo.
Amusingly enough, some in the “natural medicine” crowd are very angry with the naturopaths at Bastyr. In a hilariously over-the-top article appearing on the always hilariously over-the-top “natural medicine” website NaturalNews.com, conspiracy theorist supreme and chief cheerleader for woo Mike Adams sees in this study a dark and horrible plot by big pharma to discredit “natural medicine,” with the naturopaths at Bastyr having been the naïve dupes of this evil cabal:
Keep in mind that one of the study’s authors, Dr. Biederman, is not merely on the take from drug companies that sell competing pharmaceuticals, but that he also lied about how much money he was being paid by drug companies, hiding the truth about his income by underreporting $1.6 million he took from psychiatric drug companies. See my report on that here: http://www.naturalnews.com/023408.html
Dr. Biederman has a clear financial interest in promoting patented prescription drugs for brain chemistry disorders while discrediting competing natural alternatives such as St. John’s Wort. This blatant conflict of interest was not disclosed by JAMA, nor was it mentioned in the text of the study on ADHD and St. John’s Wort. It appears Dr. Biederman would prefer his financial ties to Big Pharma continue to remain secret, even while producing questionable studies that desperately attempt to show that herbs don’t work.
Mike Adams is lying here, plain and simple. Whatever Dr. Biederman may or may not have done with regards to reporting all of his income from drug companies, in the case of the Bastyr-UW study Dr. Biederman did disclose his conflicts of interest with regard to this particular article. At the very end of the article, where disclosures of conflicts of interest always appear in JAMA articles, I find this text:
Dr Biederman reports that he currently receives research support from the following sources: Alza, AstraZeneca, Bristol-Myers Squibb, Eli Lilly, Janssen Pharmaceuticals, McNeil, Merck, Organon, Otsuka, Shire, the National Institute of Mental Health, and the National Institute of Child Health and Human Development; that he is currently a consultant/advisory board member for the following pharmaceutical companies: Janssen, McNeil, Novartis, and Shire; that he is currently a speaker for the following speaker’s bureaus: Janssen, McNeil, Novartis, Shire, and UCB Pharma; and that in previous years, he received research support, consultation fees, or speaker’s fees for/from the following additional sources: Abbott, AstraZeneca, Celltech, Cephalon, Eli Lilly, Esai, Forest, Glaxo, Gliatech, the National Alliance for Research on Schizophrenia and Depression, the National Institute on Drug Abuse, New River, Novartis, Noven, Neurosearch, Pfizer, Pharmacia, the Prechter Foundation, the Stanley Foundation, and Wyeth. No other disclosures were reported.
That’s a pretty comprehensive disclosure of conflicts of interest, wouldn’t you say? I’ll say it again: Either Mike Adams lied and is a lying liar, or he didn’t bother to look at the article itself before composing his hysterical screed. Either way, it reflects very poorly on him. (So what else is new?) Not that that his own error or lie stops him from going on to paint Bastyr University as even more of a dupe than I’ve painted UW as being:
In the world of naturopathy, by the way, there is quite a chasm between the more “conventional” N.D.s (like Bastyr graduates) and the holistic, natural, salt-of-the-Earth kind of naturopathic healers who have no sponsoring institution. The Bastyrs of the world are working hard to get naturopathic medical practice legalized in many states, but they’re also disliked by the non-accredited naturopaths who end up being labeled criminals for practicing their own brand of natural medicine in those same states.
Many non-accredited naturopaths insist that Bastyr is just a “green” replacement for organized medicine’s tyranny. Without a doubt, when people see Bastyr graduates collaborating with top psychiatric drug pushers on a study that clearly seeks to discredit a valuable herb, it just fans the flames of dissent against Bastyr among more holistic practitioners.
What’s my take on the issue? I think Wendy Weber must be a complete fool to lend her name to such a study, because the very title of the study presupposes something that’s entirely false to begin with: That ADHD is a bonafide “disease” in the first place. She even based the entire scoring of the participants’ symptoms on the American Psychiatric Association’s DSM-IV — the tome of psychobabble “disorders” invented by a truly evil industry that seeks to label every person still breathing with some sort of brain chemistry disorder (and then demand that they all be “treated” with mind-altering drugs that just happen to enrich their corporate sponsors, the drug companies!).
Of course, Adams’ other charge, namely that the study used an “inactive” form of St. John’s Wort because big pharma (represented by Dr. Biederman, wanted this study to fail, strains credulity. Does he honestly believe that the naturopaths at Bastyr didn’t want to design this study to give it the maximal chance of showing a treatment effect due to St. John’s Wort? Or maybe he believes the naturopaths at Bastyr are utterly clueless about what is supposed to be one of their favorite and most well-known herbal medicines. Whatever I think of naturopathy, even I don’t think that naturopaths at the largest and most famous school of naturopathy in the U.S. would be so ignorant about their own supposed craft as not to be able to come up with what they consider to be an appropriate formulation of St. John’s Wort for this particular trial.
Aside from herbal and plant-based medicines, which in reality is nothing more than administering and studying unpurified and unstandardized medicines, there is little in naturopathy that could be said to be based on science or on anything other than the placebo effect. (Naturopaths embrace the ultimate in pseudoscience, homeopathy, for crying out loud!) People don’t realize that most medical doctors are not scientists and that post-modernist relativism (or even just a desire not to be perceived as close-minded) can lead physicians into either not resisting the infiltration of pseudoscience into academia or even embracing it. Collaborations such as the one between Bastyr University and UW are one result of this, as well as a carefully crafted strategy by CAM advocates to coopt science-based medicine to give pseudoscientific CAM modalities the appearance of scientific validity. Unfortunately, all these collaborations serve to accomplish is to blur the line further between scientific medicine and woo, between medicine and quackery. The naturopaths definitely have their foot in the door, but will scientific medicine have the courage to slam the door shut on it in spite of the yowls of pain doing so will no doubt cause?
REFERENCE:
1. Weber, W., Stepp, A., McCarty, R., Weisse, N., Biederman, J., McClellan, J. (2008). Hypericum perforatum (St John\’s Wort) for Attention-Deficit/Hyperactivity Disorder in Children and Adolescents: A Randomized Controlled Trial. Journal of the American Medical Association, 299(22), 2633-2641.
