Naturopaths argue that they are science-based. Hilarity ensues.

Naturopathy is a frequent topic discussed on this blog. The reasons are myriad but can be boiled down to a handful of very easy to understand rationales. First, naturopathy is based on a vitalistic worship of all that is allegedly “natural”. The “specialty” of naturopathy originated in the 19th century German “natural living” movement. As Kimball Atwood noted, early naturopaths objected to contemporary medical advances, such as germ theory of infectious disease and vaccinations, espousing instead the “water cure,” fasting, herbs, homeopathy, colonic “detoxification,” and other popular “healing” methods of the era. Of course, since the 18th century, “natural” seems to have evolved to encompass a much broader meaning among naturopaths, given that, in addition to their original 18th century treatments, naturopaths now embrace intravenous nutrients and high dose vitamin C, as well as all manner of modern modalities ranging from the unproven and not recommended (thermography to detect breast cancer; hyperbaric oxygen for everything other than chronic wounds; and stem cell treatments for, well, everything) to pure quackery (chelation therapy for autism, cardiovascular disease, and cancer; IV “micronutrient therapy,” and all manner of strange machines, like a bioresonance machine or the Quantum QXCI/SCIO).

In any event, the very worldview of naturopathy is rooted in vitalism, the idea that there is a “life force” that living creatures and plants have that nonliving matter does not. Also, not only does naturopathy embrace The One Quackery To Rule Them All, homeopathy, it devotes considerable hours to homeopathy in the curriculum of naturopathy school and includes it on the NPLEX, the naturopathic licensure examination. So tight is the embrace of homeopathy by many naturopaths that one was recently investigated for using a homeopathic remedy made from the saliva of a rabid dog to treat a child. This background is why, when I saw the latest issue of The Journal of Alternative and Complementary Medicine, I just had to discuss it. I couldn’t resist. Why? Because it’s a Special Focus Issue on Naturopathy and includes a whole lot of attempts by naturopaths to demonstrate that their specialty is science-based.

I learned about this issue from ABC News in Australia:

Southern Cross University researchers say a new ‘world-first’ study proves the benefits of naturopathy, but critics are questioning the institution’s credibility.

The scoping study, published in the latest issue of the Journal of Alternative and Complementary Medicine, was conducted by Southern Cross University researchers who say they have demonstrated naturopathic medicine works in the treatment of a wide range of chronic conditions.

Professor Stephen Myers, from the Southern Cross University, said the study was motivated by a review chaired by the former Commonwealth Chief Medical Officer that found there was no clear evidence demonstrating the efficacy of many natural therapies.

The review resulted in a decision to exclude naturopathy, homeopathy and several other natural therapies from private health insurance from April 1 this year.

“The government is proposing at the moment to take naturopathic medicine off health insurance rebates as of April 1, on an erroneous belief that there’s no evidence supporting it,” Professor Myers said.

Our very own Scott Gavura wrote about this decision when it was announced after a review of a number of alternative medicine therapies. Homeopathy and naturopathy weren’t the only modalities for which Australia withdrew coverage. The others included:

• Alexander technique.
• Aromatherapy.
• Bowen therapy.
• Buteyko.
• Feldenkrais.
• Herbalism/western herbalism.
• Iridology.
• Kinesiology.
• Massage or manual therapies.
• Naturopathy.
• Pilates.
• Reflexology.
• Rolfing.
• Shiatsu.
• Tai chi.
• Yoga.

Yes, I rather expected that the naturopaths would strike back. I’m just surprised that it took them so long. Not surprisingly, they couldn’t get these studies published in a reputable medical journal; so instead they published them in the pseudomedical journal JACM, which is edited by John Weeks who is neither a scientist nor a physician, but rather a longtime cheerleader for alternative medicine, who also recently devoted an issue of JACM to the quackery that is most of “integrative oncology.” Amusingly, Weeks is a man prone to likening critics of integrative medicine to Donald Trump and saying we have “blood on our hands” because we harshly criticized the University of California, Irvine for accepting $200 million from a homeopathy-loving billionaire couple to establish multiple initiatives to integrate quackery into medicine.

Let’s see how he and others justify this Special Focus issue of JACM.

Naturopathy is “science-based”? I don’t think so.

John Weeks seems particularly proud of his Special Focus Issue on Naturopathy, as is evidence right away in the first paragraph of his editorial, “The Early Movement for Research Evidence in Modern Naturopathy: Fighting Above Its Weight Class“. You can tell from the title that he thinks far more of naturopathic “research” than it deserves. Indeed, he’s very proud that for the first time “an entire issue of a peer-reviewed journal that does not have “naturopathy” or “naturopathic medicine” in its title is devoted to collecting and disseminating exemplars of the profession’s research output”. My first reaction was: So what? It’s still a quack journal! After that, I must admit that I laughed uproariously when I saw Weeks refer to “science-based natural medicine and naturopathy” and to “revealing the science-based commons for natural health care”. I can’t resist retorting, first, that we already have science-based natural medicine. It’s a branch of pharmacology known as pharmacognosy, or the study of medicines derived from natural sources, such as plants, fungi, etc. Secondly, regarding “science-based naturopathy”, I can’t help but say: “Science. You keep using that word. I do not think it means what you think it means.” I also can’t help but be amused at how quacks are so desperate to adopt the term “science-based.” Let’s just put it this way. If your “profession” embraces homeopathy, it can never be described as science-based. Ever.

None of this stops Weeks from exulting:

I spent a decade working closely with members of her profession on the revitalization of what in North America is called naturopathic medicine. After the awakening of the integrative era that began in the mid-1990s, I had witnessed how the naturopathic profession had—as RAND researcher Ian Coulter, PhD, once put it—“fought above its weight class” in its various roles in advancing the integrative movement.¹ Although still a small profession of not more than 8,000 practitioners in North America, members of the profession have provided leadership in integrative oncology,² functional medicine,³ modeling integrative education,⁴ formation of organizations that promoted interprofessional collaboration in politics,⁵ and education⁶ for the integrative model. Whether they knew it or not, their standard setting exemplified a cohesive training curriculum and philosophy that would influence biomedically based forms of integrative medicine that would follow. The eclectic profession’s embrace of multiple healing traditions and of biomedicine may explain the outsized influence.

Weeks writes this as though it were a good thing. Unfortunately, it is true. A few academic medical centers now have naturopathic quacks on their medical faculty (my medical alma mater even hosted a Family Practice Grand Rounds featuring two famous homeopaths thanks to the naturopath in charge of “integrative medicine” education in the Department of Family Medicine), and, as I’ve noted, the Society for Integrative Oncology not only admits naturopaths, but has had two naturopaths as its president over the last several years, as well as naturopaths as prominent co-authors of its guidelines for the “integrative oncology” treatment of breast cancer. Weeks also crows about how the National Center for Complementary and Integrative Health (NCCIH) has funded a number of studies of naturopathy, concluding, in part:

That these have reached the critical quality threshold to be included in significant policy recommendations and clinical management guidelines begs questions related to naturopathic doctors’ parallel path. Will this field’s efforts to show its value, through research, boost its cultural valuation and usage in primary and chronic disease care, and in shifting practice paradigms toward addressing whole systems and creating health as an outcome? The articles included in this special issue point to a growing sophistication in research activities that give a nod toward an affirmative response.

No. No they don’t, as you’ll see. Covering all of them in this post will be too much, but I do plan on hitting the “high points” (if you can call them that) and, if nothing catches my attention before next week that interests me more (I’m rather like Dug the Dog, you know), cover the rest of the issue.

Meanwhile, the editors of this Special Focus Issue claim:

The importance of this special focus issue must be clearly situated within this broader landscape. As highlighted in the commentary by the president and secretary-general of the World Naturopathic Federation (Lloyd and Wardle), naturopaths are producing a steady stream of research particularly for a profession that is in relatively early stages of the professionalization trajectory compared with other professions (osteopathy and chiropractic). However, the World Naturopathic Federation leadership also refers to a project to gather together the research produced by naturopaths as it is dispersed throughout the scholarly literature with no central “home” and often does not explicitly mention naturopathy despite the research question and resulting design being conceptualized from a naturopathic viewpoint. This diversity in the therapeutic armament of naturopathic doctors is a strength in clinical practice and also represents a significant challenge to understanding the current evidence base for naturopathy, that is, is research on individual nonpharmacologic therapeutics “naturopathic” and, therefore, included in the evidence base, or does the intervention need to be “whole practice” or delivered by naturopaths to be considered “naturopathic?”

Does it really matter when the vast majority of naturopathy is quackery, at least other than diet and lifestyle modifications appropriated from conventional science-based medicine, which naturopaths still somehow manage to quackify by, for instance, trying to treat disease with intravenous nutrients, something that is rarely, if ever, needed except for when a patient can’t be fed through using the gut and has to be fed intravenously? Yet, if you peruse naturopathic clinics, you’ll see that nearly all of them offer “intravenous micronutrients” to treat people who, as Mitchell and Webb would put it, “a vague sense of unease or a touch of the nerves or just more money than sense”. (Yes, I know their comedy sketch was about homeopathy, but all naturopaths are trained in homeopathy and the characterization applies to pretty much everything naturopaths do anyway).

The “state of the evidence” for naturopathy?

First, let’s take a look at the paper by Stephen Myers of the Southern Cross University. (I’m betting that Southern Cross is proud! Oh, wait. Southern Cross is a naturopathy school. Of course it’s proud, even though it shouldn’t be.) It’s a paper by Myers and Vanessa Vigar entitled “The State of the Evidence for Whole-System, Multi-Modality Naturopathic Medicine: A Systematic Scoping Review“.

My first reaction was: Huh? What, exactly are Myers and Vigar trying to do here? I had to review what a scoping review is. First, it answers “broader questions beyond those related to the effectiveness of treatments or interventions”, because of course a naturopath would choose this methodology. He’s bound to come up with a positive signal somewhere just because of random chance alone. Or, to put it another way, it “is still as comprehensive as a systematic review but much broader” and “may involve multiple structured searches rather than a single structured search.” and thus “will produce more results than a systematic review”. Moreover, critical appraisal is optional as is a Risk of Bias assessment. In fairness, in this review, the authors did do a Risk of Bias assessment on the randomized clinical trials included in the study. More on that later.

Since this is an open access article, you can read everything that I can read, and the figures and table are actually rather informative in an unintentional way. First, though, let’s look at the criteria for clinical studies to be included in this scoped review:

1. Controlled clinical trials, longitudinal cohort studies, observational trials, or case series involving five or more cases presented in any language
2. Human studies
3. Multi-modality treatment administered by a naturopath (naturopathic clinician, naturopathic physician) as an intervention
4. Non-English language studies in which an English title and abstract provided sufficient information to determine effectiveness
5. Case series in which five or more individual cases were pooled and authors provided a summative discussion of the cases in the context of naturopathic medicine

If you look at Figure 2, which is the flow sheet for the selection of studies for inclusion, you’ll see that the screening process identified 2,563 studies that might be included, but apparently only screened 208 of them. I’ve reread the article a couple of times and the methods and results sections more than that, and I don’t see a clear rationale for how the authors whittled down 2,563 studies to 208 to screen and then to only 33 to include. To get to 208, did they take a random sample? A non-random sample?

In the results they explain:

A total of 2551 titles were located and the titles and abstracts screened for eligibility. Full text of 95 primary studies and 12 gray literature studies were reviewed in detail (totalling 107). Thirty-three articles were accepted for inclusion, totalling 9859 study participants. The primary reasons for exclusion were lack of multi-modality treatment (i.e., only a single modality was practiced) or the studies did not identify that treatments were conducted by naturopathic clinicians. Notably, studies were excluded because they prescribed naturopathic-style treatments, but were administered by integrative doctors or other personnel not identified as naturopathic.

Now, in fairness, meta-analyses and systematic reviews often winnow down the results of the initial literature search rather brutally based on their inclusion criteria, but this seems even more severe than normal. It’s also confusing that in their chart they say they screened 208 studies but in their results they only reviewed in detail 107 studies. Also in fairness, it is not only naturopaths who are sometimes irritatingly opaque about how they choose their studies to include in meta-analyses and systematic reviews, but this is more annoying than what I usually encounter. In any event, of the 33 studies, 20 were observational cohort studies (12 prospective and 8 retrospective), and the rest included 11 clinical trials and 2 case series. Studies came from the United States (US), Canada, Germany, India, Australia, Japan, and the United Kingdom (UK). The studies included short-term inpatient care (primarily in studies from India and Germany) to longer term observational reports of outpatient treatment (primarily in the US and Canada), carried out in private naturopathic practice, integrative hospitals, inpatient health care clinics, or research institutes.

If you peruse the tables you’ll see that the vast majority of these studies were positive for a benefit from naturopathic “whole system” care. Sounds great for naturopathy, right? Well…not quite. As I said, it is to their credit that the authors did do a Cochrane Risk of Bias assessment of the 9 RCTs in the review. It is not to their credit that they glossed over the results of that Risk of Bias assessment. First, let’s be nice and point out what they got partially right:

In observational research, selection bias is considered high for several studies in which retrospective data have been reported and where outcomes include only patients who have returned for multiple visits. Reporting bias also is likely to be high in retrospective studies in which isolated outcomes are reported. Several of the prospective cohort studies are well-conducted, with a low level of bias, particularly those, such as Braun et al.,⁴³ Teut et al.,³¹ and Bradley et al.,²² that compared results with a “usual care” cohort.

Yes, it is correct that the risk of selection bias and reporting bias are high in retrospective trials. However, prospective cohort studies, although at less risk for bias than a retrospective trial, are not randomized. Moreover, when they compare to “usual care”, they are virtually guaranteed to be positive, thanks to placebo effects, observer bias, the Hawthorne effect, and the like. Without a good sham control, it’s hard to make much of these cohort studies, retrospective or prospective. Nor are the randomized controlled clinical trials without serious problems in this review. Let’s look first at what the authors said about it:

There is a wide range of quality in the included studies, given the breadth of the research reported. Cochrane risk of bias assessments were completed on the nine RCTs (Fig. 3), showing low risk of bias for all areas, except blinding of participants and personnel, and moderate bias for allocation concealment.

The authors are very nonchalant about this huge problem, only mentioning it briefly, but this is a big deal, particularly if you look at the chart in Figure 3:

Note that none of the studies were double blinded, producing high risk of bias from that alone. Almost as bad, six of the nine studies were at high risk of bias from allocation concealment (concealing from each patient and the investigator which to which group that patient was allocated, or assigned), and five out of nine were at high risk from blinding of the outcome assessment. Yet the risk due to bias for poor allocation concealment was described in the text as “moderate”, and the risk of bias from failure to adequately blind outcome assessment wasn’t even mentioned.

Basically, what this all means is very simple, and it’s the same thing I always say about meta-analyses and systematic reviews: GIGO (garbage in, garbage out). Look at it this way, this is the best naturopaths could come up with to support their “whole system” naturopathic treatment after screening >2,500 papers. No wonder the Australian Government Department of Health concluded this about the quality of evidence for naturopathy:

The overall quality of evidence was rated as very low according to our GRADE assessment. This was primarily because of limitations with indirectness (only North American studies) and imprecision (small sample sizes and wide confidence intervals). Risk of bias was also a factor since blinding of participants and personnel is not feasible in naturopathic practice trials. Reviewers also considered publication bias to be a potential problem since only a few studies, all favourable to naturopathic practice, were identified.

The Australian report was first published over three years ago. Only five of the studies included in this systematic scoping review were published more recently than 2015. To be fair, I note that one study from 2015 included in Myer and Vigar’s review that might not have been reviewed by the Australian Government Department of Health, given how long these things take, but it was just a case series of seven patients with bipolar disorder. As for the studies from 2016 and later, three were prospective cohort trials. Finally, the last one was a “matched case control trial”, whatever that means. Actually, I know what it means. It means that there was no randomization. That leaves only one study that was a randomized controlled trial. It’s a trial that studied lifestyle intervention versus lifestyle intervention plus 3 months of herbal treatment in obese women with polycystic ovarian syndrome. Of course, this study suffered from no blinding. I also don’t see how it fit the criteria. It really only tested one modality, an herbal medicine regimen added to lifestyle interventions. How is this a study of multimodality naturopathic therapy given that it really tested only one intervention, a “standardized herbal treatment protocol”?

Color me supremely unimpressed, both with this study and all the studies included in this scoping systematic review.

Apparently the authors were more impressed:

Although results from these studies are highly diverse, they also are predominantly positive, showing improved health outcomes and QOL across conditions and across nationalities. These studies demonstrate a broad range of naturopathic modalities, against a background of different practitioner training, legislative and regulatory jurisdictions, and different research approaches. Their results concur with Oberg et al.⁹ who determined the effect sizes of the primary medical outcomes for 13 North American studies, concluding that there were positive outcomes and improved QOL in individuals with, or at risk for, chronic conditions, including CVD,23,29 type 2 diabetes,^20–22 chronic pain,^17,18,26,27 anxiety,28 hepatitis C,²⁴ and menopausal symptoms.²⁵ The authors have updated and expanded this review to the global literature, thereby increasing the range of positive outcomes to include depression and anxiety,¹⁵ bipolar disorder,¹⁶ asthma,^39,40 PCOS,⁴⁴ and increased cancer survival time.⁴⁶ It also adds additional studies to support positive outcomes for CVD,^13,37,38,43 type 2 diabetes,³⁶ chronic pain,^14,32 and anxiety and mood disorders.⁴²

The problem is that the studies used to support these rather grandiose claims are pretty much all crap. Color me even less impressed than I was before—which was not much at all.

But what about the rest of this Special Focus issue?

I must admit that when I first started looking at this Special Focus Issue on Naturopathy, my original intent was to deconstruct the whole thing. I should have realized from the beginning that this was an impossible task to accomplish in one post, as it would require a level of logorrhea even beyond that which I normally achieve in a single post here. So, what to do?

I could either try to look at the rest of the articles in the issue or hold them over until next week. One problem, with the latter approach, of course, is that there’s so much stuff that could happen between now and then that I might never get to it. So I’ll just briefly touch on some “high points”. For instance, there is a review article comparing the bioavailability of different forms of folate. It’s actually rather boring because it’s about as close to a standard medical review as you’ll ever see from a naturopath. Or it would be if you didn’t know how much nonsense naturopaths believe about folate and the various forms used in supplements as related to MTHFR mutations of unknown significance that naturopaths like to test for and blame for a lot of conditions. Another study is a single arm safety study of a supplement intervention for prehypertension or stage I hypertension that did not test for efficacy. Another study is a pilot nonrandomized pre–post intervention trial to investigate physiological changes associated with participation in a 12-week structured naturopathic nutrition education course.

Then there were studies that really didn’t test anything about naturopathy, for instance a content analysis study of naturopathic treatment of endometriosis, dysmenorrhea, and menorrhagia. Another was a Delphi study of naturopathic approaches to treating irritable bowel syndrome. Still another was a capacity and needs assessment survey.

I remain unimpressed. Of course naturopathy can tart itself up with science all it likes, but as long as it continues to embrace vitalism, homeopathy, and all manner of quackery, it will never be science based, no matter how much Mr. Weeks believes otherwise.