A mathematical model for the persistence of quackery

I’m sure it’s obvious that I’m often puzzled (and, I daresay, many other skeptics and boosters of science- and evidence-based medicine are puzzled too) over why various forms of quackery and woo that have either about as close to zero prior probability as one can imagine and/or (more frequently “and”) have failed to show evidence of any therapeutic effect greater than that of placebo in clinical trials manage to retain so much traction among the public. Examples abound, for instance homeopathy and reiki, the former of which is nothing more than sympathetic magic prettied up with science-y sounding terms and the latter of which is nothing more than faith healing given a New Age mystic slant based on Eastern mysticism and religion rather than Christianity. Indeed, reiki was even inspired by stories of Jesus’ healing powers, complete with a trip into the wilderness for fasting and prayer, resulting in revelation. Another example is acupuncture, a modality that is seemingly more popular than ever, even invading the very sanctum sanctorum of the ivory towers of academic medicine and the hard-nosed halls of military hospitals, yet every study of which that is done under rigorous conditions with proper placebo controls shows it to be no more efficacious than a placebo. It’s easy enough to shake one’s head and chalk it all up to irrationality, ignorance of science, or even religious faith. However, I’ve always been dissatisfied with such glib explanations, even though admittedly I have myself used them on occasion–or even to what I like to refer to as “burnin’ stupid.”

That’s why a study released last week in PLoS One by Mark M. Tanaka, Jeremy R. Kendal, Kevin N. Laland out of the Evolution & Ecology Research Centre, School of Biotechnology & Biomolecular Sciences, University of New South Wales, the Department of Anthropology, University of Durham, and the School of Biology, University of St Andrews, Fife, respectively, entitled From Traditional Medicine to Witchcraft: Why Medical Treatments Are Not Always Efficacious. Besides loving the title, I also like the methodology, which in essence adapts the tools of modeling evolution and the spread of traits throughout a population and asks the question: Why do ineffective or even harmful (or, as the authors characterize them, “maladaptive”) treatments for various illnesses and conditions persist in human populations? The results that the model shows are surprising and counterintuitive, yet to me they ring true. In essence, the authors show, using (mostly) reasonable assumptions that the most efficacious self-treatments are not always the ones that spread through a population and that even harmful treatments can spread. Both of these observations are entirely plausible based on the prevalence of usage of common woo and quackery, and what the authors have done, in essence, is to model mathematically why quackery persists.

First, the authors set the stage:

In recent years, 60-80% of the world’s population, mainly from developing countries, depended primarily on traditional medicines, folk remedies and home cures, as well as treatment from witchdoctors and other ‘supernatural practices’, for their health-care needs [1]. In western societies, complementary and alternative medicine is garnering increasing interest and acceptance. At current growth rates, two-thirds of Americans are projected to be using alternative medicine by 2010 [2]. Asian governments are pouring billions of dollars into screening Traditional Chinese medicines in the hope that clinical trials will spawn lucrative drugs [3]. Traditional medicine has become big business.

Indeed, it has, which is why I always have to try desperately not to laugh out loud whenever advocates of “complementary and alternative medicine” (CAM) claim it’s all about the money for big pharma but not for CAM. But I digress:

While scientific studies have validated some traditional remedies, for instance, by confirming the biological activity of plant extracts [4], [5], the use of complementary and traditional medicines remains contentious, and doubts about the efficacy and safety of many treatments remain [1], [6], [7], [8]. Reservations over safety and efficacy underpin controversy over USA and UK universities’ attempts to bring alternative medicines into medical school curricula [9]. The active ingredients used in many traditional medicines are potentially toxic, often containing dangerous elements, including heavy metals [5], [10]. Even the use of ineffective non-toxic remedies can be harmful if it delays effective treatment. For instance, fears have been expressed that, in Nigeria, witchcraft and traditional remedies of unknown efficacy are widely employed as treatments for malaria, instead of, or delaying access to, modern medicines of proven effectiveness [11]. In sub-Sarahan Africa there is a concern that the use of traditional remedies for mastitis, a condition often attributed to sorcery, may inadvertently be contributing to the spread of HIV [12].

In 2002 the WHO [1] launched a global plan to make the use of traditional medicine safer by encouraging evidence-based research on the safety, efficacy and quality of traditional practices. Accordingly, traditional medicines are currently undergoing scrutiny to evaluate their effectiveness and monitor adverse events [3], [13]. Such analyses have often failed to confirm the efficacy of traditional remedies: for instance, of nearly 25,000 applications for registration of traditional medicines received by Malaysian authorities, 37.3% were rejected, either on grounds of safety or ineffectiveness [14]. However, there is currently no compelling explanation for the prevalence of low-efficacy treatments.

As I said before, I never found explanations of lack of scientific understanding or confusing correlation with causation entirely satisfactory, although I do make an exception for people like J.B. Handley and many antivaccinationists, who clearly do lack scientific understanding and confuse correlation with causation. Indeed, I definitely think these are major reasons why, for instance, so many parents believe vaccines cause autism or so many people think that reiki works. I just don’t think they’re enough.

Tanaka et al basically construct a model in which they attempt to estimate the factors that could affect the spread of various treatments throughout the population. One aspect of this model to be kept in mind is that it primarily looks at self-medication, although parents medicating their children, as happens in too much autism quackery, could be included under this category, mainly because it is not the children who decide for themselves whether a treatment is working or not; it is the parents. The general assumptions of the model are summarized in Figure 1 below:

Table 1 lists the parameters inputted into the model:

I’m not going to go into the equations in much detail. In fact, I’m not going to go into the equations at all. (Perhaps blogchild Mark Chu-Carroll would be interested in a detailed analysis.) The article is freely available to all; so you can read it yourself. What I will discuss are some of the assumptions going into the model. First among these is this:

We assume that individuals are either in a diseased state or in a healthy state. We model the spread of a behavioural trait expressed in treatment of disease. The behavioural trait in question is any innovation, practice or treatment that could potentially affect the outcome of this disease. To model the spread of a behavioural trait, we make the following assumptions. A new behavioural trait arises in (or is invented by) an ill individual who may then demonstrate this practice; others who are ill may adopt the practice upon being exposed to it, and then become demonstrators themselves. In other words, demonstrators convert observers. There is empirical support for the assumption that self-medicative treatments spread through social learning [22].

The most important assumption made for the model is that observers adopt the trait in an unbiased fashion. The authors don’t assume a priori that efficacy has anything to do with whether observers take up a treatment. While this may sound unreasonable on the surface, it is actually a reasonable assumption, as most observers are not very good at judging the efficacy of a treatment. Any regular reader of this blog should be aware of why this is true; it’s basically because any observer trying to determine whether or not a treatment is efficacious on the basis of watching it being used on another person or hearing about it, is using either anecdotal evidence or testimonial evidence to come to a conclusion. If there’s one theme that’s run through this blog from the very beginning, it’s how easily we as humans are deceived by anecdotal evidence and that testimonials are usually worthless. Indeed, unless a remedy is rapidly fatal or produces a miraculous recovery, given the effects of regression to the mean, confirmation bias, treatment bias, and a variety of other biases, judging the efficacy of a therapy on the level of an individual can be very problematic even for physicians. For the untrained, it’s really probably very close to random chance whether a person judges the efficacy of a treatment correctly. Actually, it’s probably even worse. As I have said, we humans are very good at deceiving ourselves.

The authors define the “cultural fitness” of a treatment as the mean number of converts produced by a demonstrator and then studied what the value of cultural fitness would be under various assumptions, both for instances of single illness and in the case of multiple relapses of illness. Interestingly, although highly efficacious treatments are predicted to spread through the population, even in the case of inefficacious treatments, multiple exposures of it and a remedy for it actually increase the chance that the treatment will spread, reagardless of whether the treatment is efficacious or not. The authors concluded that, although highly efficacious treatments have a high cultural fitness, treatments with efficacy near or at zero can also spread. The conditions under which such remedies could spread and achieve high cultural fitness are when treatments are primarily demonstrated in sickness and demonstrate low abandonment, particularly where the likelihood of relapse is small. The authors summarize:

This study offers a simple, novel and counter-intuitive hypothesis for the prevalence of ineffective medical treatments: unbiased copying of new treatments can frequently lead to the prevalence of ineffective practices because such treatments are demonstrated more persistently than efficacious alternatives, even when there is enhanced abandonment of ineffective cures. By unbiased copying, we mean copying in direct proportion to the rate at which the alternative variants are demonstrated. Here, in simple terms, treatment frequency dynamics are typically dominated by two processes, representing the rates of acquisition and loss of remedies. Maladaptive and superstitious treatments can become prevalent because their ineffectiveness prolongs illness, enhancing their rate of demonstration relative to efficacious cures, and leading to elevated rates of acquisition that may compensate for greater loss.

Our finding that superstitious treatments can easily spread is supported by reports of extraordinary treatments for conditions such as leprosy (treated with a drink made of rotting snakes) and syphilis (treated by eating a vulture), and by similar myths for poisonous snake bites (apply ‘guaco’ leaves, poisonous lizard skin or snake’s bile), dog bites (drink tea made from the dog’s tail) and scorpion stings (tie a scorpion against the stung finger) [23]. The analysis also helps explain the persistence of medical treatments of animals, such as ‘firing’ (cautery) of working horses, employed for millennia as treatment for lameness, where recovery is rare, and still widely practiced in many countries in spite of trials establishing its ineffectiveness [24].

Another conclusion of this model is that even highly effective treatments can be lost due to stochasticity, in other words, due to random chance. Indeed, the authors point out that most highly efficacious innovations would be predicted to be lost due to stochasticity. Indeed, my reading of this study would suggest that one reason why highly effective treatments actually do persist in our society is because scientists, physicians, and science-based medicine validate what treatments are efficacious, retaining the treatments that are and trying (but not always succeeding) in discarding the ones that aren’t. For people self-medicating, such a mechanism is not operative.

One aspect of this model that I consider inadequate is that it doesn’t take into account the role of peer pressure and groupthink. It certainly doesn’t take into account the rapidity with which glowing stories of success for quackery can spread through both word of mouth and other methods. In this case, the Internet truly has changed everything. Never before, has it been possible for so many people to “listen to so many stories” in such a short period of time or to form online communities made up of people who mutually reinforce each other’s beliefs, even though most of them have never met face-to-face.

I return to one of my favorite examples, autism quackery such as “biomedical interventions” and chelation therapy. Because autism is a condition of developmental delay, not stasis, it’s very easy to confuse correlation with causation and believe that a given ineffective intervention “worked.” Prometheus points out this very phenomenon:

If you have the opportunity, check in to one of the many “biomedically oriented” Internet groups and see what happens when somebody questions the idea that “biomed” can “recover” autistic children. At the least, they will be admonished – “Don’t stand in the way of other parents getting their children the help they need!”. More likely, they will be told to “Shut up!” and banned from the group. In some cases, they will be harassed and even threatened.

I’m not sure how the authors could have modeled the “Emperor’s New Clothes” phenomenon, such as Prometheus describes, but I can’t help but think that such mechanisms are at work in addition to the ones described in this paper. On the other hand, this model strikes me as useful for explaining the antivaccine movement. Vaccines are never “demonstrated” to cure a sick person; rather they prevent people from ever getting a particular disease, which means that this method of spreading in a population according to this model would not drive the use of vaccines. However, seeing rare adverse outcomes, given that the diseases these vaccines prevent are, thanks to the vaccines, no longer common, would lead people to see the outcome of vaccines as more negative than positive even though the evidence is overwhelming how the benefits of vaccines outweigh the risks.

Another aspect of this model is whether it can be modified to account for what I’ll call the “Oprah Effect.” This effect is due to highly influential (and often credulous) media people promoting a treatment to millions in the population, as Oprah does for various woo like The Secret, Mehmet Oz, or the various other dubious “experts” who are regulars on her show. I would label people like Oprah “super-demonstrators,” because through their efforts they can greatly increase the number of “demonstrators” in a population. Such a phenomenon undoubtedly went on to a much lesser extent in traditional societies in the form of shamans, chieftains, and village elders, but with mass media and the Internet the ability of such “super-demonstrators” to pump up the numbers dwarfs what could ever happen in a traditional society living a hunter-gatherer or early agrarian kind of existence. My guess is that it wouldn’t be too difficult to ad a factor to the model to account for “super-demonstrators.” I would also guess that the existence of these super-demonstrators would make it even more, not less, likely that ineffective or maladaptive treatments would spread through the population. But that’s just a guess.

As much as I like this paper for its description of a plausible mechanism by which implausible and ineffective treatments can spread through a population and remain entrenched, I have to caution that, without some sort of experimental or observational validation, it remains just that: a model. Unfortunately, I’m having some difficulty figuring out just how one might test this model against reality in a real human population.

One thing that this model leads me to wonder, though, is whether, maybe–just maybe–there is a force that can prevent or halt the spread of implausible, unscientific, and ineffective treatments, such as homeopathy, throughout the population. Perhaps such a method both serves as a memory that recalls which therapies don’t work and a method of testing which methods do work. That method is science- and evidence-based medicine.

I wonder how the effect of physicians supporting SBM and teaching it to their patients can be incorporated into this model. What I fear, given the infiltration of quackademic medicine into academia, is that what we’ll really have to add to the model is a factor to account for physicians also reccomending ineffective woo.