A fallacy-laden attack on science-based medicine, revisited

On Monday I took a blogger by the name of Dr. Marya Zilberberg to task for firing a series of profoundly anti-scientific broadsides against science-based medicine (SBM). Although I did not attack Dr. Zilberberg personally, I was quite harsh in my characterization of her attacks, because, well, they were quite bad, full of straw men, special pleading, and the claim that absence of evidence is not evidence of absence, all topped off with a particularly egregious mischaracterization of what SBM is. Steve Novella also piled on, which was appropriate because Dr. Zilberberg’s attacks were mainly directed at him.

ON MEANIES AND “TONE”

Yesterday, Dr. Zilberberg responded in a post she called Furthering the discussion. I, of course, am quite happy to further the discussion. However, before I do that, I, like Steve Novella, feel obligated to address the “tone” issue, because it comes up fairly frequently. First, I note that Dr. Zilberberg’s first response to criticism was actually quite dismissive. She accused Steve of doing “such a great job cherry picking” her arguments, and in response to Steve’s valid (in my opinion) complaint that he was tired of defending SBM against false charges of paternalism and arrogance, her response was “If the shoe fits.” Consequently, my response was tailored to Dr. Zilberberg’s original post. After all, she accused supporters of SBM of “arrogance” and “paternalism.” Indeed, her tone could not in any way be described as “civil,” her stated desire for a “civil discourse” notwithstanding. Seriously, if Dr. Zilberberg really wants a civil conversation, she has a rather odd way of showing it, particularly since I read other parts of her blog and saw evidence that this sort of nastiness is a pattern. For example, she’s referred to “rabid vaccine advocates,” accusing them of hysteria. Then she’s surprised when she receives responses that are as–shall we say?–as vociferous as a typical Orac response.

Don’t get me wrong. I love a good sarcastic throwdown more than the average blogger. After all, I cut my skeptical teeth on Usenet, that vast untamed discussion forum favored during the 1990s before the rise of weblogs and web-based discussion forums. There’s just one requirement that I learned well over a decade ago: If you’re going to be sarcastic, you’d better have all your ducks in a row. You’d better have the goods. You’d better have the science and evidence. There’s nothing more embarrassing than to get all sarcastic and self-righteous and then not have the goods to back it up. That’s exactly what happened with Dr. Zilberberg’s little broadside against SBM, as Steve and I demonstrated. Unfortunately, Dr. Zilberberg can’t seem to resist coming back for more. Fortunately, she appears to have realized her mistake, at least in that respect.

ON THE EVIDENCE BASE BEHIND SCIENCE- AND EVIDENCE-BASED MEDICINE

Issues of tone aside, Dr. Zilberberg’s arguments in her response are only marginally better than in her original post, beginning here:

I do believe our views are more same than different. We both (SBM group and I) understand that science evolves, that evidence is not stagnant and the sense of certainty frequently conveyed to the lay public by the media is oftentimes misplaced. We simply disagree on the extent to which there is uncertainty in evidence. While it is true that the oft-cited 5-20% number representing the proportion of medical treatments having solid evidence behind them is very likely outdated, the kind of evidence we are talking about is a different matter.

The problem is that that 5-20% number, which by the way is beloved of quacks and cranks, is a myth, as Steve Novella described three years ago. Bob Imrie tracked down the origin of this myth ten years ago and found that it dates back to a small survey of primary care offices in England back in 1961. Worse, the survey didn’t even really look at the evidence base behind practices; rather it was designed to look at prescribing practices and costs related to prescription drugs and generic drugs. It turns out that the numbers are much higher, ranging from 97% of anesthesia practices (32% by randomized clinical trials, RCTs) to 84% of internal medicine practices (50% by RCTs), with an average of 76% of interventions being supported by compelling evidence and 37% by the “gold standard” randomized clinical trials. Obviously, there is some subjectivity in deciding what constitutes “compelling” evidence, but by any reasonable standard it’s far higher than 5-15%. Even if you demand only randomized controlled trials as your standard of evidence, it’s far higher than 5-15%.

Contained within Dr. Zilberberg’s response is a more reasonable discussion of levels of evidence. For the most part, albeit with a rather negative spin compared to what I consider to be appropriate, she is correct when she points out that RCTs are often difficult to generalize and that metanalyses share the same sorts of problems, given that they are made up of multiple RCTs analyzed together. She is also correct that physicians often go beyond the indications supported by RCTs in presecribing treatments. However, I have a hard time viewing this problem as being more of a problem of physician education than a problem necessarily inherent in SBM. In any case, what strikes me about Dr. Zilberberg yesterday as opposed to Dr. Zilberberg several days ago is that she seems to be arguing that applying science to medicine is messy and difficult, which is something I’ve said all along and utterly not a point of contention. Remember, too, that in her first posts she characterized those of us who support SBM as being far too confident, dogmatic, and sure of the evidence, which both Steve and I both pointed out is not the case. She accused SBM of being far too “paternalistic” because of this alleged overconfidence in science and evidence and our interpretation of the evidence as being far too simplistic. My tendency towards sarcasm aside, I can’t recall ever arguing that medicine and applying scientific evidence to medicine is straightforward or simple.

Where Dr. Zilberberg goes wrong is in her own self-admitted penchant for observational data:

The next rung of the evidence ladder is observational data, specifically cohort studies first prospective, then retrospective. I am actually a great fan of observational data, as I have mentioned in the past. Cohort studies give us the opportunity to examine what happens in the real world without imposing artificial conditions necessary in a clinical trial. Observational data can be great when describing epidemiology of a particular disease, the frequency of a given exposure, how different characteristics can modify the relationship between the exposure and the outcome. One of the most attractive features of cohort studies is that the population can be observed over long period of time — just look at the Nurses’ Study, the Framingham Cohort, and others. But these types of studies also have important limitations, and these are readily acknowledged as a heightened susceptibility to bias (especially in the retrospective studies), the possibility of misclassifying important events, and, despite our best efforts to adjust for it, residual confounding. I will come clean and admit my affection for observational data, even despite the fact that it is lower on the totem pole of evidence than an RCT.

Observational studies can indeed be important. Even more importantly, sometimes for ethical reasons RCTs are not possible. When that’s the case, we have to rely on observational data of the type so beloved of Dr. Zilberberg. However, there is a problem. Such studies are are far more prone to confounders and bias. Worse, really big observational studies, like Nurses’ Study and the Framingham Study, are extremely expensive, costing many millions of dollars each. It’s often just not feasible to do huge observational studies of this sort for any but the most common conditions. Personally, I consider it a big mistake to be so fond of observational studies. They can tell us a lot, but their limitations are such that RCTs should be done whenever justified by preclinical data and financially feasible. The bottom line is that each type of study has its place based on its unique strengths and weaknesses. Moreover, as I hope Dr. Zilberberg would agree, one characteristic of pseudoscientists is that they drill down on individual studies, demanding “just one study” that “proves” the point that an advocate of evidence-based medicine (EBM) and SBM is arguing. It’s not that easy. It takes a convergence of evidence from multiple studies, all leavened with a consideration of all sources of evidence, including basic science.

Which is all we are saying when we refer to SBM, that all scientifically valid sources of evidence need to be considered.

ON “ALLOPATHIC” MEDICINE

Regarding CAM, Dr. Zilberberg tries to clarify what she was arguing (which she really needed to do because what she was arguing was a fetid load of dingo’s kidneys). Unfortunately, she can’t resist another swipe at “allopathic” medicine:

I will try to tackle my CAM argument next. If I in any way implied in my remarks that I encourage allopathic physicians (by the way, I am not using it in a derogatory way, but merely as it is defined here; in fact, until today I was blissfully unaware of its negative connotation) to be purveyors of CAM, I sincerely apologize.

This borders on a not-pology. Personally, I find it hard to believe that Dr. Zilberberg was “blissfully unaware” of the negative connotations of the term “allopathic” physicians given how often I’ve now seen her use the term while castigating SBM for “arrogance” and other shortcomings. Personally, I’d suggest that if she is so unaware of the negative connotation of the term, she is currently too uninformed to be blogging about such topics and should refrain from doing so anymore, at least until she educates herself a bit more on the basics of the tactics and arguments of CAM advocates used to distort evidence and science. Of course, if you check out the link she uses to define allopathy, I fail to see how Steve and my criticism of her for using the term “allopathic physicians” wasn’t spot on. After all, that link defines “allopathic medicine” thusly:

The system of medical practice which treats disease by the use of remedies which produce effects different from those produced by the disease under treatment. MDs practice allopathic medicine. Also called conventional medicine.

The term “allopathy” was coined in 1842 by C.F.S. Hahnemann to designate the usual practice of medicine (allopathy) as opposed to homeopathy, the system of therapy that he founded based on the concept that disease can be treated with drugs (in minute doses) thought capable of producing the same symptoms in healthy people as the disease itself.

Which is exactly the definition that Steve and I pointed out. Seriously, using a definition that the creator of what is arguably the quackiest of quackery, homeopathy, used to describe medicine is not derogatory? Does Dr. Zilberberg read various CAM sites? How could she not be aware that the term “allopathic” is a frequently used derogatory term to describe science-based medicine. Oh, well.

ON VACCINES AND UNWITTINGLY GIVING AID AND COMFORT

Dr. Zilberberg finishes up with two issues. First, she restates her position on CAM. Second, she is very unhappy at how some commenters have characterized her as “anti-vaccine.” I’ll deal with the second one first because it’s quicker:

My final words will be about vaccination. It is disheartening to be lumped with “anti-vaxers”, as has been done in the comments to Dr. Novella’s and Orac’s posts. While my bruised ego will survive this insult, I would like to question this assertion. Nowhere have I said that vaccinations are a bad idea or present a real danger to our children. The hype surrounding the vaccination-autism “debate” is abhorrent to me.

That’s a reasonable start. However, there is a strain of antivaccine thought that argues that vaccines are useless or that their benefits are hugely exaggerated and then using that argument to dismiss vaccination programs, often those for vaccines with a less compelling indication than, for instance, the measles vaccine. Kind of like this:

What I have stated, however, is that I am of the opinion that we have gone a bit overboard with some of them, one being the chicken pox vaccine. Now, this does not make me an “anti-vaxer”; this just makes me a bit skeptical. The way I view the data is that the advantages for this vaccine are mostly economic, in that they prevent parents from missing days at work. Now, I am certainly not opposed to making such a vaccine available to parents who desire it, but I am not convinced that it should be a prerequisite for my kid to go to school. Given that there is always a possibility of an adverse reaction, no matter how small that possibility is, if the risk of it may outweigh the benefit (and here I do not mean the benefit of having mom show up at work), it has to be weighed very carefully.

No one argues otherwise, really. I mean, is there anyone out there who argues that risks and benefits of vaccines shouldn’t be weighed carefully in determining vaccination policy? Admittedly, though, readers pointed out that Dr. Zilberberg had shown up in the comments of a post of mine from three months ago critizing an anti-vaccine quackery site claiming that the “whole thing feels like a schoolyard brawl,” accusing me of “parochialism,” and “hiding behind ‘evidence is on my side’ arguments.” Moreover, as part of that discussion, she wrote:

E.g., vaccinations: overall a very useful public health intervention under certain circumstances (life-threatening diseases, high contagion potential), safe when examined as single exposures. It is difficult to study potential interactions with other exposures, be it vaccines or something else, as well as long-term consequences. So, while some conclusions are warranted others are less obvious.

I realize that Dr. Zilberberg probably doesn’t realize it, but this line of argument comes perilously close to Jenny McCarthy’s beloved “too many too soon” slogan. Let me also say that I believe her when she says she is not anti-vaccine. Even so, she carelessly throws around rhetoric that, whether she realizes it or not, echoes a lot of anti-vaccine rhetoric. (And who is more of an expert on anti-vaccine rhetoric than I? Not many.) As Steve so aptly put it, what she’s doing is akin to someone expressing skepticism towards some aspect of evolution and thereby appearing to support creationism because she didn’t know the ways creationists distort and abuse science in the name of attacking evolution. She does the same thing here with vaccines. For example, elsewhere on her blog, as I mentioned before, Dr. Zilberberg referred to “rabid defenders of vaccines,” while complaining about the lack of philosophical exemption laws in most states. Let’s put it this way. If you don’t want to be perceived as an anti-vaxer, don’t refer to defenders of vaccines as “rabid” and don’t start referring to the possibility of vaccine interactions in a way that is reminiscent of the arguments that anti-vaccine advocates make. I realize that Dr. Zilberberg’s mistake is probably due to ignorance of the corrosiveness of the anti-vaccine movement, the depths of pseudoscience to which it regularly descends, and a lack of familiarity with their fallacious arguments, but hopefully this exchange will serve to educate her to be more careful in the future.

ON THE EVIDENCE BASE FOR CAM

When Dr. Zilberberg revisits the CAM issue, she uses one of the most irritating arguments that CAM apologists like to use, namely “What’s the harm?” It’s tempting simply to refer her to What’s the Harm?, but that’s a bit of the lazy way out. So would referring to Steve’s post that apparently provoked this whole exchange. Yes, many CAM therapies cause direct harm. Some cause harm by omission, by either interfering with or preventing effective conventional therapy. Then there is the harm that comes from teh erosion of the scientific basis of medicine that mixes pseudoscience with science to the point where it is hard to differentiate the two without considerable background knowledge. Besides being ignorant of how chiropractic is (and is not) regulated, Dr. Zilberberg also makes the rather bizarre argument that herbs should not be regulated because they grow naturally and we don’t generally regulate what patients grow in their backyard, except for marijuana. The problem is that most patients don’t grow their herbs in the backyard; they purchase them in the form of supplements. Moreover, this is not strictly an issue of regulation. There’s no need for a regulation to tell a patient using herbs grown in his backyard to treat cancer that the herbs won’t cure his cancer and to strongly suggest that the patient stop using them.

Perhaps the most puzzling passage is this:

What is coming through for me is that perhaps my call to equipoise was a little over the top, as I do not seem to be approaching the above CAM issues in a frequentist, but more in a Bayesian way (though I remain committed to equanimity). Yet, there is something to be said about the frequentist approach, even though it is not my way generally. The frequentist approach, which is what underlies the bulk of our traditional clinical research, does not rely on differential prior probabilities for different possible associations, but treats them all equally. Despite many disadvantages, one obvious advantage is that we do not discount potential associations that do not have biologic plausibility, given our current understanding of biology, and sometimes help us stumble on brand new hypotheses. So, clearly, there is a tension here, and I am still working on what is the better way, if any.

I was having a bit of a hard time figuring out just what Dr. Zilberberg was driving at here. Underlying Dr. Zilberberg’s argument seems to be the assumption that those of us who advocate SBM incorporating prior plausibility into analyses of possible associations think that determining these prior probabilities is straightforward. I doubt you’ll find a single SBM advocate make such a claim. What we argue is that exceedingly implausible prior probabilities should be treated that way–as exceedingly implausible. That includes modalities like homeopathy, reiki, and therapeutic touch, among others. Herbs and many other CAM modalities with a physical effect and potential mechanism of action that doesn’t violate multiple laws of physics à la homeopathy can arguably reasonably be treated by frequentist approaches. Herbs would be one example of that, actually.

Her denials notwithstanding, it is obvious that Dr. Zilberberg has a definite sympathy for CAM. Indeed, in another post entitled Allopathic medicine and CAM: Nonoverlapping magisteria revisited?, Dr. Zilberberg explicitly likens the conflict between SBM and CAM to the conflict between science and religion as exemplified by the conflict between defenders of evolution and creationists. She even argues that SBM (or “allopathic medicine,” as she puts it) and CAM are, as Stephen Gould so famously argued about science and religion, nonoverlapping magesteria. My guess is that she doesn’t realize quite how appropriate likening CAM to religion is, given that so much of CAM either derives from religious ideas (reiki, for instance) or prescientific beliefs about how the body works (acupuncture and much of traditional Chinese medicine). The money passage, however, is this:

Since we live in a time when polarization seems to be the norm (just look at our political discourse), it is natural for allopathic medicine and CAM to retreat more deeply into their own corners and to become more entrenched in and convinced of their own singularity. This is the wrong approach. Humans are not all easily-understood physiology, but we are also not all spirit and mystery. We are in fact both. Some of the conditions we define as physiologic illnesses are nothing more that the products of our distorted expectations and philosophies. Some of our impulses to treat cancer with CAM alone are misinformed. If acupuncture seems to help my neighbor with her subjective symptoms of menopause, so be it, I am happy for her, even if I do not fully understand how it works. If yoga gives me a sense of well-being, yet there are no randomized controlled trials to validate this assertion, so what?

This is yet another straw man argument. Actually, I’ve heard Kimball Atwood, for instance, say something very similar about massage: If it feels good, go for it. He even said at TAM8 that he sees no reason to do randomized clinical trials to demonstrate that massage feels good, and I had a hard time disagreeing with such an obvious assertion. The claim that defenders of SBM call for clinical trials on self-evident issues like that is a straw man. The problem advocates of SBM have with modalities such as massage, yoga, and other similar modalities is when they are represented as therapeutic modalities that treat specific conditions. When that happens, a claim is being made that these are therapies, rather than exercises or modalities that make you feel good (or “gives you a sense of well-being,” as Dr. Zilberberg puts it). In that case, there is a claim for therapeutic effect that should be tested before physicians should be recommending it. Moreover, clearly Dr. Zilberberg has tendencies towards mind-body dualism and is appealing to spirit and mystery, which may explain much of her “openmindedness” to the point of having her brains fall out. Personally, I’d be very interested in her evidence supporting the view that we are both “easily understood physiology” and “spirit.” Given how much Dr. Zilberberg pooh-poohs the evidentiary basis of modern medicine as being so weakly based in verifiable science, one can’t help but respond to her assertion with, “Pot. Kettle. Black.”

Perhaps the most disturbing aspect of Dr. Zilberberg’s argument is that she is arguing the entire issue from a false assessment of equality between SBM/EBM and CAM, or “equinamity” and “equipoise,” if you will. True, she does seem to back away from the equipoise in her latest post, but she’s also holding firm to advocating an equanimity that is not deserved, at least not on the basis of science. Regardless of whether Dr. Zilberberg realizes it or not, even in “frequentist” analyses of various interventions for diseases and conditions, we do, at least intuitively, consider prior probability, just not formally or necessarily explicitly. Basically, before the rise of CAM, scientists were unwilling to spend scarce medical research dollars and their own limited time studying interventions that had a very low (or even close to nonexistent) likelihood of being validated. Now, we spend millions of dollars on such studies, not because the science compels us but because ideology makes it attractive to do so.

Fortunately, I sense that Dr. Zilberberg appears to realize that she may have overstated her argument, at least in the way she seems to be backing off from stating that we should a priori show “equipoise” to all therapeutic modalities. Unfortunately, in considering CAM versus SBM, she continues to fall into the same trap of asserting dualism, a concept that is, like so many religion-inspired concepts, rapidly falling prey to science, in this case, advances in neuroscience, and arguments from ignorance or the unknown like this. She even recognizes that she is appealing to mind-body dualism:

But what are we unable to measure? Oh so much! The burgeoning science of neurobiology, for example, has raised so many interesting questions about not only what the mind can do to the body, but what the body can do to the mind (please forgive this dualistic language). Why is this important? Because, due to our lack of adequate tools until recently, and because of the overwhelming complexity of the subject, we have traditionally neglected to include any measures of our patients’ and trial subjects’ neurobiological milieu into the consideration of differences between groups. But if randomization takes care of other systematic differences, should it not take care of the neurobiological ones? Perhaps, but without understanding the magnitude of variability of these characteristics in a population, one cannot begin to know how large a swathe of the population has to be enrolled in a study in order to smooth out these potential differences. And this goes for other so far unknown or unidentifiable characteristics.

In other words, because we can’t measure everything about the brain and how it interfaces with the body, woo must work (or, more correctly, we can’t prove that woo doesn’t work). That’s what much of Dr. Zilberberg’s viewpoint with respect to CAM appears to come down to. Until she can let go of her dualism and appeals to things that science can’t measure, I fear that we will be talking past each other.