Straw men everywhere about evidence-based medicine

NOTE: Orac is on vacation recharging his Tarial cells and interacting with ion channel scientists, as a good computer should. In the meantime, he is rerunning oldies but goodies, classics, even. (OK, let’s not get carried away.) Here’s one from all the way back in 2007. Notice how, the more things change, the more they stay the same.

Evidence-based medicine is not perfect.

There, I’ve said it. Like anything else humans do in science or any other endeavor, evidence-based medicine (EBM) has its strengths and its weaknesses. On the whole, I consider it to be potentially vastly superior to the way that medicine was practiced in the past, bringing a systematic, scientific rigor to how we practice to replace parts of medicine that tended to be based as much (or more) on tradition or dogma as on evidence. Naturally, a common source of attacks on EBM is advocates of “alternative medicine,” who often appeal to “different ways of knowing” or postmodernist attacks on scientific medicine itself as being “just another narrative.” Of course, the reason for such attacks is obvious. EBM is a dire threat to their favored woo, which rarely, if ever, can demonstrate efficacy above that which can be attributed to placebo effect when tested in rigorous clinical trials. So it’s not unexpected that advocates of “alternative” medicine would find EBM threatening.

It’s also not entirely surprising that some physicians might also not be too enamored of EBM. After all, physicians are human, too, and we don’t always like being “told what to do,” which is how some doctors mistakenly perceive EBM as being primarily about. What’s depressing, though, is to see a physician make an all-out broadside attack on enough straw men about EBM that he probably defoliated Kansas and most of Nebraska finding enough straw. Such is the case with one Dr. Richard Dolinar, who’s penned an editorial at TechCentralStation entitled Evidence-Based Medicine Versus Patients. It isn’t pretty to watch as Dr. Dolinar either takes legitimate concerns about how EBM might be used or misused and then turns the Crank-O-Meter up to 11 or constructs elaborate straw men about what EBM is and then howls triumphantly as he tears them apart, straw hanging from his mouth like stuffing from the mouth of a dog who’s just ripped apart an unfortunate stuffed animal. The badness starts early, as Dr. Dolinar quotes Dr. John Service:

“[EBM is not] medicine based on evidence, but the equivalent in the field of medicine of a cult with its unique dogma, high priest … and fervent disciples,” says Dr. John Service, editor-in-chief of Endocrine Practice. Indeed, if a doctor questions EBM today, it seems he or she runs the risk of being branded an infidel or heretic, or worse.

Whenever I hear a physician or scientist refer to an evidence-based approach as being a “cult” or a “religion,” I know I’m probably dealing with at least a borderline crank. Such attacks are almost always simply an all-too-obvious attempt to label the object of the attack as being unscientific and based solely on dogma, particularly when the person making the attack is so eager to paint himself as an “infidel” and to claim the mantle of the persecuted martyr in the cause of Truth, Justice, and the American Way. (Can the Galileo gambit be far behind?) Usually, of course, the reason for such attacks is that the new “dogma” is a threat to dogma that the person launching the attacks prefers. Whether that’s the case with Dr. Dolinar or not is not entirely clear; I’ll give him the benefit of the doubt. But certainly what follows is one of the aforementioned straw men:

Proponents of EBM assume it will improve the quality of health care by basing medical decisions primarily on statistically valid clinical trials; therefore, information gained from randomized clinical trials (RCT) preempts information from all other sources. Yet, isn’t it ironic that a review of the literature by this author and others turns up no evidence as defined by EBM to validate this assumption?

“The failure to conduct a randomized controlled trial, the recognized best form of evidence according to EBM, and reliance on expert opinion, namely theirs (the worst form of evidence according to them), hoist EBM by its own petard,” notes Service. EBM purports to provide “statistical proof” when in fact what it provides is “statistical data.” Data does not necessarily equate to proof. Data is open to interpretation, which can change over time or vary depending upon one’s perspective.

This is just plain silly. Whenever I hear someone discuss “proof” as a “gotcha” against science, I really know I’m very likely dealing with a crank. Scientists do not generally speak of “proof.” We speak of “evidence.” Anyone who knows what EBM is knows that, although it considers data from well-designed randomized, double-blind clinical trials to be the strongest form of evidence supporting a therapeutic intervention, it recognizes that many questions do not lend themselves to such trials. For example, my field (surgery) is chock full of such questions. After all, no matter how we do a trial, it’s ethically dubious to conduct any sort of “sham” or “placebo” surgery, and it’s almost always impossible to blind the surgeon to what operation is being done. Consequently, we’re often forced to rely on the best retrospective data analyzed as carefully as possible to account for confounding factors that we can find. Yes, it’s not as “clean” as randomized trials, but it’s the data that we have and the data that we can ethically obtain. Does this mean that surgery is not evidence-based? (No sarcastic comments from internists, please.) No! Evidence-based medicine is about synthesizing the best available evidence; even though they are the strongest form of evidence, it does not require randomized clinical trials to be valid. EBM also recognizes that there are many conditions for which there aren’t enough patients to do a randomized trial with enough power to give definitive results and that such trials need careful interpretation when they are applied to individual patients. In brief, there’s nothing inherent in EBM that doesn’t recognize that evidence can be subject to interpretation.

At this point, it’s worthwhile to look at what EBM actually is, rather than the pathetic straw man version of it that Drs. Dolinar and Service create:

Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. By individual clinical expertise we mean the proficiency and judgement that individual clinicians acquire through clinical experience and clinical practice. Increased expertise is reflected in many ways, but especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients’ predicaments, rights, and preferences in making clinical decisions about their care. By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens. External clinical evidence both invalidates previously accepted diagnostic tests and treatments and replaces them with new ones that are more powerful, more accurate, more efficacious, and safer.

Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicabl e to or inappropriate for an individual patient. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients.

Sounds horribly constraining, doesn’t it? At the risk of helping him defoliate the Midwest even further, I have to ask: Is Dr. Dolinar saying that we as physicians shouldn’t be guided by the best clinical and scientific evidence available or that our personal preferences should override such evidence? Even his example isn’t convincing, namely that of the treatment of elevated intraocular pressure with no visual field loss or optic nerve damage, even though he seems to consider it a slam-dunk indictment of EBM. He notes that, without treatment, around 10% will progress to have visual field loss, while, with treatment, only 5% will. From this, he asks:

This data can be used to argue either for or against treatment, Spaeth notes, depending on one’s interpretation and incentives. The treating physician could argue that instituting early treatment would reduce visual field loss from glaucoma by 50 percent. Yet, a third-party payer with financial incentive could just as easily argue against treatment, noting that the overwhelming majority of patients with elevated intraocular pressure do not get worse, even when not treated. Consider the evidence. Who is right? They both are.

Not really. If the percentages were 1% vision loss without treatment and 0.5% with, he might–and only might–have a point. In fact, even apart from any cost considerations at all, physicians routinely ask if it’s worth treating 100 patients when only one of them will be helped. Of course, when we start talking about the difference between 5% and 10%, it’s hard to argue that doing nothing is acceptable. Even most insurance companies probably wouldn’t argue that because it would be penny wise and pound foolish. The patients losing visual fields would arguably end up costing more than the insurance company would save by denying the treatment. Indeed, insurance companies pay for the far more expensive and toxic chemotherapy that is used as adjuvant therapy in stage I breast cancer, even though it only increases survival by around 3-4% at best, meaning that for 100 women treated only three or four benefit. Far from being an indictment of EBM, what Dr. Dolinar’s example highlights is the issue of how EBM is used, which not the same as what EBM is or whether it improves medical care. While it is not unreasonable to be concerned that insurance companies may use EBM to try to cut costs, I will point out that EBM does not always support the least expensive interventions:

Evidence-based medicine is not cost-cutting medicine. Some fear that evidence-based medicine will be hijacked by purchasers and managers to cut the costs of health care. This would not only be a misuse of evidence-based medicine but suggests a fundamental misunderstanding of its financial consequences. Doctors practising evidence-based medicine will identify and apply the most efficacious interventions to maximise the quality and quantity of life for individual patients; this may raise rather than lower the cost of their care.

Indeed. Let’s say several randomized trials suggest that, for example, MRI is better for screening for breast cancer in than traditional mammography. Using MRI as a breast cancer screening tool would almost certainly raise the cost of care, given how much more expensive MRI is than mammography. There is also no guarantee that the money saved from the more cancers caught at an early stage would balance out with the cost of the far more expensive MRI as a screening test, which might mean that the savings from catching cancers early might not be enough to cover the increased cost of applying this test.

But that’s not all. Dr. Dolinar goes on to build up and attack the most common straw man argument about EBM used by physicians who fear that EBM might diminish their autonomy:

It is not the epidemiological data of EBM that I question, but rather the manner in which it is used to displace clinical judgment. The physician has taken the history, performed the physical, reviewed the labs, and discussed the illness with the patient and family. He knows the patient’s wishes, desires, and values. All this critical information must be considered when treating patients.

EBM, by contrast, relies primarily on epidemiological data, which it uses in a way that preempts all other information collected by the treating physician. In fact, non-quantifiable information such as the patient’s values and the physician’s clinical experience are not even taken into account in EBM.

It is absurd to think that a third party, operating at a distance in time and space from the patient being treated, is able to make a better medical decision than the treating physician and therefore should be allowed to preempt the treating physician’s decisions.

Dr. Dolinar also opines that EBM will endanger medical innovation:

What’s more, by following them, don’t we freeze medical practice in time? How is progress to be made in health care if we are forced to walk in lockstep with algorithms promulgated last year or the year before?

No, no, no, no. Once again, EBM is not “cookbook” medicine. It does not “freeze medicine in time,” nor does it prevent physicians from using new evidence that wasn’t available when guidelines were written. Quite the contrary:

Because it requires a bottom-up approach that integrates the best external evidence with individual clinical expertise and patient-choice, it cannot result in slavish, cook-book approaches to individual patient care. External clinical evidence can inform, but can never replace, individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision. Similarly, any external guideline must be integrated with individual clinical expertise in deciding whether and how it matches the patient’s clinical state, predicament, and preferences, and thus whether it should be applied. Clinicians who fear top-down cook-books will find the advocates of evidence-based medicine joining them at the barricades.

I rather suspect that what bothers people like Dr. Dolinar the most about EBM is the clinical guidelines that are promulgated based on it. They tend to perceive such guidelines as “telling them how to practice medicine” and like to make a big to-do about how there may be different guidelines for different diseases, never considering that perhaps part of practicing EBM is to have the knowledge, skill, and, yes, clinical acumen to evaluate the literature well enough to be able to decide which guidelines are appropriate for a given patient and when to go outside of the guidelines. In other words, critics like Dr. Dolinar seem constrained by a concept that it’s all-or-nothing. They also seem hobbled by a fear that going outside of these guidelines will leave them open to the dreaded lawyers. The only time that would be true if a physician does something that’s not justifiable by reference to the literature and the standard of care; in other words, something that’s beyond the pale. Finally, of course, to some extent guidelines do “tell us how to pratice,” but since when have physicians practiced without someone or something “telling them how to practice medicine”? In the past, it was either the standard of care or the prevailing practice (or dogma) of the area, or even the chief of service at the hospital who happened to prefer one drug over another for a given condition and made sure that the physicians working at that hospital preferred it too. When all else failed, the state medical board would be there if they strayed too far from the standard of care. Personally, given that no physician has ever practiced medicine free from interference or guidelines, I’d rather those guidelines be as explicitly based on evidence as possible, rather than based on dogma and prevailing beliefs, as was so often the case in the “golden days,” when the “gods of medicine” roamed the earth. What I suspect is that many (although certainly not all!) physicians who chafe under the concept of EBM in reality either don’t understand what it really is or resent not having total control. Indeed, Dr. Dolinar makes this point explicit:

The ultimate discretion regarding how information from multiple sources (including EBM, prior clinical experience, and the patient’s unique circumstances, wishes, and desires) are integrated for treating individuals should be in the physician’s hands. Since he has the ultimate responsibility for the patient’s care, he should have the ultimate discretion.

Nothing in EBM ever said that the ultimate responsibility for the patient’s care isn’t in the hands of the physician. Another blogger, retired doc, goes even further:

The lure of the mantra of evidence based pathways or guidelines is exactly what third party payers can use in their rational quest to control physicians to control costs to control their profit stream. Control is the operative word. Recognition of this innate uncertainty is the last thing they want to admit to or deal with.

The lure of the mantra of evidence based pathways and guidelines works well for the academics and others who “partner” with the third party payers to control the working stiff docs in their rational quest to be members of the ruling class , to be first among equals, to be the animals that are more equal that the others or for whatever reason they have.

The lure of the mantra of evidence based medicine (EBM) fits nicely with those whose philosophical mind set includes the notion that medicine is too important to be left to the individual physician and the individual patient and that wiser heads must prevail so that people will do the right things for themselves.

Give me a friggin’ break. I normally like retired doc’s (a.k.a. James Gaulte) blog, but in this post Dr. Gaulte has gone a bit off the deep end, cranking up the paranoia even further than Dr. Dolinar did. (The term “drama queen” came to mind, as well.) I don’t disagree that there is the potential for abuse of EBM by third party payers, but Dr. Gaulte takes a reasonable concern and runs with it to a ridiculous extreme. There’s a huge flaw in his sort of attack as well, which is, in essence, that because third party payers might be able to hijack EBM to control doctors and constrain choice that EBM itself is flawed. That’s like saying that, just because mortgage companies do greedy things in the search of making money resulting in messes like the present crisis caused by the default of subprime loans, the entire concept of a mortgage is invalid and should be thrown out. Any system can be abused; that does not necessarily mean that the concept underlying the system is fatally flawed. It doesn’t even necessarily mean that the potential for abuse is so great that the system itself should be rejected. All it means is that implementation is the key.

Of course, a lot of what drives physician critics of EBM tends to be a healthy helping of anti-elitism, given that many EBM guidelines arise from academic medicine. Personally, I find this sort of attack to be specious. Is retired doc saying that private practitioners are so much better that they shouldn’t be bound by the evidence, that their judgment is such that it cannot accept being constrained in the way that we in academia do? Probably not, but certainly that seems to be the attitude behind the post, even if he doesn’t explicitly say it. Either that, or it’s just a strong resentment of academia. Actually, though, when I come to think of it, he’s closer to the truth than he thinks, just not in the way he thinks, when he points out that EBM can be used as a “talisman,” concluding, “Reference to EBM can serve as a talisman. To say that such and so is evidence based is to ignore the real question which is what is the nature of the evidence.” That metaphor cuts both ways. EBM certainly can be a talisman–against accountability and consistency. And, geez, does he really think that advocates of EBM don’t look at the nature of the evidence being invoked?

As I read these two broadsides against EBM, the thought occurred to me that they sounded a lot like something that might be published in the Journal of American Physicians and Surgeons (a.k.a. JPANDS), that repository of über-crankery known to publish, alongside its attacks on mandatory vaccination, its horrible science claiming that vaccines cause autism or that abortions cause breast cancer, and its general attitude that any constraints whatsoever on how physicians practice are to be fought tooth and nail as though they came from Satan himself (or even worse, from the government as a step towards socialized medicine). This seems to derive from an attitude from the past in which physicians were accountable to no one but themselves and their consciences and seemingly god-like in their autonomy. (Never mind that those days are long gone, whether they ever existed at all.) Not surprisingly, it didn’t take long for me to figure out that Dr. Dolinar is not only a member of the Association of American Physicians and Surgeons (AAPS), of which organization JPANDS is the house propaganda organ, but was recently elected to its Board of Directors.

I would not try to deny that EBM has the potential for abuse by third party payers seeking to decrease costs. That is something that will need to be guarded against. However, to me that does not seem to be the primary objection to it by physicians like Drs. Dolinar and Gaulte. What seems to disturb them more than anything else is the perceived loss of control that EBM represents to them, in which they seem to think that they must subordinate their clinical judgment to guidelines thought up by pointy-headed academic physicians in ivory towers who Don’t Know The Real World the way that they think they do. Such physicians seem to pine for some golden era from decades ago, where the primacy of the physician was unchallenged, they didn’t have to justify themselves to anyone (much less insurance companies or practice guidelines), and anything that constrained the way they practiced was considered intolerable. The problem is that this fear and loathing of EBM is based more on fantasy than reality. Fortunately, though, as for so many things in medicine, such attitudes are likely to go into retirement with Dr. Gaulte as a new generation of doctors is trained in EBM.

That is, if the woo infiltrating the curriculum in so many medical schools doesn’t muck it all up first.