On the messiness of evidence-based medicine

I’m about a week late on this one.

In fact, it wouldn’t surprise me if some of my readers were wondering why I hadn’t weighed in on this story when it broke late last week. (Either that, or no one was wondering, and I’m just displaying some of my surgeon’s ego for all to see.) Part of the reason was perhaps because Dr. Charles had handled this whole study well, and I didn’t see any need to weigh in. Another part of the reason is because the study came out right when I had to come up with another Your Friday Dose of Woo for last Friday. But, the more I thought about it, the more I thought that this is a perfect example of something that I should have talked about a while ago: the messiness of evidence-based medicine (EBM). I’m referring, of course, to the abstract that statistician Donald Berry, Ph.D. of the University of Texas M.D. Anderson Cancer Center presented to the San Antonio Breast Cancer Symposium last week:

In 2003, breast cancer incidence in the United States dropped sharply, and this decline may largely be due to the fact that millions of older women stopped using hormone replacement therapy (HRT) in 2002, according to a new analysis led by researchers at The University of Texas M. D. Anderson Cancer Center.

At the 29th annual San Antonio Breast Cancer Symposium, the investigators report that there was an overall 7 percent relative decline in breast cancer incidence between 2002 and 2003, and that the steepest decline – 12% – occurred in women between ages 50-69 diagnosed with estrogen receptor positive (ER-positive) breast cancer. This is the kind of breast cancer that is dependent on hormones for tumor growth.

From this, the researchers conclude that as many as 14,000 fewer women were diagnosed with breast cancer in 2003 than in 2002, a year in which the American Cancer Society estimates 203,500 new cases of breast cancer were diagnosed.

“It is the largest single drop in breast cancer incidence within a single year I am aware of,” says Peter Ravdin, M.D., Ph.D., a research professor in the Department of Biostatistics at M. D. Anderson.

“Something went right in 2003, and it seems that it was the decrease in the use of hormone therapy, but from the data we used we can only indirectly infer that is the case,” he says.

“But if it is true, the tumor growth effect of stopping use of HRT is very dramatic over a short period of time, making the difference between whether a tumor is detected on a mammogram in 2003 or not,” says Ravdin.

The study’s senior investigator, Donald Berry, Ph.D., professor and head of the Division of Quantitative Sciences at M. D. Anderson, says he was, at first, very surprised by both the magnitude and the rapidity of the decline in incidence, but adds “it makes perfect sense” if you consider that use of HRT may be an important contributing factor to breast cancer development.

“Incidence of breast cancer had been increasing in the 20 or so years prior to July 2002, and this increase was over and above the known role of screening mammography,” he says. “HRT had been proposed as a possible factor, although the magnitude of any HRT effect was not known. Now the possibility that the effect is much greater than originally thought all along is plausible, and that is a remarkable finding.”


I’m a little wary of this study, mainly because, like Dr. Charles, I’m not yet entirely convinced that this isn’t a one year statistical fluke. However, if it isn’t, and breast cancer incidence either keeps decreasing or remains at the new, lower level, then this is huge. Whether the cause for this decrease is indeed the decrease in HRT that resulted from the publication in 2002 of the Women’s Health Initiative study that demonstrated that hormone replacement therapy resulted in increases in heart disease and cancer is, of course, another matter.

Of course, what confuses both patients and even many doctors is the messiness of the whole HRT story. How on earth could medicine have gotten things so wrong for so long? Let’s take a trip back in time to the early 1990’s, when the WHI study was conceived and designed. At the time, the widespread belief among medical professionals that HRT had a cardioprotective effect was based on largely retrospective studies and animal studies. For example, there was retrospective data that strongly suggested a decrease in risk for cardiovascular events in women who took HRT that was as high as 40% or 50% compared to postmenopausal women who did not take HRT. Additional studies seemed to show a positive effect on blood lipid levels. Meanwhile the evidence of adverse outcomes, particularly in terms of breast cancer, was conflicting. Put yourself in the position of a gynecologist or primary care physician (the two specialties most likely to be prescribing HRT). Nothing is more effective in relieving menopausal symptoms than replacing the hormones themselves, and the evidence seemed to suggest that HRT had the additional benefit of decreasing the risk of heart disease. Do you offer it to your patients who are beginning to have hot flashes? I bet you probably would recommend a drug that not only relieved distressing symptoms, but also appeared to protect against heart disease and osteoporosis without good evidence that it increased the rate of adverse outcomes like breast cancer. Certainly, patients wanted it, and there didn’t seem to be much, if any, downside.

Still, even while the WHI study was going on, there were rumblings. Smaller studies appeared that suggested that all was not well with HRT, that the risk of blood clots and strokes were higher, but this somewhat conflicting evidence came against fairly consistent evidence that HRT delays osteoporosis and decreases hip fractures, and decreases the risk of colorectal cancer slightly. The WHI was the first really large randomized trial, and its results showed an increased risk of stroke, pulmonary embolus, and breast cancer, to the point where the HRT arm of the study was halted early, leaving the average followup at only 5.2 years.(Ironically enough, in the very same issue of JAMA in which the WHI study appeared, there was also a study demonstrating that long term use of estrogen-only HRT significantly increased the risk of ovarian cancer, while estrogen-progestin HRT, the kind studied in the WHI, did not.) The results were widely publicized; medical practice over the last couple of decades was overthrown and changed almost overnight. Those of us in the breast cancer business faced breast cancer patients torturing themselves over whether they had caused their cancer by taking HRT and other patients asking us whether they should stop their HRT or whether they had increased their odds of getting breast cancer.

Yes, EBM sometimes bites its advocates on the behind.

The point is, physicians who recommended HRT to their patients pre-2002 were practicing EBM just as much as those who now do not recommend HRT except in the cases of highly refractory menopausal symptoms, where the patient is willing to accept the increased risk. It also points out a truism in medicine that giving a medication or using an intervention in a healthy population has to be held to a very high standard in which the benefits of the intervention far outweigh the risks of the intervention. Clearly the WHI shows that HRT does not meet this standard for long term use to prevent heart disease and osteoporosis.

But practicing EBM can be even more messier than that. Let’s look at the M.D. Anderson study. Assuming this one year decline in breast cancer incidence is not a fluke, even though the results of this study are consistent with the WHI showing that HRT increases the risk of breast cancer, it’s difficult to determine whether this decline was due to decline in the use of HRT that occurred in 2002 after publication of the WHI study. For one thing, 2003 seems as though it would be too soon for such a decline to occur, given that breast cancer generally takes years to develop from normal breast duct linings into frank cancer. Here’s another reason to wonder if it’s more than just HRT, straight from the M.D. Anderson press release itself, which shows that breast cancer incidence had been leveling off for five years before 2003:

They examined rates of breast cancer in the United States from 1990 to the end of 2003 and found that while incidence increased at 1.7% per year from 1990 to 1998, it began to decrease, relative to other years, 1% each year from 1998 to 2002. When that 1% increase was adjusted for age in each of those years, incidence from 1998 to 2002 stayed about the same, Ravdin says. “There were more cases of breast cancer being diagnosed, but that was because women were getting older and entering the higher risk pool.”

But by the end of 2003, there was a 7%, age-adjusted decrease in the number of breast cancer cases diagnosed. With further analysis, the researchers discovered that decline in incidence was far greater in ER-positive breast cancer (8%) compared to ER-negative breast cancer (4%). And when they looked at women 50-69 years old, the decline in ER-positive cancer was 12%, compared to 4%t in ER-negative breast cancers. After adjusting for age, the researchers concluded that there was an absolute decline of about 14,000 fewer women diagnosed with breast cancer in 2003 than in 2002.

Again, correlation does not equal causation. It may, but it may not. And, as Dr. Charles pointed out, there are other confounding factors. One that I’d like to mention is the increasing use of raloxifene (Evista) for the prevention of osteoporosis. The STAR trial showed Evista to decrease the risk of breast cancer in high risk patients, and the CORE trial showed a 59% decrease. The number of women taking Evista is not large enough to account for the drop, but if the 2003 numbers were a fluke and the 2004 and 2005 numbers level off, rather than continuing to decline, I would begin to wonder if this is one of the major factors responsible for the decline, an effect augmented by the plummeting of HRT use that occurred after 2002.

In the end, practicing EBM is not straightforward, contrary to the caricature of EBM that alties often paint, of physicians slavishly devoted to nothing but double-blind, placebo-controlled trials. Back in the 1980’s and 1990’s, there were no such high quality studies, and we were forced to synthesize the existing evidence as well as we could. Also, physicians do not act in a vacuum; we influence society, and society influences us. There was lots of hype about how great hormones are, a hype that continues even after the WHI trial in the form of self-proclaimed “experts” like Suzanne Somers advocating huge doses of “bioidentical hormones,” which, in the magic that is altie world, are claimed on the basis of little or no evidence to be able to preserve a woman’s youth and health while at the same time supposedly avoiding the harmful consequences of HRT of the type used in the WHI trial. (Never mind that prolonged exposure to endogenous estrogen, the very same hormones whose benefits the “bioidentical” hormone mavens tout, in the form of early menarche, late menopause, and nulliparity has been recognized as a risk factor for breast cancer for decades.) Then, in 2002, when a very large and well-designed randomized, placebo-controlled trial showed that the risks of the extended use of HRT outweighed the potential benefits, we were forced by the evidence to change. Disconcerting, yes, and not everyone was convinced right away. The transition from widespread use of HRT to only selective use was messy and disconcerting to both patients and their doctors, and the WHI and M.D. Anderson studies raise as many questions as they answer. But I’ll take this over certain kinds of “alternative” medicine any day, where in the concepts and treatments haven’t changed for decades, if not centuries, if not millennia. The key difference between scientific medicine and so-called “alternative” medicine is that scientific medicine eventually corrects its mistakes and improves the efficacy and safety of its therapies using the scientific method. The process is all too often messy and slower than we would like, but at least it has such a process.

Alternative medicine does not.