When testimonials are used in medical advertising

(NOTE ADDED 12/7/2010: Kim Tinkham has died of what was almost certainly metastatic breast cancer.)

If there’s been one theme running through this blog every since the very beginning, it’s the unreliability of testimonials as “evidence” for the success of a cancer treatment. Indeed, if you go back to one of the very first “Orac-length” posts was about that very topic. Indeed, almost exactly five years ago, I analyzed a common type of testimonial for “alternative” cancer therapies and explained why it sounds so convincing to lay people who don’t understand cancer biology and treatment. It’s a theme I’ve returned to time and time again, be it analyzing Kim Tinkham’s testimonial in which she eschewed science-based cancer therapy for Robert O. Young’s acid-base quackery, deconstructing Suzanne Somers’ accounts of her breast cancer therapy and her “whole body cancer” scare, analyzing Abraham Cherrix’s case, or even discussing more complicated cases.

The lesson to be learned from all of these stories is that single cases do not constitute a reliable basis for concluding that a cancer therapy works. Even a series of cases is hard to draw much of a conclusion from, given the biological variability in cancer and the virtually inevitable presence of outliers. Because they cannot produce strong evidence for the efficacy of their treatments based on basic science and randomized clinical trials, purveyors of woo frequently substitute seemingly dramatic cases that appear to support their claims that their preferred quackery can cure cancer. Virtually always, the problem is that they don’t give you the denominator. You have no idea how many patients the “alt-med” pracitioner has treated before he came up with this case or the real success rate, and you’ll have no way of knowing. Certainly the practitioner won’t tell you. He or she might not even know, because he or she might not even actually keep good records. The problem is, even a cancer as deadly as pancreatic cancer produces outliers, patients who survive far longer than the average, regardless of treatment. This is not just an issue for quacks; it’s an issue for practitioners of science-based medicine as well. It’s the very reason why we need randomized clinical trials with careful design and statistics to tell whether a treatment truly produces a prolongation of survival or even cures. Indeed, practitioners of science-based medicine should know better.

But they don’t, as an article in the New York Times over the weekend shows:

A print advertisement for prostate cancer surgery at Mount Sinai Medical Center in Manhattan is typical of the way many elite research and teaching hospitals sell hope to the public.

“Our newest prostate specialist, Dr. David Samadi, has pioneered a minimally invasive approach that allows him to retain the highest cancer cure rates with the lowest risk of side effects,” says the ad.

Highest cure rates. Lowest risk. What evidence does the medical center have to back up such superlatives?

The ad’s claims are based on the successful results of Dr. Samadi’s operations and testimonials from his patients, said Jane Zimmerman, Mount Sinai’s chief marketing officer.

In medical science, such anecdotal data would not be considered statistically valid. But ads for nonprofit medical centers are not held to scientific standards of evidence.

There’s an aphorism in advertising: Testimonials sell. Indeed, as Brock Henderson put it:

Testimonials add credibility about your expertise and expertise, and are worth their weight in gold. Don’t hide those endorsements from potential or even existing customers; plaster them on every ad, in every commercial, in every mailing, and anywhere else you can. When customers speak for you it is much more powerful than when you say the same thing.

Testimonials are all well and good when it comes to selling cars, TVs, furniture, or all manner of other consumer goods. They are also not such a bad thing when it comes to selling services, like accounting, lawyers, plumbers, electricians, and a lot of other services. One might argue that testimonials can mislead even in these areas, but if there’s one area where testimonials really mislead it’s in medicine. Moreover, if there’s one area of medicine where testimonials can really mislead, it’s cancer. As I said, there are always outliers, and using testimonials of the outliers tells us nothing about the overall success rates of a physician or hospital. This works both ways, too. All hospitals and doctors have outlier patients who did either much worse than would normally be expected or much better than would normally be expected. The testimonial of the former can prejudice those hearing it unfairly in a negative light, and the testimonial of the latter can paint an excessively rosy picture of how well a physician or hospital does.

Most cancer doctors know this at an instinctive level, and this knowledge sometimes directly conflicts with the marketing arm of cancer centers and cancer hospitals. Indeed, as proud as I am of my own cancer center, I must admit to being somewhat uncomfortable with one of its ads. Unfortunately, I really have no say in advertising and have on occasion pointed out my discomfort with at least one of them to my colleagues. I wish we used a different advertising technique, but seemingly “everyone does it” and mere peons like myself have little influence. In any case, as the NYT article points out, the use of ads containing testimonials by elite cancer centers is fraught with problems, including playing on the fear of cancer and the aforementioned likelihood of exaggerating how well the cancer center does. I remember when I used to live near New York City, close enough to get the Memorial Sloan-Kettering Cancer Center ads. They proclaimed that the first treatment a cancer patient gets is the best chance for a cure; so, of course, you should go to Memorial. I don’t recall the cancer center where I worked then using such ads, although, being a small cancer center, it didn’t have much of an advertising budget. It was rare to see TV and radio spots for it; sometimes we’d see billboards.

One of the key points in this article is the double standard between pharmaceutical advertising and cancer center advertising. It was pointed out again and again that if a pharmaceutical company made claims that a drug it manufactured could cure a type of cancer or that a patient using that drug would do much better than if he used a competing therapy it would have to be able to prove to the FDA and FTC that these claims are well supported by scientific and clinical evidence. Not so not-for-profit cancer centers, which can in essence say anything they want in ads. (I wish it weren’t so, but it is.) Moreover, although I have a hard time believing that some cancer centers don’t do better than others for different cancers given how some cancer centers have more expertise in different areas than others, there really aren’t any good data to document this, making claims difficult to back up. Exceptions exist for less common cancers or more complicated cancer presentations, where patients do tend to do better at elite cancer centers, but that doesn’t necessarily mean that patients with common cancers presenting in an uncomplicated fashion would necessarily do better there than in at a solid community hospital:

The problem with many ads is the implication that choosing a particular hospital could be the deciding factor in whether a cancer patient lives or dies, said Dr. H. Gilbert Welch, a medical professor at the Dartmouth Institute for Health Policy and Clinical Practice.

People with some more complicated cancers or rarer diseases like leukemia do tend to fare better at comprehensive cancer centers. But Dr. Welch and others worry that such ads could persuade people with localized cases of more common diseases like prostate cancer to travel long distances from their families at great expense to obtain treatment that may be as successful, or unsuccessful, as the treatment available much closer to home.

And this is what I tell patients for routine, uncomplicated cases of breast cancer if they ask me if they would do better at a cancer center or come in for a second opinion but aren’t sure if they should stick with their community doctor. What a woman with, for instance, an early stage cancer with negative axillary lymph nodes and nothing weird (for instance not a triple negative cancer) needs is a good surgeon, a good oncologist, and a good radiation oncologist with solid fundamentals. I know we have some of the best in the country, if not the world, at our cancer center, but there are also some excellent community cancer docs who are more than capable of handling competently less complicated cases.

As much as cancer centers tend to advertise their latest and greatest technology (often with little evidence that that latest and greatest technology actually results in better outcomes) may be a bit dicey from an ethical standpoint, using patient testimonials really risks skirting the line. Even though hospitals I’ve worked at, including my current one, have used them, I’ve never been comfortable with them. As I said, these cases are often outliers, and the NYT article points out an example of just that:

That recent print ad from Memorial Sloan-Kettering Cancer Center in Manhattan tells the story of Michelle Rogala, a patient with cervical cancer.

Ms. Rogala’s hospital in New Jersey could offer her only a hysterectomy, an operation that would have left her unable to have children. Instead, she went to Memorial Sloan-Kettering, where she entered a clinical trial that was studying less invasive surgery. Ms. Rogala now has a little girl named Maddie.

[…]

In a recent telephone interview, Ms. Rogala, a 37-year-old meeting planner in Monroe Township, N.J., said that hers had indeed been a special case.

She had early-stage cervical cancer, she said, making her eligible for a novel operation that has now become a standard treatment at the center. After her operation, doctors told her she would need fertility treatments to conceive. But she said she turned out to be one of the few patients in the study who did not need radiation — which can cause fertility problems. She later became pregnant without medical intervention.

Ms. Rogala said she wanted her ad to inspire women with cervical cancer to investigate fertility-sparing surgery. But if a woman has more advanced cervical cancer, she said, “unfortunately, it may not be an option.”

In other words, Mrs. Rogala is like the proverbial pancreatic cancer patient who lives far longer than expected. She’s an outlier. A testimonial. As much as it pains me to admit it, she’s no different than the alternative medicine testimonial in that her story tells very little about how the typical cancer patient could expect to do at MSKCC and even less about whether MSKCC does better at preserving the fertility of cervical cancer patients than any other quality cancer center in the area. Moreover, as the NYT article pointed out, if a for-profit diet center used a testimonial to advertise its services, the FTC would require a description of what a more typical paitent could expect. No such requirement applies to nonprofit cancer centers.

There’s only one area in the NYT that I strongly disagree with, and that’s the argument by Dr. Steven Woloshin that cancer centers shouldn’t advertise based on the science and clinical trials that they do. From my point of view, if there’s one area that NCI-designated comprehensive cancer centers can justifiably point to with pride to distinguish themselves from other hospitals and cancer centers, it’s their research. After all, research and improving cancer care are the raison d’être of these cancer centers. I admit that here I could be a bit biased, given that I’ve spent my entire post-residency career at NCI-designated comprehensive cancer centers, having trained or worked at three different ones. I will, however, agree that cancer centers have to be very careful not to say or imply that such trials will definitely save the lives of cancer patients, given that most clinical trials do fail. With that caveat, I see nothing wrong with pointing with pride to the cancer research and clinical trials going on at a cancer center, be it mine, MSKCC, M.D. Anderson, or my old alma mater the University of Michigan.

Unfortunately, the reality is that competition in many markets, including my own, is leading to hospitals and cancer centers doing more and more advertising for more and more diseases. In this advertising, there all too often exists an inherent conflict between a compelling message and avoiding testimonials. Statistics and description of studies and data would be great for accuracy, but they’d make for boring ads that no one would remember. Testimonials tend to be more compelling and “human,” but frequently are inherently misleading to one degree or another because the patients chosen for such ads are chosen more for their compelling stories with happy endings than for whether they represent the experience of a typical patient. You’ll almost never see a testimonial of a patient who, despite the best efforts of the medical team and hospital, died anyway. (The only medical institutions that might use such testimonials would be hospices.) What you need to remember every bit as much for ads for cancer centers as you do for “alternative medicine” cancer treatments is what I wrote nearly exactly five years ago:

Also remember that conventional medicine is not above misusing testimonials in advertisements. Treat them with the same degree of skepticism [as alt-med testimonials]. Look for the scientific and clinical evidence, not stories of great cures, regardless of the type of testimonial. If there is one principle I hope to impart here, it is that the claims of conventional medicine and alternative medicine should be treated the same and that they should be held to the same standard of scientific and clinical evidence. I do not differentiate between the two when considering evidence, nor should you.

Five years into this blogging thing, I find it a bit depressing that I keep having to refer back to these words again and again and again, be it for “alt-med” testimonials or now for testimonials used to attract patients to cancer centers. But refer back to them I do time and time again, and I fear that I’ll be referring back to them 30 years from now.

Maybe I’ll have a shorter version of the passage above inscribed on my tombstone.