It figures again.
I go a few days without Internet access again, and not only does Generation Rescue take out a full page antivaccination ad full of stupidity in USA Today, which I couldn’t resist opening both barrels on earlier, but a study’s lead senior author is someone I know (albeit not well) about three topics I’m very interested in: breast cancer, health information on the Internet, and so-called “complementary and alternative” medicine. Not surprisingly, in my absence blog stalwarts Abel Pharmboy and Steve Novella already beat me to it in fine form. You might ask if that would in any way inhibit me from adding my two cents.
I think you know the answer to that one.
The peer-reviewed study was entitled Commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer, and the senior author was Funda Meric-Bernstam, MD, a breast cancer surgeon in the Department of Surgical Oncology at M.D. Anderson Cancer Center. The question examined was one near and dear to my heart, specifically: Is there any way to identify websites containing bogus or dubious health information? As a medical skeptic and advocate of science- and evidence-based medicine, I can usually tell bullshit when I see it (if you’ll excuse the term) for most medical topics and very close to 100% of the time for breast cancer-related topics, but this study tried to identify more objective criteria than the “smell” test.
Basically, the authors used several commonly used search engines, such as Google, Alta Vista, or Yahoo!, querying them using several breast cancer-related terms. Examples include (not suprisingly) “breast cancer,” “breast cancer prognosis,” “breast lump,” “breast cysts,” “mammogram,” and numerous others. Links that showed up on the first page of each search (since most patients don’t go more than a few pages deep into the search pages) were recorded, and the websites evaluated for several measures of quality, as well as other criteria (URL, citing of authorship, date of last update, identification of authors’ credentials, etc.) that might be useful in determining whether a site is likely to contain accurate medical information about breast cancer or not. In addition, two reviewers examined each site for the accuracy of the information contained therein.
Perhaps the biggest surprise I found in this study is that overall only 5.2% of the statements examined on the websites examined was objectively inaccurate. Steve Novella may think that’s bad, but, quite frankly, it was a much lower number than I would have guessed before I read this study. Don’t get me wrong, there’s plenty of room for improvement with such a figure. However, it wouldn’t surprise me if 2% to 5% of statements in a typical breast cancer textbook were also inaccurate, although in that case it would probably be due mainly to how rapidly textbooks fall out of date.
So were there any criteria that were associated with a higher content of inaccurate health information about breast cancer? In a word: Not much.
In fact, only one criterion really stood out above some of the other factors that were more weakly associated with inaccuracy (the topics of psychology, breast reconstruction, risk factors for breast cancer, and diet or nutrition, none of which produced a relative risk of greater than 3.2) as associated with inaccurate health information. Can you guess what it is? Yes, indeed, it’s “complementary and alternative” medicine content. As the authors put it:
We found that 5 of the 15 (33%) websites that displayed information pertaining to CAM contained inaccurate statements. The odds of inaccuracy for webpages having CAM among their topics were approximately 15.6 times those of sites not having the complementary medicine topic.
Only 33%? Personally, I’m surprised. However, I can’t help but note that this figure did not include information about the CAM therapies themselves. This is only an estimate of incorrect statements about breast health and breast cancer. Moreover, I tend to doubt that Dr. Meric-Bernstam dug into the parts of the Internet that I routinely subject myself to (such as websites that lead patients to abandon conventional therapy), and admittedly that might have contributed to a jaded view on my part. I’m half-tempted to send Dr. Meric-Bernstam’s team over to Natural News or Dr. Mercola, but I wouldn’t want to subject any of my colleagues to this stuff. On the other hand, too many of my colleagues are all too oblivious of what is really out there on the Internet and what patients are finding in their searches, which is why I wish Dr. Meric-Bernstam’s study had looked at more websites. On the other hand, there is much to be reassured about in that breast cancer websites tended to be more accurate than other medical websites, some of which, as mentioned by the authors, can approach 37% inaccuracy.
Is this result of this study valid? Personally, I think it almost certainly underestimates the problem, although I’d need more objective analysis to be sure. Websites devoted to CAM tend to be selling products. Not only that, but, at least in my anecdotal experience, the quality of evidence and information on CAM websites is, by and large, atrocious (for example, this one). Finally, one thing that disturbed me about this study is that the websites studied were accessed in 2004. The web is a rapidly changing environment, and even since then there has been a proliferation of “CAM” and “conventional” medical websites. Moreover, the search engines have all been continually tweaking their algorithms. True, the authors acknowledged this problem, but I have to wonder why it took three and a half years from identification of websites to publication. That’s a long time. Worse, considering the claims of CAM itself, there’s an awful lot of misinformation out there. If you do searches for any CAM modality (homeopathy, etc., you will almost inevitably find that the vast majority of the initial hits will be credulous sites promoting CAM. The same is true of other health myths, for example, vaccines and autism; just Google “vaccines autism” if you don’t believe me. The good information is drowned out by vaccine pseudoscience.
One thing that is the most true about medical information is that it’s fairly easy to answer concrete questions but much more difficult to answer questions that require nuance and in particular whether a specific piece of information applies the patient looking at it. As the authors put it:
Our finding that breast cancer information is generally accurate does not mean that consumers are not misled by online information. Information is scattered across webpages and websites. Users find it relatively easy to answer specific questions (eg, What is the 5-year disease-free survival rate of patients with stage I breast cancer?), but users find it relatively difficult to answer questions that require comprehensive knowledge (eg, What are the indications and contraindications of trastuzumab?).[27] Therefore, even a dedicated searcher is likely to retrieve incomplete information on any particular topic. In other words, the Internet may tell the truth but not the whole truth.
In other words, health information on the Internet tends to be wide but not deep, which is one reason why scorn is rightly heaped on people with a degree from “Google University” (like Jenny McCarthy) who think they know enough from Google to make authoritative pronouncements on health issues that conflict with current medical understanding. When used properly, it can be a huge help for doctors who can’t spend the time they used to explaining things to patients and for patients who want to do their own research. However, it’s not arrogance to suggest that putting that information in context still, even in the age of Google and the Internet, requires the guidance of someone steeped in the knowledge of a topic who can assist in putting all that raw information into context.
In other words, believe it or not, we still need doctors.
REFERENCE (E-pub ahead of print):
Bernstam, E.V., Walji, M.F., Sagaram, S., Sagaram, D., Johnson, C.W., Meric-Bernstam, F. (2008). Commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer. Cancer DOI: 10.1002/cncr.23308
