On “doing your own research”

“I’ve done my own research.”

“Do your own research.”

How many times have you heard various antivaxxers, cranks, advocates of pseudoscience, and conspiracy theorists repeat these phrases, or variants thereof? In medicine, advocates of what I like to call pseudomedicine—a category that encompasses antivaxxers, COVID-19 denialists and conspiracy theorists, cancer quacks, and all manner of other quacks—are particularly prone to claim that they’ve “done their research” about, for instance, vaccines, and that’s why they think the MMR vaccine causes autism and that vaccines cause sudden infant death syndrome (SIDS), autoimmune diseases, and all manner of other diseases (and, oh, by the way, their “research” has told them that vaccines don’t protect against disease and “natural immunity is better,” too).

Of course, “doing one’s own research” and then “making up one’s own mind” makes perfect sense when it comes to, for example, choosing a place to live, buying a car, picking a smartphone, and any of a number of decisions we make in our day-to-day lives, although it should be noted that even those decisions are not necessarily so straightforward or easy to research. When it comes to science, the fact is that the vast majority of us are not capable of “doing our own research”. I started thinking about this question again with respect to science-based medicine (and science in general), thanks to an article that bubbled up a bit over a week ago on social media by former ScienceBlogs blog bud from back in the day Ethan Siegel (who now writes for Forbes), entitled “You Must Not ‘Do Your Own Research’ When It Comes To Science“.

On “doing your own research”

Before I go on, let me just take a moment to observe that the title of his article is absolutely atrocious and doesn’t quite align with what the article actually says. Nowhere does Siegel tell readers that they “must not do their own research.” It’s a title that seems designed to be clickbaity and to inflame, rather than educate. I’m going to guess that Siegel probably didn’t come up with this title, but rather some Forbes editor looking to make the article sound as controversial as possible. Mission accomplished, as you will see, but at the cost of misunderstanding what the article is about. Moreover, I don’t totally agree with everything in the article (that would be a rarity about any article!), but overall it’s a good summation of the perils and pitfalls of lay people “doing their own research” about medicine and science and why lay people, for the most part, are incapable of correctly “doing their own research” on matters of science.

There are few skeptics, if any, who would disagree with Siegel’s introduction, for instance:

“Research both sides and make up your own mind.” It’s simple, straightforward, common sense advice. And when it comes to issues like vaccinations, climate change, and the novel coronavirus SARS-CoV-2, it can be dangerous, destructive, and even deadly. The techniques that most of us use to navigate most of our decisions in life — gathering information, evaluating it based on what we know, and choosing a course of action — can lead to spectacular failures when it comes to a scientific matter.

The reason is simple: most of us, even those of us who are scientists ourselves, lack the relevant scientific expertise needed to adequately evaluate that research on our own. In our own fields, we are aware of the full suite of data, of how those puzzle pieces fit together, and what the frontiers of our knowledge is. When laypersons espouse opinions on those matters, it’s immediately clear to us where the gaps in their understanding are and where they’ve misled themselves in their reasoning. When they take up the arguments of a contrarian scientist, we recognize what they’re overlooking, misinterpreting, or omitting. Unless we start valuing the actual expertise that legitimate experts have spent lifetimes developing, “doing our own research” could lead to immeasurable, unnecessary suffering.

Anyone with expertise who dips their toes into deconstructing pseudoscientific or crank claims regarding issues about which they are deeply knowledgeable will instantly realize that one of the hallmarks of pseudoscience and conspiracy theories is the cherry picking of studies, data, facts, and observations. The reason is simple. The people espousing pseudoscience tend not to look at the evidence base and then make their conclusions fit the evidence. Rather, they start with a conclusion and then go looking for facts, observations, and studies that support that conclusion, ignoring context and, often, uncertainty. It’s known as motivated reasoning, in which a bias towards a conclusion that conforms to what a person already believes leads that person to overvalue information that supports that belief and undervalue disconfirmatory information.

We all engage in motivated reasoning to one degree or another; it’s human nature. One of the main differences between most people and scientists and skeptics is that scientists and skeptics know motivated reasoning exists and make an active effort not to engage in it. It’s not easy, either, as evidenced by the lack of difficulty I have in thinking of scientists (and, yes, some skeptics) who have come to conclusions that appear to be based more in motivated reasoning than on an actual evaluation of the state of existing evidence. It’s easy to name the scientists who have let motivated reasoning guide them down the path of pseudoscience and quackery. All I have to do is to recount the list of physicians and scientists who have gone antivaccine (or become sympathetic to antivaccine viewpoints): Andrew Wakefield, Bob Sears, Kelly Brogan, Paul Thomas, James Lyons-Weiler, Christopher Shaw, Christopher Exley, Judy Mikovits, Theresa Deisher, and many others.

Siegel describes this process rather well:

There’s an old saying that I’ve grown quite fond of recently: you can’t reason someone out of a position they didn’t reason themselves into. When most of us “research” an issue, what we are actually doing is:

• formulating an initial opinion the first time we hear about something,
• evaluating everything we encounter after that through that lens of our gut instinct,
• finding reasons to think positively about the portions of the narrative that support or justify our initial opinion,
• and finding reasons to discount or otherwise dismiss the portions that detract from it.

Of course, that’s not what we think we’re doing. We think of ourselves as the heroes of our stories: cutting through misinformation and digging up the real truth on the matter. We think that, just by applying our brainpower and our critical reasoning skills, we can discern whose expert opinions are trustworthy and responsible. We think that we can see through who’s a charlatan and a fraud, and we can tell what’s safe and effective from what’s dangerous and ineffective.

Precisely. Many, if not most, of us have a far higher opinion of our critical thinking abilities than is actually warranted. (Some might argue that this description might also apply to me, and so it might, at least for some topics. One always has to take such a possibility under consideration and keep it in the back of one’s mind.) Many, if not most, of us also have a far higher opinion of our knowledge base than is actually warranted. Many, if not most, of us have a far greater confidence in our ability to spot grifters, cranks, pseudoscientists, and charlatans than we, in fact, possess, just as Siegel observes.

The trappings of expertise

This is, of course, exactly why advocates of quackery and pseudoscience so blatantly embrace the trappings of expertise. That’s why Andrew Wakefield, for instance, touts his credentials as a physician. (Never mind that the British General Medical Council yanked his medical license a decade ago) It’s why the deniers of the severity of the COVID-19 pandemic and those who embrace hydroxychloroquine as a COVID-19 cure went full astroturf and produced a video by “America’s Frontline Doctors” claiming that hydroxychloroquine cures COVID-19 and that, because of that, we should reopen schools without concern that doing so might facilitate the spread of coronavirus and not bother with all those pesky mitigation measures like wearing masks and engaging in social distancing. After all, if there’s a cure, why bother with all the difficult methods of slowing the spread of the virus? Unfortunately, the viral video and was viewed tens of millions of times before social media companies removed it from their platforms. Worse, people believed it.

Taking a starring role in this rogue’s gallery of quacks was a Houston doctor and religious minister named Dr. Stella Immanuel:

A Houston doctor who praises hydroxychloroquine and says that face masks aren’t necessary to stop transmission of the highly contagious coronavirus has become a star on the right-wing internet, garnering tens of millions of views on Facebook on Monday alone. Donald Trump Jr. declared the video of Stella Immanuel a “must watch,” while Donald Trump himself retweeted the video.

Before Trump and his supporters embrace Immanuel’s medical expertise, though, they should consider other medical claims Immanuel has made—including those about alien DNA and the physical effects of having sex with witches and demons in your dreams.

Immanuel, a pediatrician and a religious minister, has a history of making bizarre claims about medical topics and other issues. She has often claimed that gynecological problems like cysts and endometriosis are in fact caused by people having sex in their dreams with demons and witches.

She alleges alien DNA is currently used in medical treatments, and that scientists are cooking up a vaccine to prevent people from being religious. And, despite appearing in Washington, D.C. to lobby Congress on Monday, she has said that the government is run in part not by humans but by “reptilians” and other aliens.

In addition, there were several other doctors, including an ophthalmologist and founder of a cryptocurrency company named Dr. James Todaro; a emergency medicine doctor named Dr. Simone Gold known for promoting hydroxychloroquine; another emergency medicine doctor named Dr. Dan Erickson, co-owner of a chain of urgent care centers in Bakersfield, CA who made a name for splash in April with a poorly conceived and carried out “study” full of bad epidemiology that concluded that COVID-19 was far more widespread than thought and that the infection fatality rate was far lower than thought; Dr. Joseph Ladapo, an associate professor of medicine at UCLA and an ardent proponent for the “open up the economy” message who has downplayed the lethality of COVID-19; and Dr. Robert C. Hamilton, a Santa Monica pediatrician who argued for reopening the schools in the fall based on cherry picked studies and information and also made his political orientation very clear with a gratuitous swipe at teachers’ unions in his part of the video (Gizmodo later reported on a letter he wrote in the 1990s expressing anti-homosexual bigotry); and Dr. Richard Urso, an ophthalmologist who has been touting hydroxychloroquine for COVID-19 and claims never to have seen a patient with a heart issue attributable to the drug.

Unsurprisingly, MedPage Today reported that there’s no evidence that any of the doctors in the video got near the “COVID front lines”. Over several days last week, the claims of these doctors were thoroughly refuted, in particular the claim that hydroxychloroquine is, in essence, a cure for COVID-19 that eliminates the need for social distancing and masks to prevent the spread of coronavirus. In fact, more and more, the evidence is trending strongly in the direction that hydroxychloroquine doesn’t work.

What this incident demonstrated very clearly is that, not only are most people unable to evaluate medical claims, but there are a lot of actual physicians out there who are similarly unable to evaluate certain medical claims. It is very telling that these physicians consisted of ophthalmologists, emergency medicine doctors, and doctors who don’t actually have any expertise in virology, epidemiology, clinical trials, or the treatment of COVID-19. This is not a knock on ophthalmologists, emergency medicine doctors, etc., but rather an observation I use to make a point. Even experts in medicine frequently go astray when they wander outside of their field of expertise, particularly when the belief that they want to accept is linked somehow with their sense of self. In this case, it was the politics. These physicians all support President Trump, and President Trump has promoted hydroxychloroquine, downplayed the seriousness of COVID-19, and strongly pushed for the schools reopening, even when there is still doubt about whether it is safe to do so in many areas where the virus is still running rampant.

Basically, like any lay person, when faced with beliefs that they wanted to embrace, “America’s Frontline Doctors” engaged in motivated reasoning and sought out observations, evidence, and cherry-picked studies to give them a reason to support the belief, regardless of whether science actually did support the belief or not. Worse, they very intentionally used their status as physicians to promote those beliefs and persuade lay people to believe them, too. It worked to some extent, as well, although it would have worked a lot better if “America’s Frontline Doctors” had left Dr. Immanuel out of their membership. Her history of claims involving alien DNA and that gynecological problems like cysts and endometriosis are in fact caused by people having sex in their dreams with demons and witches led the hashtag #DemonSperm to trend on Twitter and provided an easy approach to debunking and discrediting her and, thus, the entire group. (It didn’t help that she called doctors doubting hydroxychloroquine fake doctors and the negative studies of the drug “fake studies,” all while accusing those same doctors of being like the “good Germans, the good Nazis” who “watched Jews get killed” and didn’t speak up.) Indeed, it wasn’t long before Dr. Hamilton sent out an open letter to the parents of his patients disavowing Dr. Immanuel’s hydroxychloroquine claims and the claims of other doctors that facemasks are of “no value” in combatting the spread of COVID-19.

Not “one must not do,” but “one must be humble”

Getting back to whether one “must not” do one’s own research, Siegel includes examples other science whose conclusions are not controversial among scientists but are ideologically or politically contentious. These include water fluoridation, human-caused climate change, vaccines, and many of the conclusions regarding COVID-19, correctly noting that, when it comes to evaluating most scientific claims:

Except, for almost all of us, we can’t. Even those of us with excellent critical thinking skills and lots of experience trying to dig up the truth behind a variety of claims are lacking one important asset: the scientific expertise necessary to understand any finds or claims in the context of the full state of knowledge of your field. It’s part of why scientific consensus is so remarkably valuable: it only exists when the overwhelming majority of qualified professionals all hold the same consistent professional opinion. It truly is one of the most important and valuable types of expertise that humanity has ever developed.

But only if we listen to it. It’s absolutely foolish to think that you, a non-expert who lacks the very scientific expertise necessary to evaluate the claims of experts, are going to do a better job than the actual, bona fide experts of separating truth from fiction or fraud. When we “do the research for ourselves,” we almost always wind up digging in deeper to our own knee-jerk positions, rather than deferring to the professional opinions of the consensus of experts.

This is largely true, but I do have a little quibble that I’ll get to in a moment.

First, though, in terms of “doing your own research”:

If you “do your own research,” you can no doubt find innumerable websites, social media accounts, and even a handful of medical professionals who are sharing opinions that confirm whatever your preconceived notions about COVID-19 are. However, do not fool yourself: you are not doing research. You are seeking information to confirm your own biases and discredit any contrary opinions. Each time you do this, you exemplify the problem of anti-science bias that Dr. Fauci warned about in June:

“If you go by the evidence and by the data, you’re speaking the truth and it’s amazing sometimes, the denial there is. It’s the same thing that gets people who are anti-vaxxers – who don’t want people to get vaccinated, even though the data clearly indicate the safety of vaccines. That’s really a problem.”

That is, of course, the problem today. There is a veritable tsunami of misinformation about science out there on social (and, truth be told, old) media. Some of it is intentional disinformation (e.g., antivaccine misinformation to sell products to treat autism or denial of human-caused climate change in order to protect the fossil fuel industry); much of it is ideological (e.g., the promotion of hydroxychloroquine for COVID-19 by Trump supporters); and some of it is religious (e.g., the denial of evolution by natural selection as the major driving force producing the diversity of life). All of it appeals to different people for one reason for another, and all of it can be justified by its believers using motivated reasoning.

There once was a time when it was possible for people without formal education in science to make observations about the universe and formulate them into laws and hypotheses that characterize reality. That time ended a long time. The reason is that science builds on what was discovered. The more it builds, the more background information there is that has to be mastered in order to be able to make useful contributions. Although there can be lots of controversy in science, certain fundamental things are agreed upon because overwhelming evidence has led scientists to provisionally accept them as correct. For instance, you can’t suddenly posit a “theory” that says that atoms aren’t made up of protons, neutrons, and electrons, because there is a massive body of evidence that has led to a scientific consensus that they are, in fact, made up of such particles, whatever scientists choose to name them. At least, you can’t do it and have scientists take you seriously unless you can produce evidence that is at least compelling enough to call such well-established science into doubt. Cranks don’t acknowledge this and, through arrogance, think that they alone are able to see what all of science isn’t. As a result, they tend to be upset that science doesn’t recognize their apparent genius.

Many years ago, I once examined the contention that a “real skeptic always sides with the scientific consensus.” As I noted at the time, in matters of science it is undoubtedly true that the scientific consensus is always the best place to start when evaluating unfamiliar issues. While it is certainly possible that a given scientific consensus regarding an issue can be wrong in almost any area, it nonetheless almost always represents the best current scientific understanding. It is also correct that legitimate authority matters. I emphasize the word “legitimate” because in pseudoscience arguments from authority are common, but rarely is the authority relevant to the point being argued. Often it’s not even legitimate, as in when anti-vaccine activists point to Andrew Wakefield’s work as justification for their claims that vaccines cause autism and other conditions. I also noted that not all scientific consensuses are created equal because, in different fields the strength of scientific consensus can vary quite markedly depending upon the topic or even the subtopic within the topic. For example, the scientific consensus supporting the theory of evolution, particularly common descent, is exceedingly strong. It’s one of the strongest of all scientific consensuses, arguably the strongest. Similarly, the consensus that natural selection is a major driving force behind evolution is very nearly as strong. However, as the discussion devolves into more detailed areas, inevitably the consensus weakens. Eventually, subsidiary areas of a discipline are reached where the consensus is weak or where there is no consensus, such as what the function of “junk DNA” is, whether it is subject to natural selection, and if so how much. (Real evolutionary biologists could probably come up with a better example.) These sorts of questions are often at the cutting edge of scientific knowledge, and it is not always easy to recognize what they are. It is also these issues at the edge of our knowledge that are attacked as proxies for the much more strongly supported core theory.

This brings me back to COVID-19. What makes scientific conclusions about COVID-19 somewhat different than conclusions about vaccines is that the pandemic is new, having only been going on since the disease and virus were first recognized in China in late 2019, and the science is rapidly evolving. That makes it a particularly ripe area for cranks to promote bad science and pseudoscience, particularly given that they can easily invoke the “science was wrong before” trope in real time as new findings come in. However, there are several conclusions that are now pretty firm, including that masks work to slow the spread of coronavirus; that the virus spreads through respiratory droplets, particularly in enclosed spaces; that social distancing works. To that I add that hydroxychloroquine is almost certainly ineffective against COVID-19. It’s possible to challenge these conclusions, but if you do so, you’d better have strong evidence.

In the end, the way to judge claims that go against the current scientific consensus boils down at least as much to tactics and how evidence is used to support such contrarian arguments. Scientific skepticism looks at the totality of evidence and evaluates each piece of it for its quality. In contrast, cranks are very selective about the data they choose to present, often vastly overselling its quality and vastly exaggerating flaws in current theory, in turn vastly overestimating their own knowledge of a subject and underestimating that of experts. In medicine in particular, denialists frequently emphasize anecdotes over epidemiology, clinical trials, and science. They also tend to leap to confuse correlation with causation. Similarly, crankery, denialism, pseudoskepticism (or whatever you want to call it) tends, either intentionally through ideology or unintentionally through an ignorance of the scientific method, to conflate and/or confuse emotional, nonscientific, and/or ideological arguments with scientific arguments. This is not to say that scientists and skeptics and supporters of SBM are free from their own biases, whether ideological or simply a desired result that they hope to find. Far from it. However, skepticism means applying the scientific method to claims, and whatever its faults, the scientific method is the best method thus far devised to minimize these biases.

As scientists, the reason we use the scientific method is not because we consider ourselves superior to the cranks, but rather because we recognize that we are human too and thus just as prone to falling into the same traps as they. As Richard Feynman once famously said, “The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.” The scientific method is, above all, a methodology by which scientists try to avoid fooling themselves. Skeptics cross the line dividing skepticism and denialism and quacks the line between science and quackery when they forget that. Doubting a scientific consensus is not in and of itself the mark of the crank. It’s how and why that skepticism exists that distinguishes crankery from genuine scientific skepticism.

The problem with “doing your own research” is that rarely does a lay person (or even a physician or scientist venturing too far outside of his area of expertise) have the background knowledge and skillset to be confident of avoiding crossing that line, whether intentionally or not. It’s not so much that you “must not do your own research.” It’s that you really need to understand that you probably can’t “do your own research” and that the conclusions you reach “doing your own research” are highly likely to be more in line with your prior beliefs than scientifically correct.

As Siegel puts it:

But that requires a kind of transformation within yourself. It means that you need to be humble, and admit that you, yourself, lack the necessary expertise to evaluate the science before you. It means that you need to be brave enough to turn to the consensus of scientific experts and ask, legitimately, what we know at the present stage. And it means you need to be open-minded enough to understand that your preconceptions are quite likely to be wrong in some, many, or possibly even all ways.

Humility! What a concept!