In 2007, I wrote a series of posts about what I found to be a fascinating yet at the same time disturbing phenomenon, specifically self-experimentation by cancer patients using an as yet unapproved drug called dichloroacetate. If you’ll recall, DCA is a small molecule drug that was used to treat congenital lactic acidosis in children through its inhibition of the enzyme pyruvate dehydrogenase kinase. This inhibition shifts the metabolism of glucose towards oxidative metabolism in the mitochondria and away from glycolysis, the product of which is lactic acid. In January 2007, Dr. Evangelos Michelakis of the University of Alberta had the clever idea of using DCA to target the Warburg effect in tumors and published an article in Cancer Cell describing preclinical experiments in rats in which DCA resulted in marked shrinkage of multiple tumor types. The Warburg effect was first described in 1928 by Otto Warburg and refers to the tendency of many tumors to rely on glycolysis for their energy supply rather than oxidative phosphorylation, even in the presence of oxygen, which in the case of normal cells favors oxidative metabolism.
Because DCA is a small molecule and relatively easy to synthesize, however, a pesticide salesman named Jim Tassano decided that he would sell “pet DCA” to treat cancer in pets. Tassano fooled no one, of course; the real purpose was to take advantage of desperate cancer patients, some of whom flocked to his BuyDCA.com website to purchase what he claimed to be pharmaceutical grade DCA. It also led to a most disturbing phenomenon, namely self-experimentation by desperate cancer patients with DCA and discussion among patients on online forums, which was even likened to “clinical trials,” complete with reports of success based on magical thinking more than anything else. This observation led me to take interest in the more general question of the “right” of patient access to experimental drugs versus the negative effect that such unrestricted access would have on clinical trials. Last year, the full D.C. Circuit Court of Appeals reversed an earlier decision by its own three-judge panel and ruled 8-2 against a patient’s right to have access to any experimental therapy. I thought things were fairly settled then, but apparently not, as this recent story points out:
Until last year, Alan Felzer was an energetic engineering professor who took the stairs to his classes two steps at a time. Now the 64-year-old grandfather sits strapped to a wheelchair, able to move little but his left hand, his voice a near-whisper.
Felzer suffers from ALS, also known as Lou Gehrig’s disease. The fatal neurological disorder steals the body’s ability to move, speak and ultimately to breathe. But rather than succumb to despair along with his illness, Felzer turned to the Web to become his own medical researcher — and his own guinea pig.
Dozens of ALS patients are testing treatments on their own without waiting on the slow pace of medical research. They are part of an emerging group of patients willing to share intimate health details on the Web in hopes of making their own medical discoveries.
ALS is a horrible disease. Indeed, it’s arguably more horrible than most forms of cancer, robbing, as it does, people of the ability to control their bodies. It’s fatal, and before it kills it renders its victims helplessly dependent on others before it takes away their ability to breathe. Although a minority of ALS victims can live a long time with the disease (Steven Hawking, for instance), the majority succumb within five years of the onset of symptoms. If ever there was a disease that understandably spurs people to take desperate measures to slow or arrest its course, ALS is it. This group testing lithium for ALS is no different than the desperate cancer patients trying out DCA, and it’s subject to the same shortcomings and opportunities for bias and self-deception that were so apparent on Jim Tassano’s DCA discussion boards.
Just as Tassano sparked “wild experimentation” of patients using DCA to treat cancer with little or no medical supervision based on Michelakis’ single study using rat tumor models, patient experimentation with lithium to treat ALS appears to have been sparked by an Italian study that reported a significant slowing of the progression of ALS. This study looked at both a mouse model and reported the results of a small randomized trial of lithium plus riluzole (riluzole is the only drug approved by the FDA to treat ALS) versus riluzole alone. As a result of this study, ALS patients are taking lithium off-label:
Felzer began taking lithium in January, and his scientifically minded family reached out to other ALS patients. “All those people are taking it anyway,” said Alan Felzer, whose smile remains bright and his gaze sharp even as the rest of his body fails him. “So it only made sense to keep track of what was happening.”
The task of leading the ALS-lithium project fell to Felzer’s daughter, Karen, a U.S. Geological Survey earthquake researcher. Her partner in the effort was Humberto Macedo, a 42-year-old computer systems analyst, father of six and ALS patient in Brasilia, Brazil.
The study grew naturally out of the strong reliance of ALS patients on one another for information, Macedo said.
Working online, Karen Felzer and Macedo recruited nearly 200 patients worldwide to take a specific lithium dosage and answer standard surveys to gauge their symptoms. They began running their study through a Web site called PatientsLikeMe.com, using it to attract volunteers and track their progress.
Sound familiar? This is simply a somewhat more organized method of doing the same thing that patients did on Jim Tassano’s discussion boards. Heck, Tassano even tried to set up a “clinical research” protocol that was a travesty. The discussion boards on PatientsLikeMe.com are not quite so bad (it would be hard for anyone’s studies to be more poorly designed or self-deluding), but I can’t help but think that the designers of this project are fooling themselves if they think an unblinded, patient-driven clinical trial will accomplish what they think it will, although they do exhibit appropriate caution:
History also teaches us that patients sharing stories with each other will not answer the question alone. Chinese stem cells, herbal supplements, nutraceuticals — all have been discussed extensively on the internet with some claiming cures and some describing great harm; yet we have no definitive answer.
Except that in many cases we do have definitive answers, and even where we don’t scientific plausibility (specifically the lack of scientific plausibility) argues strongly against the utility many of these therapies. However well-intentioned Humberto Macedo and Karen Felzer, the organizers of this effort, may be, they clearly don’t understand why unrandomized, uncontrolled trials are so prone to bias and false positives. After all, what they are doing is nothing more than the collection of anecdotes, and, as I like to say, the plural of “anecdote” is not data. Granted, that’s a bit of an overstatement in some cases, but not by much. The reason is that this survey doesn’t even rise to the level of a case series, given that it relies only on Internet reporting rather than the actual direct assessment of the patient’s condition by a physician or other health care professional:
Together, with all the patients involved, we will run the first real-time, real-world, open and non-blinded, patient-driven trial. We believe we will have the power, within months, to begin answering the question of how much lithium modifies the progression of ALS. Unlike a blind placebo control trial, we are watching the use of this drug in the real world, and because of the number of patients and our system’s sophisticated data modeling, we can determine the significance of each reported change in each patient as he/she deviates from his/her predicted course. There are many risks to our approach, patient optimism, the placebo effect, uncertain quality, and many other variables will compromise our data. Despite these, and many other challenges, we remain committed to solving this problem.
The problem, unfortunately, is that this “problem,” as it is described, probably cannot be solved because it is inherent in the very design of this study, which is not randomized, not controlled, and not blinded. There are no controls, only in essence historical controls (i.e., the “predicted course”). What’s even more puzzling is that the organizers of this trial even seem to recognize these problems, acknowledging that the placebo effect, patient optimism, and many other variables may compromise the data. Yet they nonetheless express optimism that their “sophisticated data modeling” will overcome these problems. How? They don’t answer. They don’t seem to realize, either, that what they are doing is not new. There’s a reason why randomized, double-blind clinical trials using either placebo or the standard of care as the control are viewed as the strongest form of clinical evidence for the efficacy of a treatment, and it’s because it became appreciated that non-blinded and non-randomized trials are prone to all sorts of biases. They just don’t give answers reliably unless the superiority of one therapy is so much greater than that of the control that it’s undeniable and detectable after just a few patients treated.
So what’s the harm? This is, of course, not an unreasonable question for patients faced with a fatal disease like metastatic cancer or ALS. As hard as it is to believe, even for patients with ALS it is possible to make things worse. Lithium, for instance, is not a benign drug. It can cause neurological complications and diabetes insipidus. It also requires monitoring to achieve therapeutic serum lithium levels. In the case of DCA for cancer, the drug could cause neuropathy and other complications. Overall, when taken as a whole, it is far more likely that patients will be harmed by taking experimental or off-label drugs than significantly helped. Worse, in the “real world” case of most drugs, effects that are not as dramatic as an antibiotic curing a urinary tract infection, it’s difficult, if not impossible, using methodology like that of PatientsLikeMe.com to detect an actual therapeutic effect:
Heywood too hoped that lithium was the breakthrough he and others had been seeking.
But after six months, none of the 87 people who stuck with the study showed any letup in the disease’s progress, said Karen Felzer. She now doubts the Italian study’s results.
“It’s obvious to everyone it’s not the miracle drug we thought at first,” she said. She also thinks other tests of lithium for ALS should be halted to spare patients the drug’s possible side effects, such as tremors, weakness and difficulty breathing. Her father stopped taking the drug, though Macedo is continuing.
However, other reseachers say professional lithium studies should go forward. Dr. Merit Cudkowicz, a Harvard Medical School professor, is set to begin one in December with 84 patients. Her study will stick to the so-called gold standard of research, in which each patient will be randomly chosen to take the drug in question or a placebo. Neither patients or researchers will know who got the drug to avoid introducing bias.
Because the patient-led lithium study lacked those tight controls, it is unreliable as a measure of safety and effectiveness, Cudkowicz said. With an incurable disease, she said, “You don’t want to be throwing something away that works because of a bad study.”
Exactly right. Just because a badly designed study failed to find a result does not mean that lithium is not an effective treatment for ALS. It might well be. Unfortunately, the PatientsLikeMe.com study was (and is) a bad study, which by its very design was highly unlikely to find any but the most dramatic treatment effect from lithium. The same was true of the sharing of anecdotes about DCA for cancer. The only way Jim Tassano would have found a treatment effect for DCA is if it were truly the “miracle” drug that patients hoped it to be. Anything less, and unrandomized, unblinded patient-driven studies don’t have the reliability or sensitivity to detect differences between the experimental and control groups–or worse, they are prone to bias that may lead to a false positive. As understandable as it may be that patients want to do something, these efforts are more likely to mislead than enlighten.
There will always be a conflict between wanting to do something now for suffering patients, damn the consequences, and following the scientific method to demonstrate efficacy and safety. It’s perfectly understandable and the need of dying patients is especially compelling. We have seen both extremes. Indeed, until 1906, drug sellers could make essentially any claim and sell essentially anything to the public as a drug without regulation. We all know how well that worked out. Early in the history of the FDA, companies often tested new drugs by sending them to doctors to offer to their patients, asked for little information regarding side effects and complications, and had no standard criteria for efficacy. Efforts of groups like PatientsLikeMe.com may be an improvement over a century ago in that they do go to great efforts to ask patients about side effects and changes they note in how they feel, but, boiled down to its essence, this patient-driven trial is the same thing as the unscientific experimentation of that era. It benefits neither science nor patients.
