Patient-led “clinical trials” versus clinical research (2012 edition)

Dying of cancer can be a horrible way to go, but as a cancer specialist I sometimes forget that there are diseases that are equally, if not more, horrible. One that always comes to mind is amyotropic lateral sclerosis (ALS), more commonly known as Lou Gehrig’s disease. It is a motor neuron disease whose clinical course is characterized by progressive weakness, muscle atrophy and spasticity, with ultimate progression to respiratory muscles leading to difficulty breathing and speaking (dysarthria) and to the muscles controlling swallowing. The rate of clinical course is variable, often beginning with muscle twitching in an arm or a leg or slurring of speech. Ultimately, however, ALS progresses to the loss of ability to move, speak, eat, or breathe. The most common cause of death is from respiratory failure, usually within three to five years after diagnosis, although there is the occasional outlier with a less malignant form of the disease with a slower course of progression who can live a long time, such as Steven Hawking.

In other words, ALS is a lot like cancer in some ways. For instance, it is a progressive, fatal disease that usually kills within a few years at most. On the other hand, it is different from cancer in that, at least for many cancers we actually do have effective treatments that prolong life, in some cases indefinitely. In contrast the most effective treatment we currently have for ALS is a drug (riluzole) that is not particularly effective—it prolongs life by months—and can be best described as better than nothing, but not by a whole lot. So it is not surprising that ALS patients, like cancer patients, become desperate and willing to try anything. This is completely understandable, but sometimes, as with cancer, this desperation leads to activities that are far more likely to do harm than good. I was reminded of this when I came across a post in the antivaccine propaganda blog, Age of Autism, referring to an article in The Scientist entitled Medical Mavericks. (No, not “brave maverick doctors” but misguided “mavericks” nonetheless.) The fortuitous posting of this story, which was apparently designed to try to show that it’s not as crazy as critics have said to be treating autistic children with “Miracle Mineral Solution” (MMS) (which is a bleach) given that the introduction explicitly mentioned Kerri Rivera and the patient described in the article used sodium chlorite to treat his ALS, provided me the opening to discuss a group whose existence and advocacy brings up a complex tangle of issues that boil down to questions of how far patient autonomy should be allowed to go. I’m referring to a company, PatientsLikeMe, which describes itself thusly:

PatientsLikeMe was co-founded in 2004 by three MIT engineers: brothers Benjamin and James Heywood and longtime friend Jeff Cole. Five years earlier, their brother and friend Stephen Heywood was diagnosed with ALS (Lou Gehrig’s disease) at the age of 29. The Heywood family soon began searching the world over for ideas that would extend and improve Stephen’s life. Inspired by Stephen’s experiences, the co-founders and team conceptualized and built a health data-sharing platform that we believe can transform the way patients manage their own conditions, change the way industry conducts research and improve patient care.

What I see PatientsLikeMe as doing is trying to mine a very unreliable source, namely thousands of testimonials. However, one other thing that needs to be kept in mind is that PatientsLikeMe is a for-profit company. Consequently, it’s selling a product, and that product is its analyses of the information patients who sign up with the service reveal about their experiences with various medicine and products. It is true that the company has an admirable list of core values, such as putting patients first, promoting transparency, fostering openness, and creating “wow” (i.e., information that “wows” you), but the company is still selling a platform to its partners and using that platform to attract investors. It is described as a platform “where patients can share and learn from real-world, outcome-based health data.” In essence, PatientsLikeMe (PLM) is a social media company for patients that mines the reported experience of its patients as, if you believe the hype, thousands of “N of 1” clinical trials. Unbeknownst to PLM, the autism “biomed” blogger who wrote the AoA post has a better analogy for what PLM is.

I first learned about PatientsLikeMe four or five years ago, when I wrote a series of posts about self-experimentation by cancer patients with an unproven, unapproved drug, dichloroacetate (DCA). As regular readers might recall, DCA is a small molecule drug that was used to treat congenital lactic acidosis in children through its inhibition of the enzyme pyruvate dehydrogenase kinase. This inhibition shifts the metabolism of glucose towards oxidative metabolism in the mitochondria and away from glycolysis, the product of which is lactic acid. In January 2007, Dr. Evangelos Michelakis of the University of Alberta had the clever idea of using DCA to target the Warburg effect in tumors and published an article in Cancer Cell describing preclinical experiments in rats in which DCA resulted in marked shrinkage of multiple tumor types. The Warburg effect was first described in 1928 by Otto Warburg and refers to the tendency of many tumors to rely on glycolysis for their energy supply rather than oxidative phosphorylation, even in the presence of oxygen, which in the case of normal cells favors oxidative metabolism. It turns out that DCA showed some promise against glioblastoma in a phase I clinical trial, but it is a long way from being approved.

Because DCA is a small molecule and relatively easy to synthesize, a man named Jim Tassano decided that he would sell “pet DCA” to treat cancer in pets. Tassano fooled no one, of course; people were not buying DCA to treat their dogs. His real purpose was to take advantage of desperate cancer patients, some of whom flocked to his BuyDCA.com website to purchase what he claimed to be pharmaceutical grade DCA. It also led to what I saw as a most disturbing phenomenon, namely self-experimentation by cancer patients with DCA and discussion among patients on online forums, which was even likened to “clinical trials,” complete with reports of success based on magical thinking more than anything else. You will see echoes of this same story and the issue it raises in the article referenced by AoA.

This brings us to the link between MMS and PatientsLikeMe. MMS, as you recall, is being touted as a “miracle cure” for conditions as disparate as cancer and autism. It is nothing more than a 28% solution of sodium chlorite, a chemical widely used for water purification. A couple of months ago at that yearly quackfest of the “autism biomed” movement, Autism One, Terri Rivera gave a talk in which she advocated using MMS to treat autism. Her method of delivery included oral, bathing in it, and per rectum, as in MMS enemas. Yes, we’re talking bleach enemas to treat autism. Administered orally according to Rivera’s protocol, the amount of chlorine dioxide (the active chemical liberated when sodium chlorite is mixed into aqueous solution) is more than 120 times the amount that a child could expect to consume drinking tap water.

It turns out that the story being touted by AoA is about a man named Eric Valor who is taking sodium chlorite to treat his ALS:

In June 2010, he learned of an experimental ALS drug called NP001, being developed by Palo Alto–based Neuraltus Pharmaceuticals. Then just beginning Phase I trials, the drug targets ALS patients’ overactive immune cells, attempting to reduce the chronic neuroinflammation associated with the disease. It seemed promising to Valor, but unfortunately, his disease was too advanced for him to qualify for the trial. “I made various attempts to get [enrolled], but failed,” says Valor, who responded to The Scientist via e-mail because his ventilator limits his speech.

If he couldn’t participate in the trial, maybe he could get the drug another way, Valor reasoned. After an exhaustive literature search on PubMed, he identified the drug’s “cruder” precursor as WF10, which is available for purchase abroad. But he quickly learned that importing it from Thailand would cost more than $12,000 for a year’s supply. He then set out to see if he could get his hands on what he suspected—based on a literature search and a 2006 patent tracked down by fellow ALS patient and friend Rob Tison—was the active ingredient of NP001: sodium chlorite, which is used in water-purification kits for campers and in municipal water treatment.

This is an interesting story on many levels, not the least of which is the apparent finding that sodium chlorite might have an actual therapeutic use. But what is NP001? Due to its proprietary nature, it is not straightforward to figure out just what NP001 is from online sources. For instance, I found what was purported to be the structure of NP001 at the ALS Therapy Development Institute. Notice that there’s no chlorine atom anywhere in chemical structure, making it incredibly unlikely that this molecule would generate chlorite as its active compound. However, more searching around the web made me question whether this is the actual structure of NP001, starting with this press release from the company describing promising results for the drug in a phase I study. Unfortunately, the trial appears not to have been published yet (PubMed searches using the usual suspects, such as “NP001,” “Neuraltus Pharmaceuticals,” and the study’s principal investigator Robert G. Miller, among other strategies, failed to turn up anything); so I went to patent searches and found that Neuraltus Pharmaceuticals has patents for various buffered release systems for sodium chlorite, for example, this one, although searching using the term “NP001” didn’t turn up anything related to ALS. Elsewhere, it is stated that NP001 is a modified form of WF10, which is basically sodium chlorite and has been tested with varying results as an immune modulator for various conditions. In brief, WF10 is a stabilized chlorite matrix that appears to have immunosuppressive effects, although it also appears to stimulate natural killer cell cytotoxicity and decrease macrophage activity. This patent describes the claims for WF10.

To sum it all up, NP001 appears to be some form of sodium chlorite in a matrix designed to control its release, perhaps with another compound. Certainly, this FDA application for orphan drug status by Neuraltus Pharmaceuticals, Inc. for sodium chlorite pretty much cemented in my mind that that’s what NP001 is, as did the information on this discussion forum, which described how WF10 and NP001 relate to the drug from Thailand referenced in The Scientist article. Moreover, NP001 was granted not only orphan drug status but fast track status as well.

Now here’s the problem with the sort of “do it yourself”-style “N of 1” clinical trials advocated by PatientsLikeMe:

Last June, Valor began taking oral doses of the chemical, which is not approved for the treatment of any disease. And he thinks it’s working. “I have improved breathing, which makes transfers [off his ventilator] much more comfortable,” Valor says. “My voice is louder and speech somewhat clearer. I am able to flex muscles that were previously still (though not enough for useful movement).”

So, basically Valor is taking MMS for his ALS. We don’t know the concentration or the dose that he’s taking. The key phrase above is that Valor thinks it’s working; we have no idea whether there is any objectively measured improvement in his motor function. There are well-established rating scales for severity of ALS symptoms, and it would be much more useful to know whether these patients have any measurable improvement in concrete, objective measures of motor neuron and muscle function. I’ve looked, and I haven’t been able to find any. Valor’s impression could be the result of expectation bias, confusing correlation with causation (i.e., waxing and waning of symptoms correlating with drug administration by random chance alone), or other cognitive quirks to which humans are prone that lead to a mistaken impression of causation.

In addition, in The Scientist article, there is the story of Ben Harris, an ALS patient who was enrolled in the phase II clinical trial of NP001 last year:

At first, Valor didn’t tell other ALS patients about his experimentation, hoping to first establish that it was safe, “but by September, it had leaked,” he says. Medical physicist and ALS patient Ben Harris, who was enrolled in the NP001 trial, was the one to get the ball rolling. NP001 had improved his strength, speech, breathing, and ability to swallow, and he wanted to keep taking the drug after his participation in the trial ended. “The more I learned about [sodium chlorite], the more excited I got, and I felt it was too important to keep a secret,” says Harris, who also responded to questions via e-mail due to limitations associated with his disease. So he started a discussion thread on the website of Cambridge, Massachusetts-based biotech ALS Therapy Development Institute (ALS TDI), sparking widespread interest that helped launch the do-it-yourself (DIY) trial. Now more than two dozen other ALS patients are taking oral sodium chlorite, and recording their results on the social networking site PatientsLikeMe.

Here is Harris’ phase II trial report, as posted on PatientsLikeMe, along with others. Note that most of the patient evaluations for NP001 or sodium chlorite can only be viewed if you have a PatientsLikeMe account. However, the two reports that are publicly available list the drug as having “major effectiveness,” and have a lot of subjective accounts of symptomatic improvement with precious few objective measures (one, actually). At one point, Harris reports a large increase in the grip strength of his hands (the only objective measurement), but by the end of his participation in the trial in November 2011 his impression was this:

I just received my very last infusion of NP001. I think overall if I have declined during this study period it has been at a small fraction of the rate before starting the study. The only thing I can say that is worse is that I am more frequently wiping my lips. I think my lip muscles have atrophied but my tongue has not, neither have my left arm or leg. My left arm was getting weaker just before I started the study but it has held strong since. Along with the decrease in strength on my lips has been a very slight decline in my speech. I have been recording my speech weekly since August and it is very difficult to detect a change so the difference is very small. I believe overall NP001 has drastically slowed my progression if it hasn’t stopped it. Overall, I am better than I was when I started the study.

However, elsewhere, he wrote:

The one way I think I may have declined is in drooling. I seem to have to wipe my lips more often than before and I noticed night time drooling for the first time about one month after starting the study. But the difference is very very small compared to what it was like about 9 months ago. I think perhaps my tongue is stronger but my lips are a little weaker. It is difficult to decide whether I am improving or declining, which in itself is a good thing. Before starting the study I would notice differences in my state every month and they were obvious to me. Right now I am struggling to decide whether there are changes in my state from 4 months ago.

Another thing that you should note: There is a 50-50 chance that Mr. Harris received the placebo. He even acknowledges this elsewhere by telling a reporter that he always tells patients to whom he speaks of this possibility. Yet he wrote his PatientsLikeMe diary as though he knew he was receiving the experimental drug. He seemed to assume that he was in the experimental group and, when the NP001 trial was over, decided to do what Eric Valor has been doing and start taking sodium hypochlorite on his own, as described in this Wall Street Journal story back in April:

Although no patient knows whether or not he received the drug or a placebo, Mr. Harris says he experienced dramatic improvement in his ability to swallow after his first infusion of NP001. After he finished with the infusions on the Neuraltus trial, he wanted to continue. He worried that swallowing sodium chlorite wouldn’t be effective because it would be neutralized in the stomach, so in January, he started injecting it every three weeks. He says he feels no side effects. Mr. Harris says he didn’t inform Neuraltus about what he is doing, but feels that it shouldn’t have bearing on the formal trial since he didn’t start infusing himself until after the NP001 infusions were completed.

Mr. Harris isn’t sure he is benefiting but said last week that he gave himself an infusion and by evening, his swallowing had improved. The real proof, he says, will come when the participants do an analysis of the data collected online. Mr. Harris says he knows that the results will never compare to what Neuraltus is doing.

Eventually, the blinding will be lifted, and Mr. Harris will know. However, he could well be mistaken when he says that his use of sodium chlorite after his treatment as part of the trial was done has no bearing on the trial. If there are late measurements of motor neuron function and survival (and I assume there are), his use of what he believes to be NP001 could skew the trial data.

Unfortunately, this PatientsLikeMe “trial” is not a proper clinical trial at all, and it is highly unlikely that the “analysis” of the patient reports and data collected online will show any sort of conclusive evidence of benefit or lack thereof. The reason, of course, is that this is no more than a collection of anecdotes without a control group, without a standardized protocol covering dosage and administration of the drug, and without even a standardized source of the drug. That’s not to say that analyzing the data might not be useful for examining patients’ perceived side effects and for comparing perceived benefit to actual benefit measured objectively, but in reality what PatientsLikeMe is doing is more likely than not pretty close to useless for determining actual efficacy.

The Scientist article mentions a previous PatientsLikeMe trial looking at lithium as a treatment for ALS. It was this trial, as it turns out, that brought PatientsLikeMe to my attention in the first place, leading me to discuss the trial in depth using story from 2008 about ALS patients trying lithium to treat their ALS based on one promising study. Just as Tassano sparked “wild experimentation” of patients using DCA to treat cancer with little or no medical supervision based on a single study using rat tumor models, patient experimentation with lithium to treat ALS appears to have been sparked by an Italian study that reported a significant slowing of the progression of ALS. This study looked at both a mouse model and reported the results of a small randomized trial of lithium plus riluzole versus riluzole alone. As a result of this study, ALS patients started taking lithium off-label. Meanwhile, PatientsLikeMe proclaimed:

Together, with all the patients involved, we will run the first real-time, real-world, open and non-blinded, patient-driven trial. We believe we will have the power, within months, to begin answering the question of how much lithium modifies the progression of ALS. Unlike a blind placebo control trial, we are watching the use of this drug in the real world, and because of the number of patients and our system’s sophisticated data modeling, we can determine the significance of each reported change in each patient as he/she deviates from his/her predicted course. There are many risks to our approach, patient optimism, the placebo effect, uncertain quality, and many other variables will compromise our data. Despite these, and many other challenges, we remain committed to solving this problem.

As I pointed out before, this “problem,” as it was described, probably cannot be solved because it was inherent in the very design of this study, which was neither randomized nor blinded. There were no valid controls, only in essence historical controls (i.e., the “predicted course”). What’s even more puzzling is that the organizers of this trial even seemed to recognize these problems, acknowledging that the placebo effect, patient optimism, and many other variables may compromise the data. Yet they nonetheless expressed optimism that their “sophisticated data modeling” could overcome these problems.

As it turns out, lithium doesn’t improve survival or neurologic function in patients with ALS. Ironically, PatientsLikeMe published its observations several months before the official phase IIb clinical trial in a Nature Biotechnology paper and trumpeted on its blog that it had “refuted a published clinical trial,” specifically, the original clinical trial in PNAS that suggested that lithium might be useful in ALS, stating:

PatientsLikeMe developed a novel algorithm designed to match patients who reported taking lithium with a number of other ALS patients that had similar disease courses. By using a matched control group, PatientsLikeMe was able to reduce biases associated with evaluating the effects of treatments in open label, real world situations and improve the statistical power of the study making each patients contribution more meaningful.

Maybe. More likely, it was nothing more than wishful thinking to believe that this novel algorithm, whatever it is, does an adequate job of controlling for the enormous number and quantity of biases that exist in anecdotal reports from patients. Indeed, I can’t help but think that it is a rather apt comparison when AoA equates PatientsLikeMe “to many of the autism biomed boards.” The main difference is that in the case of PatientsLikeMe there is a company asking members of its website to take polls about the effectiveness of whatever they are trying at the moment and then trying to analyze the anecdotes in order to sell the results to its business partners. In fact, when you come right down to it, PatientsLikeMe is a lot like another social networking site, namely Facebook. Like Facebook PatientsLikeMe has members communicating online. Also like Facebook, its business model is to sell the results of its analyses of the data these members provide. Facebook sells to advertisers; PatientsLikeMe sells to its business partners. In all fairness, to be sure, PatientsLikeMe appears to be a lot more transparent than Facebook. It also doesn’t sell its members’ personal information to advertisers and marketers. Even so, its business model is not that different from that of Facebook. Certainly, there is a lot of marketing going on.

ALS is a horrible disease. I know I’ve already said that, but it is something that bears repeating. Consequently, I can’t blame a single patient for wanting to get his hands on NP001, even to the point of being willing to mix up some home-brewed sodium chlorite or buy non-pharmaceutical grade chemical from whatever supplier he can find. I get it. They don’t have the time to wait for drugs to wend their painfully slow way through the FDA approval process any more than patients with advanced cancer do. Like Derek Lowe, I can’t even say that I wouldn’t be sorely tempted to do the same thing myself, particularly with chemicals that are small molecules that can be purchased elsewhere or synthesized in my laboratory. I can even somewhat understand the motivation behind PatientsLikeMe. No doubt the founders of the company think they are somehow accelerating drug discovery while doing good. Unfortunately, they have yet to show that they are accelerating anything. If you look at their publications, none of them demonstrate positive efficacy, and most of them (for instance, the latest three) are about patient attitudes and validation of ratings scales. Nowhere is there any example of a study that has shown anything about the efficacy of a new treatment that adds anything to what conventional clinical trials show. For example, in the case of the lithium trial, no one’s going to believe the results of and Internet survey (which is what, in essence, the lithium trial was); they’re going to wait for the results of a real clinical trial.

PatientsLikeMe has always promoted a rather radical idea, namely that it is possible to tell whether a drug works by analyzing, in essence, a bunch of testimonials from a group of self-selected patients self-medicating with therapies in an unblinded fashion with no proper control group. I suppose it’s worth seeing if it is possible to glean any useful information out of the morass of stories that constitute the database utilized by PatientsLikeMe, but I’ve never had much confidence that the company would be successful at it, and I’ve seen no publications yet that suggest to me that my original assessment was wrong. From what I’ve seen thus far, from a scientific and drug development standpoint PatientsLikeMe appears to have failed, even as it has been wildly successful as a social network for patients. Unfortunately, with this most recent foray into reporting on home-brewed NP001-lookalike drugs (a.k.a. sodium chlorite), PatientsLikeMe is entering dangerous territory that it has eschewed in the past. Remember DCA? PatientsLikeMe could have done then for DCA what it’s doing now for sodium chlorite, but at the time it did not—and rightly so, I might add. Indeed, from a strictly analytic standpoint, DCA would have been a better test case because the DCA patients were using was chemically identical to DCA that showed promising results in the original rodent studies. In contrast, we have no idea whether the sodium chlorite patients reporting to PatientsLikeMe are using is chemically identical to NP001. Very likely it is not.

There’s a reason why AoA loves PatientsLikeMe, and that’s because it cannily sees it as a more sophisticated version of what “autism biomed” proponents have been doing in discussion groups for a long time: Setting up an online place where people trying anything under the sun up to and including the rankest form of quackery can get together and swap anecdotes and testimonials. One wonders how long it will be before members of the “autism biomed” movement starts infiltrating PatientsLikeMe in order to see if the company will examine their assessments of the various interventions to which they subject their autistic children, such as MMS enemas, hyperbaric oxygen chambers, untold varieties of supplements, chelation therapy, and other dubious and sometimes dangerous interventions. Indeed, given that PatientsLikeMe has recently published on its assessment of the efficacy of drugs given for off-label indications, one wonders whether its founders would be willing to examine whether Lupron works for autism. The precedent has been set with sodium chlorite, after all.

I hope that in the future PLM sticks with off-label drug uses and exploring the experiences of patients on existing clinical trials as an adjunct to the actual trial. There, it might actually be able to do some good. It’s highly unlikely that anything good can come of “studies” like the one being undertaken to ask patients taking sodium hypochlorite if they think it’s working and very likely that bad things will happen.