The Dark Art of Outcome Switching

Outcome switching is the process of altering the measures of a clinical trial to improve the chances that a drug is approved by the FDA…

Specifically, outcome switching is when a parameter of the clinical trial is adjusted after the results are in to improve the numbers. Some outcomes are not reported, and some outcomes are added after the fact, i.e. “switched”.

Nexium is a drug we have already written about as a treatment whose trial outcomes were massaged to ensure its approval. In the case of Nexium they doubled the window of time, having seen no difference in improvement after four weeks, they changed the trial to eight weeks. After eight weeks, the difference between the healing rate for Nexium over Prilosec was 3%. The trial was submitted and that 3% difference in outcome launched a prescription juggernaut that received a patent and cost the system billions.

Dr. Benjamin Goldacre is a doctor and science writer, author of Bad Pharma, and longtime critic of the pharmaceutical industry. He has uncovered serious flaws in the ways clinical trials are reported and published in major medical journals. His new effort, the COMPare Project is an online dashboard dedicated to exposing scientific errors and misrepresentations that are endemic in how clinical trials are published. At time of writing, 66 trials were checked by him and a team of students, yet only 9 were deemed scientifically perfect.  From his site:

Whenever we detect unreported or added outcomes, we write a letter to the journal pointing them out, so that readers are aware of the problems. We are tracking which journals have published our letters after 4 weeks – and which haven’t.”

Of the 57 letters they have sent to the editors of the science journals, only 5 were published. The letters address the scientific flaws of a trial or the unreported abandonment of a clinical trial’s registered lines of inquiry. The COMPare Project explains the pre-trial problems in their mission statement, Why This Matters.

“Before carrying out a clinical trial, all outcomes that will be measured (e.g. blood pressure after one year of treatment) must be pre-specified in a trial protocol and on a clinical trial registry (e.g. This is because if researchers measure lots of things, some of those things are likely to give a positive result by random chance (a false positive). A pre-specified outcome is much less likely to give a false-positive result. In the trial report, all pre-specified outcomes must then be reported, to ensure a fair picture of the trial results. However, in reality, pre-specified outcomes are often left unreported, while novel outcomes that were not pre-specified are reported.”

A drug company will register their trial by indicating what they are looking for.  And like with Nexium, they will adjust them to get approval by the FDA.  This is outcome switching and none of this is a surprise for an industry unbound by the ethics practiced in other areas of  medical science. The real outrage is how ineffective the medical journals are in policing this misbehavior. This is what spurred COMPare’s creation.

COMPare is not exactly tilting towards windmills, but there is a lot of institutional inertia contributing to outcome switching. When the economic incentives encourage dubious practices and oversight is lax, we all suffer. Goldacre is a crusader for better outcomes and better health. The COMPare Project will not likely stop the problem of outcome switching by itself, but hopefully by holding the journals accountable they will improve their culture and publishing standards. Transparency is a wonderful thing to strive for, but change (in the absence of meaningful regulation) can only be achieved if the public exerts consequences on those companies whose practices are problematic.

We applaud Dr. Goldacre’s efforts. He is one of the voices in the industry we most admire.