All Trials (and Aston Villa) UPDATED

All Trials (and Aston Villa) UPDATED

When it comes to transparency clinical trials could learn a lot from football. While there are plenty of complaints and controversies about conduct within a particular match, the end result is a hard outcome. It is easy to judge the relative performance of a team and a comparator team looking at their wins, draws, and losses. For example, let’s compare Sunderland’s percentage of wins over the past 25 games to Aston Villa’s:

Sunderland Won 28% (Won 7, Drew 8, Lost 10)

Aston Villa Won 16% (Won 4, Drew 9, Lost 12)

Let’s hide those Aston Villa negative results, they were certainly a trial to watch for their fans. With just the 9 draws and 4 wins considered they look somewhat better.

Aston Villa Won 31%

So that’s Aston Villa ahead of Sunderland! Sadly for Villa, bad results cannot be hidden and they remain in the danger zone for relegation. There is no-where to hide their data.

It’s a different matter for drugs that are prescribed for patients. There have been several examples of drugs that could have been relegated sooner, but managed to survive a few more seasons to the detriment of patients.

Underneath you can see a Forest Plot showing a drug that has a benefit. Because the diamond at the bottom is in the “Favours Treatment” area, and doesn’t touch the orange line of no difference, this means the drug is of some use, such as reducing the risk of depression.

The orange line is the line of no difference. The black lines with boxes are clinical trials (with 95% confidence intervals). The diamond at the bottom is the the summary measure made from all the trials combined.
The orange line is the line of no difference. The black lines with boxes are clinical trials (with 95% confidence intervals). The diamond at the bottom is the summary measure made from all the trials combined.

The problem is, if some of clinical trials are missing then the diamond may not be in the right place. If a number of negative clinical trials (showing the drug doesn’t work) are missing, and evidence suggests that positive studies are easier to find that negative studies, then the diamond will be too far to the left. Let’s see what effect adding in some missing negative clinical trials does to the above Forest Plot.

The red lines are the hidden clinical trials added in. The effective summary diamond has shifted to the right, crossing the line of no difference.
The red lines are the hidden clinical trials added in. The summary measure diamond has shifted to the right, crossing the line of no difference.

The diamond has moved to cross the line of no difference. This drug now has no benefit.

So does this happen? Well back in 2008 UK regulators were shocked to find that a number of clinical trials of paroxetine (an anti-depressant) in children had been withheld by GSK, which showed harm. The government considered prosecuting GSK, and you can read the investigation into GSK here [PDF]. The investigation concluded with the following statement:

In particular, we want to see absolute clarity in the legislation as to the information that must be supplied to the regulator, regardless of its source (eg inside and outside the EU, arising as a result of any use including use outside the terms of the marketing authorisation, use in any clinical trials, as well as use as defined in the marketing authorisation) and clear time scales within which such information must be supplied and sanctions for failing to comply with the legislation.

In a separate letter [PDF] the CEO of the UK’s regulator, while noting the importance of legislation to enforce the industry, lamented the failure to do so without.

Such a course of action should be unnecessary in an indsutry which relaies so heavily on public trust and aspires to high ethical standards. I would have thought it self-evident that such information should be made available promptly to the regulator in order that action can be taken to protect public health. However moral responsibility now needs to be insisted upon by the unambiguous force of law.

As well as clinical trial data being provided to regulators, there is an equally pressuring need for the data to be available to third parties. This is not fixed, as the ongoing campaign by the Cochrane Collaboration and the BMJ shows. Ben Goldacre has increased the pressure with his book Bad Pharma (Here he is in the New York Times as he takes the book to North America). Half of clinical trials remain unpublished.

There’s something you can do about this.

Go to alltrials.net and sign the petition.

If you are part of a professional body, then lobby your organisation. A number have already signed up, including the Medical Research Council and the Cochrane Collaboration.

Click on the folder below, add your voice, and let’s hope more folders are opened as a result.

All Trials

UPDATE (5th February 2013): It would be churlish, given the example given above, not to note the fantastic news that GSK has signed up to AllTrials.

GSK today further demonstrated its commitment to clinical trial transparency by announcing its support for the AllTrials campaign. The campaign is calling for registration of clinical trials and the disclosure of clinical trial results and clinical study reports (CSRs) to help drive further scientific understanding.

[.] GSK is committing to make CSRs publicly available through its clinical trials register. CSRs are formal study reports that provide more details on the design, methods and results of clinical trials and form the basis of submissions to the US Food and Drug Administration (FDA), European Medicines Agency (EMA) and other regulatory agencies. From now, GSK will publish CSRs for all of its medicines once they have been approved or discontinued from development and the results have been published. This is to allow for the data to be first reviewed by regulators and the scientific community. Patient data in the CSRs and their appendices will be removed to ensure patient confidentiality is maintained.

In addition, while there are practical challenges, the company also intends to publish CSRs for clinical outcomes trials for all approved medicines dating back to the formation of GSK.

Wow.