New Study Latest to Shine Light on Unpublished Medical Data

Health Letter, October 2014

By Sammy Almashat, M.D., M.P.H.

When you go to the doctor, you assume he or she is up to speed on — or at the very least has access to — the latest medical knowledge necessary to make a decision about your treatment. However, over the past two decades, it has become increasingly clear that the medical community sees only a fraction of the myriad data on the benefits and risks of the drugs prescribed on a daily basis. And the portion of data available does not fairly represent the whole.

The published literature serves as the primary source of medical knowledge for physicians and medical researchers. Evidence suggests that “positive studies,” those with conclusions favorable to a drug or other therapy, are more likely to be published than “negative studies” showing that a drug is harmful or ineffective. This trend is known as publication bias and has been increasingly identified as a pervasive threat to evidence-based medical decision-making.

Multiple studies highlight deficiency of published literature

The decision to seek publication of a study is typically made by the study’s funder, such as a pharmaceutical or medical-device company. Journals must then decide whether to accept the study for publication. However, because journals do not usually disclose lists of rejected manuscripts, it is often impossible to determine whether a study went unpublished because it was rejected or because it was not submitted for publication in the first place.

A study published in August in the journal Arthritis and Rheumatology is the latest to estimate the extent of unpublished trials.[1] The study focused on randomized controlled trials of subjects with rheumatoid arthritis, a debilitating disease for which treatments are often toxic and, in some cases, ineffective in slowing its progression.

Of all such trials registered with ClinicalTrials.gov (a federally run clinical trials database; see “ClinicalTrials.gov: A woefully neglected solution”) and completed prior to 2010, one-third, which had enrolled approximately 10,000 subjects, were still unpublished. Two-thirds of published studies from the database had positive outcomes compared with less than one-third of unpublished studies, with statistical analyses demonstrating that positive studies were almost five times more likely to be published than negative studies. Positive studies were also published much sooner (an average of 30 months after study completion) than negative studies (49 months).

The Arthritis and Rheumatology study confirmed earlier trends. A 2008 study of industry-funded antidepressant trials found that 31 percent were never published. Studies deemed positive by the Food and Drug Administration (FDA) were much more likely to be published (only a single positive trial went unpublished) than those the agency deemed negative.[2] In a striking conclusion, the authors found that were a physician to review only the medical literature, it would appear that 94 percent of all antidepressant trials had positive results, whereas the actual rate, as determined by the FDA, was 51 percent.

A wide-ranging 2013 analysis reviewed all large randomized trials that had been registered at ClinicalTrials.gov and completed prior to 2009.[3] Of all reviewed trials, 29 percent, involving almost 300,000 subjects, went unpublished. The authors found that industry-funded trials were far more likely to go unpublished than studies with other funding sources (32 percent versus 18 percent, respectively).

Publication bias: Roots in profit-driven research

Why do so many trials go unpublished? First and foremost, publication is not mandatory. In this context, as the publication process is labor- and time-intensive, many researchers choose to submit for publication only studies they deem important or likely to be accepted. There is, in turn, a tendency among some researchers and journal editors to gravitate toward publishing positive or novel findings, rather than those with negative or confirmatory outcomes.

As suggested by the 2013 analysis, another fundamental factor is the profit motive. In an age where the multibillion-dollar drug industry has emerged as the chief funder of pharmaceutical medical research, the choice to publish — as the above studies demonstrate — all too often comes down to whether the study’s outcome is favorable to the studied drug’s commercial success.

These same commercial considerations often lead to decisions that cross the line from withholding publication (which, while unethical and dangerous for medical practice, is not illegal) to criminal concealment of data from the federal and state governments. Over the past two decades, at least six pharmaceutical companies have paid millions of dollars to settle allegations that they concealed vital study data from government agencies or consumers.[4] In one of the highest-profile cases, GlaxoSmithKline pleaded guilty to withholding from the FDA data on the cardiovascular safety of its dangerous diabetes drug rosiglitazone (Avandia) and was forced to pay a $243 million criminal fine for the offense as part of a larger $3 billion settlement with the federal government.[5]

ClinicalTrials.gov: A woefully neglected solution

An attempt to remedy such pervasive publication bias came in the form of ClinicalTrials.gov, a government-run website launched in 2000.[6] In 2007, Congress passed a law requiring that virtually all U.S.-based clinical trials (with several significant exceptions[7]) be registered on the website.[8] The purpose of the database is to make the public aware of the existence and the outcomes of virtually all major clinical trials, including, crucially, those that are never published in the medical literature. However, recent studies have highlighted inadequate compliance among trial researchers and sponsors with both the registration and reporting requirements.

The 2014 study of rheumatoid arthritis trials found that less than a third of all trials had registered with ClinicalTrials.gov before the start of the study, and almost a third of all trials had still not registered by study completion.[9] A 2012 study in The British Medical Journal found that only 22 percent of all trials of approved drugs had complied with the legal requirement to report summary results on ClinicalTrials.gov within one year of trial completion.[10]

Withholding publication a breach of ethical obligations

The authors of the 2013 analysis of unpublished clinical trials pointed out that “[t]rial [researchers] and sponsors have an ethical obligation to study participants to publish trial results.”[11]

The active dissemination of medical knowledge gleaned from clinical trials is essential not only to allow physicians and patients to make fully informed treatment decisions but also to ensure that human subjects who incur the serious and often life-threatening risks of medical research do not do so in vain. To this end, ClinicalTrials.gov and other efforts to make all clinical trial data public must be strengthened so the public does not have to rely on a dangerously biased fraction of data that underpins life-and-death medical decisions.

Access to Clinical Trial Data: The FDA’s Role

Clinical trials are a type of study conducted to evaluate whether an intervention (typically a drug or medical device) is safe and effective. These studies are usually conducted by the pharmaceutical or medical device industries for the purposes of obtaining approval to market a drug or medical device from the Food and Drug Administration (FDA). After completion of the trials, companies wishing to gain approval of their products submit the trial results to the FDA, which reviews the data in order to decide whether to approve or reject the product.

Thus, the FDA has access to clinical trial data for all products for which approval is sought for a given disease. However, the agency does not release these data to the public, instead disclosing only its own internal analyses of the data. (An unprecedented, far-reaching European Union proposal would, if finalized, publicly release the actual data of all clinical trials on drugs approved by the European Medicines Association.[12])

This limited disclosure is undertaken only for products that ultimately receive approval to treat a disease, while FDA analyses of clinical trials on products that are rejected for a proposed use are not released. Therefore, the public is not made aware of the existence or results of clinical trials of products that are rejected by the FDA (or of products for which approval for a certain disease is never sought), unless they are published in a medical journal or released in some other form.


References

[1] Khan NA, et al. Randomized controlled trials of rheumatoid arthritis registered at clinicaltrials.gov: What gets published and when. Arthritis Rheumatol. August 25, 2014. doi: 10.1002/art.38784. [Epub ahead of print]

[2] Turner EH, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252-260. http://www.nejm.org/doi/pdf/10.1056/NEJMsa065779. Accessed August 28, 2014.

[3] Jones CW, et al. Non-publication of large randomized clinical trials: Cross sectional analysis. BMJ. 2013;347:f6104.

[4] Public Citizen. Biased data can lead to substandard drug treatment. Health Letter. September 2012. https://www.citizen.org/sites/default/files/hl_201209.pdf. Accessed August 29, 2014.

[5] U.S. Department of Justice. GlaxoSmithKline to plead guilty and pay $3 billion to resolve fraud allegations and failure to report safety data. July 2, 2012. http://www.justice.gov/opa/pr/2012/July/12-civ-842.html. Accessed August 29, 2014.

[6] ClinicalTrials.gov. History, Policies, and Laws. http://clinicaltrials.gov/ct2/about-site/history#NIHReleases. Accessed September 5, 2014.

[7] Public Citizen. Biased data can lead to substandard drug treatment. Health Letter. September 2012. https://www.citizen.org/sites/default/files/hl_201209.pdf. Accessed August 29, 2014.

[8] ClinicalTrials.gov. FDAAA 801 requirements. Which trials must be registered and have results submitted to ClinicalTrials.gov? http://clinicaltrials.gov/ct2/manage-recs/fdaaa#WhoIsResponsibleForRegistering. Accessed August 29, 2014.

[9] Khan NA, et al. Randomized controlled trials of rheumatoid arthritis registered at ClinicalTrials.gov: What gets published and when. Arthritis Rheumatol. August 25, 2014. doi: 10.1002/art.38784. [Epub ahead of print]

[10] Prayle A, et al. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ. 2012;344:d7373. http://www.bmj.com/content/344/bmj.d7373

[11] Jones CW, et al. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ. 2013 29;347:f6104.

[12] Abbott A. European Medicines Agency set to publish clinical-trial reports. Nature. June 13, 2014. http://www.nature.com/news/european-medicines-agency-set-to-publish-clinical-trial-reports-1.15410. Accessed August 29, 2014.