Intended for healthcare professionals

  1. Research
  2. Identifying outcome...
  3. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors
Papers

Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors

BMJ 2005; 330 doi: https://doi.org/10.1136/bmj.38356.424606.8F (Published 31 March 2005) Cite this as: BMJ 2005;330:753
  1. An-Wen Chan, resident physician (anwen.chan{at}utoronto.ca)1,
  2. Douglas G Altman, director2
  1. 1 University Health Network, Department of Medicine, Suite RFE 3-805, 190 Elizabeth Street, Toronto, ON M5G 2C4, Canada,
  2. 2 Cancer Research UK/NHS Centre for Statistics in Medicine, Oxford
  1. Correspondence to: A-W Chan
  • Accepted 20 December 2004

Abstract

Objective To examine the extent and nature of outcome reporting bias in a broad cohort of published randomised trials.

Design Retrospective review of publications and follow up survey of authors.

Cohort All journal articles of randomised trials indexed in PubMed whose primary publication appeared in December 2000.

Main outcome measures Prevalence of incompletely reported outcomes per trial; reasons for not reporting outcomes; association between completeness of reporting and statistical significance.

Results 519 trials with 553 publications and 10 557 outcomes were identified. Survey responders (response rate 69%) provided information on unreported outcomes but were often unreliable—for 32% of those who denied the existence of such outcomes there was evidence to the contrary in their publications. On average, over 20% of the outcomes measured in a parallel group trial were incompletely reported. Within a trial, such outcomes had a higher odds of being statistically non-significant compared with fully reported outcomes (odds ratio 2.0 (95% confidence interval 1.6 to 2.7) for efficacy outcomes; 1.9 (1.1 to 3.5) for harm outcomes). The most commonly reported reasons for omitting efficacy outcomes included space constraints, lack of clinical importance, and lack of statistical significance.

Conclusions Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance. The medical literature therefore represents a selective and biased subset of study outcomes, and trial protocols should be made publicly available.

Footnotes

  • Contributors AWC is the guarantor of the study, had full access to all the data in the study, and takes responsibility for the integrity of the data and the accuracy of the data analysis. AWC contributed to the study conception and design, acquisition of data, analysis and interpretation of data, and drafting the article. DGA contributed to the study conception and design, analysis and interpretation of data, and drafting the article.

  • Funding AWC was funded by the Rhodes Trust; DGA is funded by Cancer Research UK. The funding sources had no role in any aspect of the study.

  • Competing interest None declared.

  • Ethical approval None required.

  • Accepted 20 December 2004
View Full Text
Back to top