Intended for healthcare professionals

Letter

Review of publication bias in studies on publication bias: Here's a proposal for editors that may help reduce publication bias

BMJ 2005; 331 doi: https://doi.org/10.1136/bmj.331.7517.638-a (Published 15 September 2005) Cite this as: BMJ 2005;331:638
  1. M Maria Glymour, instructor (mglymour{at}hsph.harvard.edu),
  2. Ichiro Kawachi, professor of social epidemiology
  1. Department of Society, Human Development and Health, Harvard School of Public Health, Boston, MA 02215, USA

    Editor—Publication bias is a pervasive problem in biomedical research,1 Dubben and Beck-Bornholdt providing further evidence on its importance.2 The preference for publishing papers with significant results may seriously compromise the ability to draw valid conclusions from the published literature. This problem seems particularly relevant to results from epidemiological research.

    We offer a solution to this problem that lies at the disposal of journal editors. Preliminary editorial decisions could be based solely on the peer review of the introduction and methods sections of submitted papers. These two sections deal with the key issues on which editorial decisions would ideally be based: the importance of the research question and the potential for the study design and proposed analyses to inform that question.

    Blinding reviewers to the results and discussion sections may pose some challenges to the reviewing process because elements of these later sections are also relevant for editorial decisions. However, these difficulties would probably be outweighed by the benefits of reducing publication bias. Peer reviewers might be asked to make a preliminary recommendation to the editor (reject or continue further review) on the basis of the merit of the study design and proposed data analyses—not on the findings themselves.

    If manuscripts pass this initial stage then reviewers could be unblinded to the results and discussion sections. Our proposal could have the additional benefit of improving the clarity and detail of methods sections.

    Our proposal may be particularly appropriate for papers dealing with topics that are susceptible to publication bias—those in which prior hypotheses are biased strongly in one direction. The usefulness of this proposal could be further evaluated in a randomised trial: submitted manuscripts could be randomly allocated to either a traditional review process or a review process blinded to the results. Editors could then assess whether papers with non-significant results are more likely to be published under the alternative review process.

    Footnotes

    • Competing interests None declared.

    References

    1. 1.
    2. 2.