Systematic reviews from astronomy to zoology: myths and misconceptions

Systematic literature reviews are widely used as an aid to evidence based decision making. For example, reviews of randomised controlled trials are regularly used to answer questions about the effectiveness of healthcare interventions. The high profile of systematic reviews as a cornerstone of evidence based medicine, however, has led to several misconceptions about their purpose and methods. Among these is the belief that systematic reviews are applicable only to randomised controlled trials and that they are incapable of dealing with other forms of evidence, such as from non-randomised studies or qualitative research.

The systematic literature review is a method of locating, appraising, and synthesising evidence. The value of regularly updated systematic reviews in the assessment of effectiveness of healthcare interventions was dramatically illustrated by Antman and colleagues, who showed that review articles failed to mention advances in treatment identified by an updated systematic review.1

It is nearly a quarter of a century since Gene Glass coined the term “meta-analysis” to refer to the quantitative synthesis of the results of primary studies.2 The importance of making explicit efforts to limit bias in the review of literature, however, has been emphasised by social scientists at least since the 1960s.3 In recent years systematic reviews have found an important role in health services research, and the growing interest in evidence based approaches to decision making makes it likely that their use will increase. Not everybody accepts that systematic reviews are necessary or desirable, and as one moves further away from the clinical applications of systematic reviews cynicism about their utility grows. Several arguments are commonly used to reject a wider role for systematic reviews, and these arguments are often based on major misconceptions about the history, purpose, methods, and uses of systematic reviews. I have examined eight common myths about …