BMJ  2008;336:995-998 (3 May), doi:10.1136/bmj.39490.551019.BE

Analysis

Rating quality of evidence and strength of recommendations

What is "quality of evidence" and why is it important to clinicians?

¹ Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada L8N 3Z5, ² Norwegian Knowledge Centre for the Health Services, PO Box 7004, St Olavs plass, 0130 Oslo, Norway, ³ Basel Institute of Clinical Epidemiology, University Hospital Basel, Hebelstrasse 10, 4031 Basel, Switzerland, ⁴ Division of Gastroenterology, Case Medical Center, Case Western Reserve University, Cleveland, OH 44106, USA, ⁵ Department of Epidemiology, CLARITY Research Group, Italian National Cancer Institute Regina Elena, Rome, Italy

Guideline developers use a bewildering variety of systems to rate the quality of the evidence underlying their recommendations. Some are facile, some confused, and others sophisticated but complex

The first 150 words of the full text of this article appear below.

In 2004 the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group presented its initial proposal for patient management.¹ In this second of a series of five articles focusing on the GRADE approach to developing and presenting recommendations we show how GRADE has built on previous systems to create a highly structured, transparent, and informative system for rating quality of evidence.

Summary points A guideline’s formulation should include a clear question with specification of all outcomes of importance to patients GRADE offers four levels of evidence quality: high, moderate, low, and very low Randomised trials begin as high quality evidence and observational studies as low quality evidence Quality may be downgraded as a result of limitations in study design or implementation, imprecision of estimates (wide confidence intervals), variability in results, indirectness of evidence, or publication bias Quality may be upgraded because of a very large magnitude of effect, a dose-response gradient, . . . [Full text of this article]

A guideline’s formulation should include a clear question

Guideline developers should address the importance of their outcomes

Judging the quality of evidence requires consideration of the context

Factors in deciding on quality of evidence
Factors that might decrease quality of evidence
Factors that might increase quality of evidence

Study design is important in determining the quality of evidence

Five limitations can reduce the quality of the evidence

Study limitations
Inconsistent results
Indirectness of evidence
Imprecision
Publication bias

Three factors can increase the quality of evidence

Critical outcomes determine the rating of evidence quality across outcomes

Evidence profiles provide simple, transparent summaries

Conclusion

Related Articles

Grading quality of evidence and strength of recommendations for diagnostic tests and strategies

Holger J Schünemann, Andrew D Oxman, Jan Brozek, Paul Glasziou, Roman Jaeschke, Gunn E Vist, John W Williams, Jr, Regina Kunz, Jonathan Craig, Victor M Montori, Patrick Bossuyt, Gordon H Guyatt for the GRADE Working Group
BMJ 2008 336: 1106-1110. [Extract] [Full Text] [PDF]

Going from evidence to recommendations

Gordon H Guyatt, Andrew D Oxman, Regina Kunz, Yngve Falck-Ytter, Gunn E Vist, Alessandro Liberati, Holger J Schünemann for the GRADE Working Group
BMJ 2008 336: 1049-1051. [Extract] [Full Text] [PDF]

When are randomised trials unnecessary? Picking signal from noise

Paul Glasziou, Iain Chalmers, Michael Rawlins, and Peter McCulloch
BMJ 2007 334: 349-351. [Abstract] [Full Text] [PDF]

Simultaneous comparison of multiple treatments: combining direct and indirect evidence

Deborah M Caldwell, A E Ades, and J P T Higgins
BMJ 2005 331: 897-900. [Extract] [Full Text] [PDF]

Grading quality of evidence and strength of recommendations
BMJ 2004 328: 1490. [Abstract] [Full Text] [PDF]

Differences between perspectives of physicians and patients on anticoagulation in patients with atrial fibrillation: observational study • Commentary: Varied preferences reflect the reality of clinical practice

P J Devereaux, David R Anderson, Martin J Gardner, Wayne Putnam, Gordon J Flowerdew, Brenda F Brownell, Seema Nagpal, Jafna L Cox, and Tom Fahey
BMJ 2001 323: 1218. [Abstract] [Full Text] [PDF]

Rapid Responses:

Read all Rapid Responses

Can GRADE motivate better use of evidence in developing countries?

Timur Y. Aripov
bmj.com, 7 May 2008 [Full text]

GRADES of Evidence need local relevance

Marko Tostad
bmj.com, 8 May 2008 [Full text]