BMJ  2003;327:E137 (4 October), doi:10.1136/bmjusa.02090002 (published 27 November 2002)

BMJ USA: Editor's choice

The randomized trial on its pedestal

From BMJ USA 2002;September:477

The news about hormone replacement reminds us of the distinction between observational and randomized data. Treatments that seem promising in observational studies are often refuted in randomized trials. The comparison groups in observational studies often differ in characteristics other than the intervention of interest. Observational evidence linking coffee with pancreatic cancer has less to do with coffee than with the frequency with which coffee drinkers smoke. Randomized trials distribute confounding variables equally, making it safer to conclude that differences across groups reflect the intervention in question.

For years doctors forgot this principle in assuming, largely from observational data, that hormone replacement prevents heart disease. Skeptics wondered whether health habits or other factors more common in women who take hormone replacement were the real reason for their lower risk. Now amassing evidence points to the seemingly indisputable conclusion that the skeptics were right (BMJ USA p 526).

Observational studies have long shown that people who take vitamin E have lower rates of cardiovascular disease, but randomized trials show no such thing. In this issue a randomized trial deals another blow to vitamin E by questioning its efficacy in preventing macular degeneration (BMJ USA p 491).

Worshipers at the altar of randomized trials often conclude that treatments dethroned by trials are ipso facto ineffective. But they can be wrong, especially when trials lack the design features to prove ineffectiveness. For example, Hall and Gale (BMJ USA p 485) note that the trial testing vitamin E for macular degeneration lasted only 4 years. Concluding that vitamin E is ineffective is wrong if it takes longer to work.

Caution is also warranted in interpreting positive trial results. Trials show that COX 2 inhibitors have greater gastrointestinal tolerability than conventional nonsteroidal anti-inflammatory drugs, but Jüni and colleagues (BMJ USA p 520) reach a different conclusion after examining data excluded from a trial's final report. Some might see only arcane statistics in the drug company's defense (BMJ USA p 522) and response from Jüni et al (BMJ USA p 523), but astute clinicians see a larger lesson: In making their points the debaters peel away the packaged simplicity of trials as reported in journals and expose their true complexity and ambiguity. Change the length of follow-up and outcome measure, and a definitive trial becomes inconclusive. A trial's persuasiveness is in the eye of the beholder.

Journalists and often the medical community miss this point in reacting to trials. From current coverage one would think that the book is closed on hormone replacement, but Wang-Cheng and Rosenfeld (BMJ USA p 484) emphasize that the trials leave many questions unanswered. How data are presented skews public perception. The news media prefer to dramatize the relative increase in risk—eg, a 41% increase in stroke incidence—but the absolute increase in risk (8 strokes per 10 000 women-years) gives women a clearer perspective.

Caveat emptor. Randomized trials have strengths over observational studies, but they are hardly the last word.

Articles cited in Edited Choice are listed below, beginning with their BMJ USA page number:

BMJ USA p 526 Journal Rack, bmj.com/cgi/doi/10.1136/bmjusa.02090006

BMJ USA p 491 Vitamin E supplementation (Taylor), http://bmj.com/cgi/content/full/325/7354/11

BMJ USA p 485 Prevention of age related macular degenration (Hall), http://bmj.com/cgi/content/full/325/7354/1

BMJ USA p 520 Are selective COX 2 inhibitors superior (Juni), http://bmj.com/cgi/content/full/324/7349/1287

BMJ USA p 522 CLASS clarification (Geis), bmj.com/cgi/doi/10.1136/bmjusa.02090004

BMJ USA p 523 The authors respond (Juni), bmj.com/cgi/doi/10.1136/bmjusa.02090005

BMJ USA p 484 Hormone replacement therapy (Wang-Cheng), bmj.com/cgi/doi/10.1136/bmjusa.02090003

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?