Editor's choice

To tell the truth

This article originally appeared in BMJ USA

Carl Bernstein says that journalists seek "the best obtainable version of the truth." The same applies to researchers, who test hypotheses about the truth. The quest for facts is an asymptotic process: better evidence makes us ever more certain about the truth, but we never quite get there.

Evidence that parathyroid hormone decreases osteoporotic fractures (BMJ USA p 202) or that tumor necrosis factor  blockade diminishes rheumatoid symptoms (BMJ USA p 193) seemingly demonstrates their effectiveness. But whether that is true depends on how effectiveness is defined. Some would say the evidence must show that the magnitude of benefits outweighs the harms.

Enthusiasts for evidence-based medicine sometimes forget the limitations of evidence in telling the truth. Casual readers of the systematic review on page 207 might conclude that cough medicines are ineffective. But a closer look reveals that even effective drugs would not appear effective in many trials because of inadequate statistical power.

Whether the data tell an accurate story is most unclear when the topic is human behavior. Is it true that 1 in 2455 births occur in women who are unaware of their pregnancy until they go into labor (BMJ USA p 212)? Are patients nine times more likely to get a drug if they ask for it (BMJ USA p 204)? Do cold remedies decrease visits to the doctor (BMJ USA p 197)? Why do doctors rarely ask about domestic violence (BMJ USA p 191)? The February 9 BMJ was devoted to war. What causes people to engage in a behavior that has claimed 107 million lives in the past century (BMJ USA p 233)? Designing studies to answer such questions is intensely challenging.

In this issue we explore which treatments for mental illness are most effective. Randomized trials place cognitive behavioral therapy at the top of the list, but is its superiority real or artifact? Holmes (BMJ USA p 225) notes that psychotherapy is not a drug, and research designs intended for drugs may distort the truth when applied to counseling. Drug trials are easily standardizede.g., each patient takes the same pillbut the methods of psychotherapy "arise out of an intimate relationship between two people that cannot easily be reduced to a set of prescribed techniques." Standardizing what doctors say eliminates the individualization on which good relationships depend. Hinshelwood notes that the trialist views these unpredictable influences as something to be stripped away by randomization. But in psychotherapy they are the intervention, making the randomized trial "almost completely helpless to assess relationship change" (BMJ USA p 228).

Neighbour, a general practitioner, takes all this in stride (BMJ USA p 229). He observes that "the Lancelots and Galahads of the research world tilt at each other with their controlled trials until a winner emerges to claim the lady's hand. . .. We general practitioners are expected to fall like terriers on every latest pronouncement from our betters and implement it gratefully." But what constitutes fact is often a matter of opinion. Science may offer the best obtainable version of the truth, but it is hardly the last word.

Reeve (BMJ USA p. 202), http://bmj.com/cgi/content/full/324/7335/435

Emery (BMJ USA p. 193), http://bmj.com/cgi/content/full/324/7333/312

Schroeder (BMJ USA p. 207), http://bmj.com/cgi/content/full/324/7333/329

Wessel (BMJ USA p. 212), http://bmj.com/cgi/content/full/324/7335/458

Mintzes (BMJ USA p 204), http://bmj.com/cgi/content/full/324/7332/278

Hueston (BMJ USA p 197), http://bmj.com/cgi/doi/10.1136/bmjusa.02030002

Jewkes (BMJ USA p 191), http://bmj.com/cgi/content/full/324/7332/253

War or Health? (BMJ USA p 233), http://bmj.com/cgi/content/full/324/7333/369

Holmes (BMJ USA p 225), http://bmj.com/cgi/content/full/324/7332/288#art

Hinshelwood (BMJ USA p 228), http://bmj.com/cgi/content/full/324/7332/288#resp3

Neighbour (BMJ USA p 229), http://bmj.com/cgi/content/full/324/7332/288#resp1