BMJ  2003;327:E246 (4 October), doi:10.1136/bmjusa.03080002 (published 3 September 2003)

BMJ USA: Letter

RAPID RESPONSES FROM BMJ.COM

Following is an edited excerpt from one of the Rapid Responses generated by this article, which can be read in their entirety at http://bmj.com/cgi/eletters/326/7400/1175 — Editor

From BMJ USA 2003;August:430

What's wrong with Riemsma's review?

Editor — The conclusions of Riemsma et al are contradicted by Spencer et al's comprehensive review of 148 studies that applied the transtheoretical model (TTM) to smoking ( Am J Health Promot 2002;17:7-71[ISI][Medline]). In that review, of 22 studies evaluating stage-matched interventions, 16 were positive, 5 were negative, and 1 had mixed results. The authors concluded that the literature supports stage-matched programs and that "Studies with positive outcomes had better designs and fewer validity threats than those with mixed or negative outcomes."

Riemsma et al included only 7 of the 22 studies cited in the Spencer review, and they omitted a number of important studies with no explanation. They seemed to give equal weight to studies with statistically significant and non-significant results. Non-significance can easily be produced by small samples, by short follow-ups that do not permit sufficient smokers to progress to action or maintenance stages, and by interventions based on a single TTM variable, such as stage.

In our research we usually need 400 smokers per group to achieve 80% power, and at least 18 months of follow-up to attain abstinence rates of 23% to 25%, and we must tailor on all TTM variables to reach the predictable abstinence range. Of the studies examined by Riemsma et al, 16 had smaller samples (<300/group), 11 had short follow-ups (<12 months), and 16 matched only on stage. Three of four studies with full TTM tailoring produced significant results, compared with 6 of 16 that matched just on stage.

In the studies reviewed by Spencer et al, 4 of 5 with full TTM tailoring were positive, compared with 4 of 13 that matched only on stage. The two negative fully tailored studies were with adolescents. Although the number of fully tailored TTM studies is small, the sample size is large (>10 000).

By chance, only about 1 of 23 studies should be statistically significant. Thus, the number of significant studies in Riemsma's review was 8 times more than would be expected by chance. Studies with strong designs (larger sample, high participation rate, longer follow-up, and fully tailored) were more likely to report significant results. Using Riemsma's quality ratings, significant results were reported by 1 of 5 (20%) of low quality studies, 8 of 14 (57%) medium quality studies, and 3 of 4 (75%) high quality studies. Riemsma's own data support Spencer's conclusion that the higher the quality, the greater the probability of positive results.

University of Rhode Island, Kingston, RI. jop{at}uri.edu

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?