Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
What do the authors mean by Staged-matched?
- Jean-Francois ETTER (2 June 2003)
-
The Emperor has no clothes!
- Gabriel Symonds (4 June 2003)
-
What’s Wrong with Riemsma’s Review?
- James O Prochaska (12 June 2003)
-
Stage based interventions versus smoking cessation
- Abu SM Abdullah (14 June 2003)
-
Why meta-analysis has become the standard for integrative reviews
- Wayne F. Velicer, Colleen A. Redding (7 July 2003)
-
Narrative Reviews Versus Meta-Analysis
- Seth M Noar, Jay Maddock (9 July 2003)
|
|
|||
|
Jean-Francois ETTER, Lecturer University of Geneva, Switzerland
Send response to journal:
|
In their review, Riemsma et al. make the common mistake of confounding a concept and a theory. Stage of change is only one element in Prochaska's transtheoretical model of change, along with processes of change, decisional balance and self-efficacy. And contrary to what Riemsma et al. state (p. 1175), stage of change is not only readiness to change. Stage includes three additional elements: having made a quit attempt in the previous year (stage 3), being a smoker (stages 1-3) or an ex-smoker (stages 4-5) and duration of smoking abstinence (stages 4-5). Prochaska's theory certainly does not say that interventions should be matched only to stage, and within stage, only to motivation to quit. Thus, the inclusion criteria in this review are problematic, since the review includes studies that apparently used only a motivation scale such as Biener's contemplation ladder as matching criterion. But the main weakness of this review, is, as the authors themselves admit, that it is unclear whether the interventions included in the review were really stage-matched. No details are given about what the authors mean by 'stage-based intervention' and about the extent of the matching. As a result, there is a large heterogeneity in the interventions included in the review. The authors do not report whether the other components of the transtheoretical model, in addition to readiness to quit, were also used to match the interventions. Many of the studies included in the review tested the effect of multiple-component interventions (e.g. tailored letters versus no intervention, e.g. ref. w7) and can therefore not be considered only as tests of whether matching interventions to stage of change is effective. For all these reasons, this review is not very convincing. Competing interests: None declared |
|||
|
|
|||
|
Gabriel Symonds, General Practitioner Tokyo, Japan
Send response to journal:
|
It seems the penny has finally dropped. In spite of the ‘substantial volume of research focusing on stages of change...’ it is not surprising that only ‘limited evidence exists for the effectiveness of stage based interventions in changing smoking behaviour.’ The whole concept of ‘stages of change’ is nonsense. It matters not one jot or tittle whether a smoker is at the ‘precontemplation, contemplation, preparation’ or whatever, stage, as long as he or she continues to smoke. It is useless to call for more studies, however ‘methodologically sound and theoretically consistent’ on ‘stage based approaches to changing smoking behaviour.’ What is needed is a new approach aimed towards the eventual banning of tobacco, because no real progress will be made in dealing with the smoking epidemic unless it is seen for what it is: legalised drug addiction. Competing interests: None declared |
|||
|
|
|||
|
James O Prochaska, Director and Professor University of Rhode Island - CPRC, 2 Chafee Rd, Kingston, RI 02881 USA
Send response to journal:
|
The conclusions of Riemsma et al.[1] are contradicted by Spencer et al.’s comprehensive review of 148 studies applying the Transtheoretical Model (TTM) to smoking[2]. Of 22 studies in Spencer evaluating stage- matched interventions, 16 were positive, 1 mixed and 5 negative. They concluded that the literature supports stage-matched programs and that, “Studies with positive outcomes had better designs and fewer validity threats than those with mixed or negative outcomes” (Table 23). Riemsma’s review included only 7 of the 22 studies in Spencer. Riemsma et al. omitted a series of important studies with no explanation. They seemed to give non-significant studies equal weight to significant ones. Non-significance can easily be produced by small samples and with TTM interventions, short follow-ups that do not permit sufficient smokers to progress to action or maintenance, and interventions based on a single TTM variable like stage. In our research we usually need 400 smokers per group for .80 power, at least 18 month follow-ups to attain predictable outcomes of 23% to 25% abstinence and tailoring on all TTM variables to reach the predictable abstinence range.[3 4 5] In Riemsma’s review, 16 had smaller samples (<300/group), 11 had short follow-ups (<12 month) and 16 matched just on stage. In Riemsma, 3 of 4 studies with full TTM tailoring produced significance compared to 6 of 16 matched just on stage. In Spencer, we found 4 of 5 with full TTM tailoring were positive compared to 4 of 13 matched just on stage. The two negative fully tailored studies were with teens. The number of fully tailored TTM studies is small, but the sample is large (>10,000). By chance only about 1 of 23 studies should be significant. Riemsma counted 8 times more studies as significant than would be expected by chance. Further, the more demanding the studies in Riemsma’s and Spencer’s reviews (larger N, high percentage of a population participating, longer follow-ups and fully tailored), the more likely the results were significant. With Riemsma’s quality ratings, 1 of 5 low quality studies (20%) produced significance (Table 3). In their middle category, 8 of 14 (57%) produced some significance. In their top category, 3 of 4 (75%) produced significance, with the negative study being small, short and matched just on stage. Riemsma’s data support Spencer’s conclusion that the higher the quality, the greater probability of positive results. References 1. Riemsma, RP, Pattenden, J, Bridle, C, Sowden, AJ, Mather, L, Watt, IS, & Walker, A. Systematic review of the effectiveness of stage based interventions to promote smoking cessation. BMJ 2003; 326: 1175- 1177. 2. Spencer L, Pagell F, Hallien, ME, & Adams, TB. Applying the transtheoretical model to tobacco cessation and prevention: A review of the literature. A J Health Pro 2002; 17: 7-71. 3. Prochaska, JO, DiClemente, CC, Velicer, WF, & Rossi, JS. Standardized, individualized, interactive and personalized self-help programs for smoking cessation. Health Psychol 1993; 12: 399-405. 4. Prochaska, JO, Velicer WF, Fava JL, Rossi JS, & Tsoh JY. Evaluating a population-based recruitment approach and a stage-based expert system intervention for smoking cessation. Addict Behav 2001; 26: 583-602. 5. Prochaska, JO, Velicer, WF, Fava, J, Ruggiero, L, Laforge, R, Rossi, JS, Johnson, SS, & Lee, P. Counselor and stimulus control enhancements of a stage matched expert system for smokers in a managed care setting. Prev Med 2001; 32: 23-32. Competing interests: I am one of the developers of TTM and I receive royalties from the University for one of the TTM programs included in the reviews. According to university, state and NIH guidelines, neither of these count as conflicts of interest. |
|||
|
|
|||
|
Abu SM Abdullah, Research Assistant Professor Hong Kong SAR, PR China
Send response to journal:
|
Several intervention approaches have been looked at to help smokers quit smoking during the last decade. Because all interventions are not designed or conducted in similar manner, it is important to think through these efforts before the research community discards or accepts one type of intervention. Riemsma et al found limited effectiveness of stage-based interventions in smoking cessation 1. However, important flaws in the review process limit their conclusions. First, they included studies which examined the effectiveness of stage based interventions in influencing behaviour, but Prochaska's stage of change construct has been validated as an approach for managing behaviour change not as an event, but rather as a process 2. Furthermore, behaviour change could be defined in many ways such as quitting smoking or reducing smoking or making more quitting attempts or changing to occasional smoker. It was not clear how the authors dealt with these results, which are often discussed as secondary outcomes in the discussion part of published reports. Authors' inclusion only of primary outcomes might have led to these being missed. Second, the process of intervention delivery can affect the outcome. Many studies, which are designed not to deliver stage-based interventions, may actually end up giving stage-based interventions indirectly. Because interventions are provided with a focus on motivation to quit or directly to change behaviour (i.e. quit smoking), which is also a component of Prochaska's transtheoretical model. Not reporting such information does not necessarily explain the ineffectiveness of a study. Third, the review process was too rigorous. The authors set up too many criteria (18 variables) to evaluate the studies. It is certain that the more rigorous the review, the less evidence there will be that the intervention is effective 3. Making conclusions based on a few unmet criteria could be misleading. However, one important criterion, adherence to stage-based smoking cessation interventions was missed, but it could have provided useful information about the outcome. There is evidence that non-adherence to interventions could result in a poor outcome4. I would argue that future trials on smoking cessation should assess and report information on adherence to the intervention to help evaluate the effectiveness of interventions. While I commented the authors' efforts to look at the evidence on stage-based smoking cessation interventions, their conclusion could prejudice researchers and service providers against the substantial evidence accumulated over the last decade that stage-matched intervention is effective. This may also reduce the motivation of medical professionals, who are already reluctant to give advice to smoker patients to quit smoking. Professionals should remember that absence of clear evidence from systematic reviews does not mean that inertia is the recommended course of action 3, and should continue to use every opportunity to promote smoking cessation as recommended in the United States Clinical practice guidelines: Treating Tobacco Use and Dependence 5. References: 1. Riemsma RP, Pattenden J, Bridle C, Sowden AJ, Mather L, Watt IS, Walker A. Systematic review of the effectiveness of stage based interventions to promote smoking cessation. BMJ 2003; 326: 1175-1177. 2. Joseph J, Breslin C, Skinner H. Critical perspectives on the transtheoretical model and the stages of change. In: Tucker JA, Donovan DM, Marlatt GA (Eds). Changing addictive Behaviour: Bridging clinical and public health strategies, 1999, pp 160-180. Guildford press: New York. 3. Petticrew M. Why certain systematic reviews reach uncertain conclusions. BMJ 2003; 326: 756-8. 4. Haynes RB, Dantes R. Patient compliance in the design and interpretation of clinical trials. Control Clin Trial 1987; 8:12-19 5. Public Health Service. Clinical practice guideline: Treating tobacco use and dependence. US Department of Health and Human Services, USA June 2000 (ISBN 1-58763 007 9). Competing interests: None declared |
|||
|
|
|||
|
Wayne F. Velicer, Professor and Co-Director CPRC, University of Rhode Island, Colleen A. Redding
Send response to journal:
|
One goal of science is cumulative knowledge production. Historically, scientists attempted to reconcile conflicting research results by using subjective narrative reviews. However, different reviews on the same topic would often come to opposite conclusions. Integrative reviews have now moved beyond qualitative approaches to quantitative methods called meta-analysis [1, 2]. Meta-analyses over the last three decades have demonstrated that research results can accumulate and that useful and sound conclusions can be drawn using appropriate methodology. The review by Riemsma et al. [3] illustrates many of the problems with a qualitative approach that can lead to faulty conclusions. 1. Inadequate report sampling. Just as subject sampling plays an import role in primary research reports, research report sampling plays a critical role in secondary reviews. Riemsma et al. [3] included only 7 of the 22 RCT’s previously included in a more comprehensive review by Spencer et al. [4]. 2. Intervention heterogeneity. To combine studies, comparable interventions must have been employed. This review [3] combines studies with different interventions (manuals, tailored reports, counseling, and videos), different amounts of tailoring (many studies tailored on stage only), and different intervention doses. In a meta-analysis, these factors would have been coded and conclusions might have been drawn about which type of intervention, degree of tailoring, or dose is related to effectiveness. 3. Not weighting by sample size or study quality. A study based on larger sample sizes and better methodology should be given more weight in forming conclusions. While it is noted that ‘Overall, larger studies tended to include more positive outcomes…” [3, p.4], the analysis was limited and weighted all 23 studies equally. Power and effect size estimation are key components of design sensitivity [5]. A simple power analysis found that 70% of studies [3] had small sample size and inadequate power. Additional critical design issues [5], including different lengths of follow-up, degrees of intervention tailoring, intervention types, and population characteristics, are not addressed. 4. Misinterpreting results. Riemsma et al.’s conclusions [3] appear to be at odds with their data. Spencer et al. [4] reaches an opposite conclusion that is consistent with the data [3]. Since 73% of RCT’s had positive outcomes [4], the evidence supports stage-matched programs and “studies with positive outcomes had better designs and fewer validity threats than those with mixed or negative outcomes”. Meta-analysis is designed specifically to prevent conflicting conclusions in reviews such as these. [1] Glass G. V. Primary, secondary and meta-analysis of research. Educational Researcher 1976; 5: 3-8. [2] Schmidt, F. L. & Hunter, J. E. Meta-analysis. In J. Schinka & W. F. Velicer (Eds.), Research Methods in Psychology 2003: 581-606; Volume 2 of Handbook of Psychology (I. B. Weiner, Editor-in-Chief); New York: John Wiley & Sons. [3] Riemsma, RP, Pattenden, J, Bridle, C, Sowden, AJ, Mather, L, Watt, IS, & Walker, A. Systematic review of the effectiveness of stage based interventions to promote smoking cessation. BMJ 2003; 326: 1175- 1177. [4] Spencer, L, Pagell F, Hallien, ME, & Adams, TB. Applying the transtheoretical model to tobacco cessation and prevention: A review of the literature. A J Health Promotion 2002; 17: 7-71. [5] Lipsey, MW. Design sensitivity: Statistical power for experimental research 1990; Newbury Park, CA: Sage Publications. Competing interests: Wayne F. Velicer is one of the developers of TTM and receives royalties from the University of Rhode Island for one of the TTM programs included in these reviews. According to university, state and NIH guidelines, neither of these count as conflicts of interest. |
|||
|
|
|||
|
Seth M Noar, Assistant Professor Department of Communication, University of Kentucky, Lexington, Kentucky, 40506, USA, Jay Maddock
Send response to journal:
|
Dear Editor, A review in 2002 by Spencer et al. (1) concluded, “the TTM applies well to tobacco cessation…” (p. 63). Now, one year later, Riemsma et al. (2) conclude that “The evidence suggests that stage based interventions are no more effective than non-stage based interventions…” (p. 6). Both were scientific articles reviewing largely the same literature on stage- based interventions for smoking cessation, and yet both came to completely different conclusions. How can this be so? It is our view that while narrative reviews of the literature can be useful in some circumstances, meta-analysis is a superior technique for integrating the literature on a topic. What does meta-analysis have to offer that narrative reviewing does not? First, narrative reviews simply cannot provide the type of quantitative synthesis of the literature that meta-analysis provides. While narrative reviewers can summarize and make comments about a collection of studies, such reviews do not include the calculation of effect sizes that examine the strength (or lack thereof) of the effectiveness of an intervention. Thus, while a narrative reviewer might focus on a question such as “do stage-matched interventions work,” that same question to the meta-analyst becomes “what is the magnitude of effect (null, small, medium, large) for stage-matched interventions?” Reframing the question allows for a more sophisticated and sensitive analysis of the data (3). Second, meta-analysis can in a sense “correct” for studies that were not optimally designed. In any review, there are bound to be studies for which this is the case. For instance, a study may not find significant effects for an intervention, but this may be due to low statistical power rather than an ineffective intervention (a type II error). Published studies with less than adequate power are still commonplace in many high impact journals (4). Meta-analysis allows one to correct for such issues as effect sizes are calculated across a group of studies, whereas narrative reviewing does not allow for such corrections. Third, moderator effects can be examined through meta-analysis. This is very important since studies nearly always have heterogeneity among them. Thus, if we do find that stage-matched interventions are effective generally, what does this really tell us? The truly important question, it seems to us, is why they are effective. Examination of moderator variables (e.g., demographics, characteristics of smokers, etc.) can reveal under which circumstances such interventions are likely to be more or less effective. Finally, meta-analysis provides a more unbiased look at the data as compared to narrative review. Thus, what we have seen here is two narrative reviewers looking at largely the SAME set of studies, and coming to completely DIFFERENT conclusions. It is much less likely that this would be the case if meta-analysis was utilized, as this quantitative technique is less apt to yield results that are subjective and open to so many interpretations (5). In conclusion, a well-conducted meta-analysis of the literature on stage-matched interventions for smoking cessation could certainly quell this heated debate, and we strongly advocate the use of meta-analysis for this and related purposes. In our opinion, a very strong review would be a well-conducted meta-analysis with detailed scrutiny of the individual studies, including the examination of methodological and other factors that relate to the quality and descriptive aspects of the studies. References 1. Spencer L, Pagell F, Hallien, ME, & Adams, TB. Applying the transtheoretical model to tobacco cessation and prevention: A review of the literature. Am J Health Pro 2002;17: 7-71. 2. Riemsma RP, Pattenden J, Bridle C, Sowden AJ, Mather L, Watt IS, Walker A. Systematic review of the effectiveness of stage based interventions to promote smoking cessation. BMJ 2003;326:1175-77. 3. Lipsey, MW, Wilson, DB. Practical Meta-Analysis. Thousand Oaks, CA: Sage; 2001. 4. Maddock, JE, Rossi, JS. Statistical power of articles published in three health psychology related journals. Health Psychol 2001;20:76- 78. 5. Cooper, HM, Rosenthal, R. Statistical verses traditional procedures for summarizing research findings. Psychol Bull 1980;87: 442- 49. Competing interests: None declared |
|||