Randomised controlled trial of the Lidcombe programme of early stuttering intervention
BMJ 2005; 331 doi: https://doi.org/10.1136/bmj.38520.451840.E0 (Published 22 September 2005) Cite this as: BMJ 2005;331:659
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Jones et al. have not directly responded to the arguments put forward
in my rapid response. For example, they say that "... spontaneous recovery
has been a recurring theme in the rapid responses to our article... The
rate of natural recovery from stuttering in young children in the general
population has been estimated at 74%(1), but in clinical populations with
this condition this estimate is expected to be much lower...Evidence from
a previous study(2) suggests that approximately 40% of children identified
as stuttering had the condition for less than 6 months." However, they do
not directly relate this or other statements to my arguments to disprove
my case, so it is difficult for me to respond in a constructive way. My
guess is that they imply that any effect due to the spontaneous recovery
rate on the statistics is very small because the rate in the clinical
setting is only about 40% and not about 75%. But this argument is not
correct. The great irony is that the closer the recovery rate is to 50%,
the greater the impact on the statistics!! So by claiming a 40% rate
instead of a 74%, they have actually strengthened my argument that they
need to revise the statistical significance and effect size of their study
to take into account the natural recovery rate! The details of my
arguments are described in my first rapid response, and below I will
explain why a lower natural recovery closer to 50% makes the need for a
correction greater.
In a RCT, the subjects are split into two groups in such a way that
both groups can be considered to start from the same baseline. This is
standard procedure. However, this condition is violated in their trial,
because they cannot control for the fluctuation in the inherent natural
recovery rate of children who stutter. Two groups are in general NOT
equal, because they do not have the same inherent natural recovery rate.
The rate fluctuates in each group. For example, by chance one group might
have a recovery rate of 80% and the other group of 40%. So both groups in
general do not start out from the same baseline. And they need to correct
their statistical measures like effect size or statistical significance to
include this effect. Failure to include the effect leads to exagerated
effect size and significance, e.g. an observed treatment effect even if
there is not treatment effect! (see my first post)
The interesting observation is that this effect is greatest for a
recovery rate of 50%, because at 50% the average difference of the two
groups in the natural recovery rate is greatest. The strength of the
effect declines and goes to zero for a rate of zero or one, because for
these limiting cases the rate is the same in both groups. So ironically
claiming that the recovery rate is closer to 40% than to 75% increases the
effect, as 40% is closer to 50% than 75%!
To conclude, I re-emphasise that the statistical measures are not
correct, and that they need to revise their statistics. This correction is
not straightforward, but can be done using Monte Carlo simulation
techniques. I am happy to cooperate. Again, I point out that I do not
question the study as a whole, and I admire the efforts they put in. But
they need to realise that they cannot just apply the standard RCT
framework. They need to correct for the natural recovery rate, which is
not present in a standard RCT setting.
Competing interests:
None declared
Competing interests: No competing interests
The issue of spontaneous recovery has been a recurring theme in the
rapid responses to our article. There have also been a number of other
issues raised. Therefore we feel it is appropriate to respond to provide
additional information that will hopefully clarify the issues raised in
the rapid responses.
The rate of natural recovery from stuttering in young children in the
general population has been estimated at 74%(1), but in clinical
populations with this condition this estimate is expected to be much
lower. Children would normally present to the clinic only after stuttering
has been apparent for some time and it is unlikely that those experiencing
transient stuttering would seek help at a speech clinic. Evidence from a
previous study(2) suggests that approximately 40% of children identified
as stuttering had the condition for less than 6 months. Consequently,
natural recovery rates for children who present to a speech clinic are
likely to be significantly lower than the broader population recovery
rates, although precise estimates of this rate are still unknown. In order
to enrol in our study children needed to have been stuttering for at least
6 months.
This RCT is the most recent of a series of peer reviewed scientific
studies of outcomes of the Lidcombe Program (LP). Previous studies include
earlier phase trials as well as retrospective file audits both in
Australia and the United Kingdom(3). The combined data from these
retrospective studies show that 93% of children were able to attain levels
of stuttering of less than 1% of syllables stuttered (<1%SS). The
“treatment failures” were due to the abandonment /non-compliance of the
treatment due to reasons such as family relocation, family break-up,
identification of other speech or language problems that needed to be
addressed more urgently than the stuttering and behaviour issues with the
children. In these studies, on two occasions treatment was abandoned due
to problems with treatment delivery. A study examining the long-term
effects of the LP showed treatment gains were maintained for between 2-7
years(4). Additionally, treatment success was not compromised, at least
within the preschool years, when treatment was delayed for a year or more
after onset of stuttering. In our study, children were followed for 9
months on the allocated treatment but we felt that it would be unethical
to delay treatment in the control group after this time. Consequently, not
all the children allocated to the LP had completed their treatment within
the follow up period.
The RCT design ensures as far as possible that the groups of subjects
are alike in every respect and differences observed are those due to
either the intervention or chance. In our randomisation procedure we
stratified by age, gender, severity of stuttering, family history of
recovery from stuttering, and treatment site. These factors were thought
to possibly influence the outcome and we sought to maintain reasonable
treatment balance within these subgroups and hence the rate of natural
recovery would have been similar in the two groups.
As to the comment that only children with mild to moderate levels of
stuttering were recruited to the study, our only exclusion criterion based
on severity was mild stuttering (<2%SS) and children with stuttering
levels as high as 20%SS or more were recruited. Hence the study population
comprised moderate to severe early stutterers.
Another comment referred to the sampling of children’s speech. We
point out that speech samples included the child speaking to a family
member at home, to a non-family member at home and to a non-family member
away from home, reflecting a variety of speaking situations. More widely,
children do not move to Stage 2 of the program until the "daily global
severity ratings made by parents" drop to a very low level.
References
1. Yairi, E., & Ambrose, N. (1999). Early childhood stuttering I:
Persistency and recovery rates. Journal of Speech, Language, & Hearing
Research, 42, 1097-1112.
2. Andrews, G., & Harris, M. (1964). The Syndrome of Stuttering.
Clinics in Developmental Medicine, No. 17. London: Spastics Society
Medical Education and Information Unit in association with Wm. Heinemann
Medical Books.
3. Onslow, M., Packman, A., & Harrison, E. (Eds.) (2003). The
Lidcombe Program of early stuttering intervention: A clinician’s guide.
Austin, TX: Pro-Ed.
4. Lincoln, M., & Onslow, M. (1997). Long-term outcome of an
early intervention for stuttering. American Journal of Speech-Language
Pathology, 6, 51-58.
Competing interests:
None declared
Competing interests: No competing interests
The findings from this study conducted in New Zealand are also
significant for the management of stuttering in the developing world.
Stuttering in this region is generally regarded as self-limiting and not
requiring professional intervention, a view possibly reinforced by the
reported 74% natural recovery rate [1]. Consequently, most parents of
children with this disorder rarely seek help. For instance from our recent
work among children with communication disorders aged 6 months to 15 years
that were referred to our audiology clinic, only 2.2% presented with
stuttering compared to hearing impairment (65.2%), specific language
impairment (23.9%) or central speech disorders (6.5%) [2]. We believe that
many more did not seek medical intervention because it was also not
considered as an unusual developmental phase in early childhood.
Stutterers often devised individual coping strategies for interpersonal
communication as they grow older. As a result stuttering has become least
associated with social stigma (if any) compared to other communication
disorders.
Given the high proportion of those for whom intervention may not be
necessary, it will be of interest to ascertain conditions that could
facilitate or hinder natural recovery in any environment. However,
identifying those who may follow this course is would be difficult in
practice. Introducing an intervention programme in developing countries
would therefore be worthwhile except that some practical challenges are
foreseeable in the short-run. Firstly, a public health programme in
preschool years is often difficult to implement without considerable
investment in special awareness campaigns. A school entry programme may
prove more cost-effective for ease of implementation provided this
threshold is not too late for optimal intervention. Secondly, parental
literacy is a key factor in this intervention plan and this may restrict
its wide application. Additionally, the dearth of speech pathologists
required to oversee the programme is a major constraint that may be
difficult to resolve rapidly without some non-specialist training at
community levels. However, these challenges are not insurmountable. The
knowledge that stuttering can be effectively treated in early childhood
and that early intervention is more efficacious than natural recovery
should ultimately serve as an impetus for appropriate service development
in this region.
References
1. Yairi E, Ambrose N. Early childhood stuttering I: persistency and
recovery rates. J Speech Lang Hear Res 1999;42:1097-12.
2. Somefun OA, Lesi FEA, Danfulani MA, Olusanya BO. Communication
disorders in Nigerian children. Int J Pediatr Otorhinolaryngol (in press).
Competing interests:
None declared
Competing interests: No competing interests
I just noticed a spelling mistake in my rapid response.
The number 200 is wrong
"For a large sample of 1000 children with dysfluencies, about 200
would not recover."
Needs to be changed to
"For a large sample of 1000 children with dysfluencies, about 300
would not recover."
Competing interests:
None declared
Competing interests: No competing interests
I believe that the authors of "Randomised controlled trial of the
Lidcombe programe of early stuttering intervention" need to review their
statistical analysis. My main argument is that the authors have failed to
include the effect of the statistical fluctuations of the natural recovery
rate within the two randomized samples. The inclusion could considerably
reduce the statistical significance and effect size of their results.
Especially in combination with other important but not here discussed
methodological issues like possibility of relapse and different subtypes
(sex and family history). However, I want to point out that my comments on
the statistical study of the trial do not necessarily imply that the
Lidcombe treatment by itself is not effective.
I will first show why the natural recovery rate is important. Then, I
will discuss the impact of the effect. Using a Monte Carlo simulation I
will show that I can reproduce in 10% of the scenarios the authors'
"minimum worthwhile difference between the two arms set at 1.0% syllables
stuttered" by including the statistical fluctuation of the natural
recovery rate without any treatment effect! Finally, I wonder whether this
effect does not question the usefulness of a randomized controlled trial
setup in this special case.
Why does the statistical fluctuation of the natural recovery rate
needs to be included in the statistical analysis? For simplicity and
ignoring definition issues, I assume a natural recovery rate of 70%. 70%
of all children with dysfluencies become fluent speakers. For a large
sample of 1000 children with dysfluencies, about 200 would not recover.
However, for small samples, the rates could fluctuate significantly. It is
perfectly possible to by chance pick a sample with 20 children who will
all naturally recover or 20 kids that will not recover. So drawing several
samples, I will end up with a different rate for each sample, e.g. 61%,
92%, 100%, 78%, and so on. But only the average of the rates tends to 70%.
And this is the problem. The authors have split the children into two
groups, and performed a t-test to compare both samples. Implicitly, they
therefore assume that both groups have the same natural recovery rate.
However, this assumption is violated as the two groups might considerably
differ, as mentioned above, the rate could be 61%, 92%, 100%, 78%, and so
on. For example, they could by chance have started with the treatment
group having an 92% recovery rate and the control group having a rate of
61%. The bigger the sample size, the closer both groups are around 70%
recovery rate. To summarise, I argue that the authors need to discuss this
effect in their statistical analysis, and at least say why the effect is
irrelevant to their results.
The existence of this effect does not necessarily mean that by
including the effect their results are not significant. However, in this
paragraph I argue that the inclusion of the effect most likely dilutes the
significance of their results. First, the inclusion of statistical
fluctuation can only decrease and not increase the significance. The
question is by how much. Second, I can come up with realistic scenarios
that have a significant impact. For example, let us assume that by chance
the control group has an instrinsic rate of 50% and the treatment group of
70%. Further, I assume that all children stuttered at 5% before therapy,
and the recovered ones have 0%. I also assume no treatment effect. So I
end up with a dysfluency rate for the control group of ( 50% * 5% + 50% *
0 ) = 2.5% and for the treatment group of ( 30% * 5% + 70% * 0 ) = 1.5%.
This 1% difference is not due to treatment effect but statistical
fluctuation. Thus, I can construct scenarios that reproduce a clear
difference without treatment effect. Third, the scenario is not only
possible but is a probable scenario. I did a Monte Carlo simulation with
the parameters of the trial. I randomly created thousands of pairs of
control (20 children) and treatment (27 children) samples, and computed
the sample difference between them. I used the same method to get the
syllables stuttered as in the previous and second argument. I therefore
have thousands of possible sample differences, and was able to plot a
histogram with difference against probability. I find a 10% probability
that a "minimum worthwhile difference between the two arms set at 1.0%
syllables stuttered" occurs due to statistical fluctuation and not due to
treatment effect. This strongly indicates that the true statistical
significance and effect size of the treatment could be much lower with
inclusion of the effect.
To summarise, I believe that the authors need to include the natural
recovery effect in their analysis, because I have shown that the effect
exists and that it could impact the results. Finally, I am wondering
whether it is even useful to use the randomized trial setup due to this
effect and due to the fact that there is no need to filter out a placebo
effect? Another setup could have been to only have a treatment group
(which is then twice as large) and test whether this group has a higher
rate of recovery (natural and treatment-wise) than the natural recovery
rate while including the statistical fluctuation of the rate. But a more
detailed analysis is needed to decide between both setups, and not the
subject of this letter.
EMAIL: tom.weidig AT physics.org
BLOG: http://thestutteringbrain.blogspot.com/
Competing interests:
None declared
Competing interests: No competing interests
Treating developmental stuttering in its incipient stages makes
intuitive sense, especially considering the propensity for the
disorder to cement itself within the central nervous system, making
it highly resistant to treatment at later stages (i.e., in adolescents
and adults). However, regardless of therapeutic intervention,
Mother Nature appears to have her own recovery agenda,
allowing 74% (by these authors admission) of children incipiently
stuttering to complete recovery. For much of the first 50 years of
speech pathology's professional development, the world followed
Wendell Johnson's protocol, which simply advocated removing
unwanted attention from stuttering behaviors1. Because of what
we now understand about these natural recovery rates, we can
clearly see how Johnson created an illusion of therapeutic
success when 74% of his young patients were pre-ordained to
recover. However, for those children that continued to stutter,
implicit blame and guilt were cast a shadow upon the parents for
labeling and reacting to their child behavior and drawing negative
attention to their speech.
Our understanding of stuttering has come a long way
since the Johnsonian days and though the approach to treating
children is now diametrically opposite (i.e., we now advocate direct
intervention), it is still implemented with the assumption that taking
advantage of early neural plasticity is the best recourse for halting
the stuttering pathology. The question is, are the results achieved
nowadays any better? The Lidcombe program provides a parent-
based behavioral, response contingent approach to stuttering. In
this therapy, parents attempt to reduce stuttering by presenting
verbal positive affirmations for fluent speech behaviors and for
reducing " bumpy speech”. The ability of these children to reduce
their stuttering in front of clinicians is not new. Employing parents
as clinicians and collecting data in multiple settings is a new
approach in stuttering therapy and seems the signature of the
Lidcombe program, yet still does not eliminate a potential
Hawthorne effect during data and does not mean that any
observed reduction or elimination of stuttering occurs across
speaking environments and situations. Short-term reduction in
stuttering frequency is easy to demonstrate with any kind of
therapy2, whereas recovery appears to be difficult to achieve in
persistent stutterers at any stage of development.
Lidcombe behavioural therapy has been practiced for
over a decade. Its popularity has spread across the globe. In fact,
specialized manuals and training centers exist to help clinicians
become proficient in implementing Lidcombe procedures. Now, in
this day of "evidence-based" therapy, the first randomized trials by
these authors should be applauded. They took substantial efforts
to ensure that groups were adequately randomized and balanced,
the treatment was administered according to protocol and the
'counting of stuttering' was conducted by observers who were blind
to treatment allocation. However, knowing the natural recovery
rates, we are unsure whether the advantages of employing an
untreated control group outweigh the fact that a potential remedy
was withheld from children during a critical time in the pathological
development. Regardless, the authors suggest that the results
support the 'efficacy' of the Lidcombe program. An important
question in this study is how efficacy is defined. In this case,
efficacy may merely be an acceleration of the natural healing
process in those children prone to recover or a simple temporary
reduction in stuttering that that can occur with any therapeutic
program for incipient stuttering.
For any program to be considered truly effective in the
treatment of incipient stuttering it should be able to 'beat' Mother
Nature and show complete recovery (i.e. forever free from all
stuttering symptoms in most situations and the perception of being
a person who stutters) rates in excess of 74%.3 However, since its
inception, one of the main points of contention with this program,
as well as others that claim efficacy in remedying childhood
stuttering, has been the separation of therapeutic effects from
those of natural 'spontaneous recovery'. In this study, 134 children
were referred for research program but the results are only
reported 47 children, although the other children may have not
meet some research-imposed criteria it would have been very
informative to see the natural progression of the disorder. As a
group these 47 children demonstrate a mild to moderate
pathology, the use of mild to moderate children is typical of this
research group when they show results of less than 1 or 2%
stuttering.4 Results show that after nine months the mean
stuttering frequency in the treatment group was still 1.5%, with only
52% of children showing a stuttering frequency of less than 1%.
Additionally, in an earlier investigation into the impact of Lidcombe
program on early stuttering, the mean stuttering frequency
following a 12 week treatment was 3.5 % syllables stuttered,5
indicating that traces of stuttering were most likely still evident in
many and the pathology may be subject to 'balloon' again at a later
date. These data show no evidence of complete recovery
exceeding 74%. Therefore, this program cannot claim to heal any
child that would not be otherwise remitted if simply left under the
care of Mother Nature. Similarly, Franken et al. (2005)6 compared
the Lidcombe approach to a Demands and Capacities approach
and both treatments yielded similar positive outcomes. We suggest
that almost all therapies can be helpful in alleviating some of the
symptoms of childhood stuttering.7 The simple fact that children
are learning some means of temporarily 'inhibiting' stuttering in
nurturing environments (e.g., with parent therapists) can help ease
the burden of stuttering and train coping mechanisms. For those
who are likely to recover anyway, almost any type of therapeutic
intervention may accelerate the process. Thus, it is doubtful that
the Lidcombe or any other behaviorally based program is powerful
enough to rewire neural networks and alter the epidemiology of
stuttering. Treating children who stutter may be analogous to
attempts to increase adult height by administering human growth
hormone to children who are not growth hormone deficient.
Though short periods of accelerated growth can be observed, the
final height attained is often similar to what matched children
achieve without the hormone treatment.8 In other words,
regardless of therapy, Mother Nature still seems to have the final
word on who recovers and her agenda is not fooled by temporary
periods of accelerated gains.
We are advocates for providing children who stutter with
all the help in the world. Accelerating recovery and providing
stuttering children with means of coping with the pathology
remains a most noble cause. However, when employing the
Lidcombe program and stuttering persists, we must also caution
against falling into the Johnsonian snake pit of blaming parents for
an involuntary childhood pathology whose full recovery is most
likely beyond their control.
References:
1. Johnson W. A study of the onset and development of stuttering.
In Johnson W, Leutenegger RR, eds, Stutttering in Children and
Adults. Minneapolis: University of Minnesota Press, 1955.
2. Bloodtein O. A Handbook on Stuttering. San Diego, CA:
Singular Publishing Group, 1995.
3. Kalinowski J, Saltuklaroglu T, Dayalu V, & Guntupalli VK. Is
it
possible for speech therapy to improve upon natural recovery
rates in children who stutter?. Int J Lang Comm Disord
2005;40:349-58.
4. Onslow M, Andrews C, Lincoln M. A control/experimental trial of
an operant treatment for early stuttering. J Speech Hear Res
1994;37:1244-59.
5. Harris V, Onslow M, Packman A, Harrison E, Menzies R. An
experimental investigation of the impact of the Lidcombe Program
on early stuttering. J Fluency Disord 2002;27:203-14.
6. Franken MC, Schalk CJ, Boelens H. Experimental treatment of
early stuttering: A preliminary study. J Fluency Disord, doi:10.1016/
j.jfludis.2005.05.002.
7. Kalinowski J, Dayalu VN, Saltuklaroglu, T. Cautionary notes on
interpreting the efficacy of treatment programs for children who
stutter. Int J Lang Comm Disord 2002;37:359-61.
8. Kawai M, Momoi T, Yorifuji T, Yamanaka C, Sasaki H, Furusho
K. Unfavorable effects of growth hormone therapy on the final
height of boys with short stature not caused by growth hormone
deficiency. J Pediatr 1997;130:205-09.
Joseph Kalinowski, PhD
http://www.ecu.edu/cs-dhs/csd/stutt.cfm
Competing interests:
None declared
Competing interests: No competing interests
Re: Randomised controlled trial of the Lidcombe programme of early stuttering intervention
Participants.
I was wondering if any of the participants chosen for the randomised control trial had any co-occuring speech and/or language difficulties, learning disabilites or genetic conditions such as Down's Syndrome and whether their therapy outcomes differed from those of children without co-occuring difficulties or disorder?
Regards
Competing interests: No competing interests