Population based randomised controlled trial on impact of screening on mortality from abdominal aortic aneurysm
BMJ 2004; 329 doi: https://doi.org/10.1136/bmj.329.7477.1259 (Published 25 November 2004) Cite this as: BMJ 2004;329:1259All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
The front cover of the BMJ (27 Nov 2004) declares boldly that
"Screening for aortic aneurysm does not reduce overall death rates". This
headline misrepresents the conclusions of the study itself, let alone all
the evidence from other population based studies, including evidence on
efficacy and cost-effectiveness from the rigorous, UK population based
Multicentre Aneurysm Screening Study of nearly 68000 men aged 65-74 years
(the MASS trial) (1). The MASS trial demonstrated that aneurysm screening
almost halves the risk of aneurysm related death, particularly by reducing
the risk of aneurysm rupture, which is a laudable enough aim in itself
given the catastrophic nature of an aortic tear. The Australian trial
which you report studied 41000 men aged 65-83 years and the authors admit
that their target group was not suitable. Almost 50% of the men over 75
years who were invited for screening did not attend for their ultrasound
scan, although two thirds of the aneurysm related deaths occurred in these
non-attenders. In the 65-74 year old age group not a single patient died
from aneurysm disease in the 8641 men who attended for screening, compared
to 11 deaths in the patients who failed to attend for their scans and 13
deaths in the control group. The authors conclude "The chief reasons for
our overall result seem to be our failure to identify and exclude men who
were unlikely to attend..." and a significant proportion of these were the
over 75 year old patients who were not included in the MASS trial.
Compliance with the screening invitation was clearly a problem in this
study.
In the UK, screening studies have been based in GP surgeries, with
invites to suitable patients on the GPs' lists being sent out from the GP.
The Australian study was based on the electoral roll and carried out in
specially set up screening clinics. Experience from the screening
programme in Gloucestershire (2) suggests that GP based screening can
achieve over 85% compliance in the NHS setting, compared to the 63%
achieved in this Australian study. The lesson from the Australian
experience, as acknowledged by the authors but not by the BMJ headline, is
that aneurysm screening programmes must be designed carefully and
monitored rigorously to be effective. The design of the MASS trial clearly
shows how to do this effectively and in a recent survey of the membership
of The Vascular Society, the establishment of a national screening
programme for the detection of abdominal aortic aneurysms was considered
to be the highest priority amongst consultant vascular surgeons in the UK.
Peter Lamont
Honorary Secretary
The Vascular Society
at The Royal College of Surgeons
35-43 Lincoln's Inn Fields
London
WC2A 3PE
(1) Multicentre Aneurysm Screening Study Group. The Multicentre
Aneurysm Screening Study (MASS) into the effect of abdominal aortic
aneurysm screening on mortality in men: a randomised controlled trial.
Lancet 2002; 360: 1531-9.
(2) Earnshaw JJ, Shaw E, Whyman MR, Poskitt KR, Heather BP.
Screening for abdominal aortic aneurysms in men. BMJ 2004; 328: 1122-4.
Competing interests:
None declared
Competing interests: No competing interests
The effectiveness of aortic aneurysm screening
Simon Thompson1, Lois
Kim1, Alan Scott2
1MRC Biostatistics Unit,
Cambridge, UK; 2St Richard’s Hospital, Chichester, UK
The front cover of the BMJ (27
November 2004) proclaims “screening for aortic aneurysm does not reduce overall
death rates”.The Australian randomised
trial (1), on which this headline is based, observed 18 aneurysm-related deaths
in the group of men invited for screening as against 25 in the control group.The corresponding reduction in mortality was
39% (relative risk 0.61 with a 95% confidence interval from 0.33 to 1.11),
which the authors summarise as showing that screening did not reduce overall
death rates.The authors and editors
have fallen into the common trap of interpreting a non-significant difference
as evidence of no difference.
The stated conclusion is all the
more surprising given the available evidence from other randomised trials
(table).In each trial, the number of
aneurysm-related deaths in the men invited for screening is lower than in the
control group, and so the relative risks are all below 1.The widths of the confidence intervals vary
according to the size and power of the trial.The largest trial, the Multicentre Aneurysm Screening Study (MASS),
shows a statistically significant benefit (2).So does the Danish trial, based on the published in-hospital
aneurysm-related mortality (3).The
other trials, the Chichester (4) and Australian trials, were too small to show
the difference convincingly.But it
does not need a formal meta-analysis to deduce the high level of evidence,
across the four trials, that screening reduces aneurysm-related mortality, by
the order of 40% (corresponding to a relative risk of 0.60).Speculation about possible reasons for the
‘differences’ between the results of the trials is unhelpful, when what is more
notable is their consistency.
The effectiveness of screening in
reducing aneurysm-related mortality is established.We should now focus on the timely delivery of a national screening
programme for abdominal aortic aneurysm in men of retirement age (5).
References
(1) Norman PE, Jamrozik K, Lawrence-Brown MM, Le MT, Spencer CA, Tuohy RJ,
Parsons RW, Dickinson JA.Population based randomised controlled trial on impact of screening on
mortality from abdominal aortic aneurysm.BMJ 2004; 329: 1259-62.
(2)Multicentre Aneurysm Screening Study Group.The Multicentre Aneurysm Screening Study
(MASS) into the effect of abdominal aortic aneurysm screening on mortality in
men: a randomised controlled trial.Lancet 2002; 360: 1531-9.
(3) Lindholt JS, Juul S, Fasting H, Henneberg EW.Hospital costs and benefits of screening for
abdominal aortic aneurysms. Results from a randomised population screening
trial.Eur J Vasc Endovasc Surg; 23: 55-60.
(4) Vardulaki KA, Walker NM,
Couto E, Day NE, Thompson SG, Ashton HA, Scott RA.Late results concerning feasibility and
compliance from a randomized trial of ultrasonographic screening for abdominal
aortic aneurysm.Br J Surg 2002; 89: 861-4.
(5) Earnshaw JJ, Shaw E,
Whyman MR, Poskitt KR, Heather BP.Screening for abdominal aortic aneurysms in men.BMJ
2004; 328: 1122-4.
Table: Most recent
published results from the randomised trials of abdominal aortic aneurysm
screening in men
Trial |
Age |
Number |
Number |
Relative (95% |
||
Invited |
Control |
Invited |
Control |
|||
Australian (1) |
65-83 (5 years) |
19352 |
19352 |
18 |
25 |
0.61 (0.33 to 1.11) |
MASS, UK (2) |
65-74 |
33839 |
33961 |
65 |
113 |
0.58 |
Denmark (3) |
65-73 |
6339 |
6319 |
6 |
19 |
0.32 |
Chichester, UK (4) |
65-80 |
3000 |
3058 |
24 |
31 |
0.79 |
Competing interests:
None declared
Competing interests: Simon Thompson
The authors conclude that population level screening for aortic
aneurysm was not effective in men aged 65-83 years.
In this study the point estimate was a (surely worthwhile) 40%
reduction in mortality, and although the upper confidence limit was
greater than 1, the lower confidence limit was consistent with a massive
67% reduction. Failure to demonstrate a statistically significant
reduction in mortality is not the same as ruling out such a benefit.
Essentially the sample size was not large enough. One reason may be
that as the authors observe, the control group mortality rate was lower
than that based on published statistics. It is possible that detection of
the shortfall in the anticipated number of deaths would have permitted an
interim sample size re-estimate and maintenance of the required power.
However the authors also state that the study was intended to focus
on the 65 to 74 age group (for which the subgroup analysis was, as it
happens, statistically significant), so why was this age group not used
for the primary analysis?
Lastly, even if the original intention was an overall analysis for
the 65-83 year group, once the Multicentre study results were available it
would have been legitimate to revise the analysis plan to focus on the
restricted age group (provided this was done before analysis of the data).
The lesson from this is that indeterminate results from trials can
sometimes be avoided if studies are designed to cope with unforeseen
practicalities. Provided this is done a priori the trial integrity
needn't be compromised
Competing interests:
None declared
Competing interests: No competing interests
The dramatic headline that accompanied the recent paper by Norman et
al (1) suggesting that “screening for aortic aneurysm does not reduce
overall death rates” seems contrary to the current views of most vascular
surgeons in the UK (2, 3). Consideration of the reasons for this apparent
contradiction may provide some useful insights into the potential
shortcomings of randomised controlled trials and associated cost-
effectiveness analysis in determining policy with respect to screening
programmes.
For a screening programme to be effective it must be carried out
before morbidity from the condition is incurred. In this study screening
resulted in about 50 additional elective aneurysm repairs compared to the
control group, whilst there were 249 procedures for aneurysm carried out
prior to the planned screening date. This suggests that screening may have
been carried out too late for it to be effective and one must question the
rationale for screening men up to the age of 83 when the majority of
aneurysm complications occur before this age.
Screening relies on offsetting early costs in both morbidity and
resource use, against deferred benefits. For this reason trials are very
sensitive to censoring due to loss of patients to follow-up or restricted
trial duration. The trial reported by Norman et al (1) and the previously
reported MASS trial (4) were of limited duration (5 and 4 years
respectively), whereas estimates of the effects of screening currently
suggest that a screening programme at age 65 would take approximately 15
to 20 years to reap the full benefits (3). The MASS trial demonstrated a
benefit for screening on a relatively short time horizon, but in the cost-
effectiveness analysis alongside the trial screening appears quite costly
(£35k per QALY) (5).
Whilst RCTs are likely to provide the best possible data upon which
to base decisions, they may not provide clear answers, particularly where
effects are likely to persist for many years or where there are complex
issues around policy with respect to management algorithm or subgroup
selection. Under such circumstances the use of best available evidence in
an economic model with a suitable time horizon and probabilistic
sensitivity analysis is likely to provide a far more robust basis for
decision making and to allow the consideration of many alternative
scenarios. This is the approach that has recently been recommended by NICE
for carrying out technology appraisals (6). In the case of aneurysm
screening, even a pessimistic model based upon the MASS trial and a 10-
year time horizon suggests it to be cost-effective at around £10,500 per
QALY (5). More detailed modelling suggests it to be even more cost-
effective under a range of possible scenarios (7).
The trial by Norman et al (1) is a thorough and valuable study but it
would be unfortunate if the results were misinterpreted as undermining the
argument for the development of a national screening programme that is
likely to save many lives and would almost certainly be highly cost-
effective.
1. Norman PE, Jamrozik K, Lawrence-Brown MM, Le MT, Spencer CA,
Tuohy RJ, et al. Population based randomised controlled trial on impact of
screening on mortality from abdominal aortic aneurysm. BMJ
2004;329(7477):1259.
2. Greenhalgh RM. National screening programme for aortic aneurysm.
BMJ 2004;328(7448):1087-8.
3. Screening for Abdominal Aortic Aneurysm. Proceedings of a Meeting
held at the Governor's Hall, St Thomas Hospital, London on 31st March 2004
under the auspices of the Vascular Surgical Society of Great Britain and
Ireland; 2004. (available at
http://www.vascularsociety.org.uk/Docs/3208_A5Booklet1.pdf)
4. Ashton HA, Buxton MJ, Day NE, Kim LG, Marteau TM, Scott RA, et
al. The Multicentre Aneurysm Screening Study (MASS) into the effect of
abdominal aortic aneurysm screening on mortality in men: a randomised
controlled trial. Lancet 2002;360(9345):1531-9.
5. Multicentre aneurysm screening study (MASS): cost effectiveness
analysis of screening for abdominal aortic aneurysms based on four year
results from randomised controlled trial. BMJ 2002;325(7373):1135.
6. National Institute for Clinical Excellence (Great Britain). Guide
to the Methods of Technology Appraisal. London: National Institute for
Clinical Excellence, 2004.
7. Boll AP, Severens JL, Verbeek AL, van der Vliet JA. Mass
screening on abdominal aortic aneurysm in men aged 60 to 65 years in The
Netherlands. Impact on life expectancy and cost-effectiveness using a
Markov model. Eur J Vasc Endovasc Surg 2003;26(1):74-80.
Competing interests:
None declared
Competing interests: No competing interests
We would like to comment on the statistical analysis of the data,
presented in Table 4. The authors of the paper split the data into two
subgroups, and analyse both separately. A statistically significant
result in one group, and a non-statistically significant result in the
other group lead the authors to conclude that there is a difference
between the two groups. This is incorrect. The appropriate analysis is
to use an interaction test, to find out if the effects in the two groups
are significantly different from one another – this is a test of the
ratios of the odds ratios.
The odds ratios presented in the paper were 0.19 (for younger men) and
1.13, for older men. The ratio of these (0.19 / 1.13) is 0.17 (95% CIs
0.03, 0.92; p 0.039).
The authors have come to the correct conclusion, but not necessarily for
the correct reason.
Competing interests:
None declared
Competing interests: No competing interests
Sirs,
Norman PE. et al. conclude their intriguing paper stating that “At a
whole population level screening for abdominal aortic aneurysms was not
effective in men aged 65-83 years and did not reduce overall death rates”.
Interestingly, the corrected response to invitation to screening was 70%,
and among those who underwent to screening the crude prevalence was 7.2%
for aortic diameter 30 mm and 0.5% for diameter 55 mm., i.e. 7,7% of a
population at risk! In my opinion, based on 47-year-long steady clinical
experience, during which no one patient died from AAA, we have to
recognize “all” patients, both men and women in whatever age, affected by
abdominal aortic enlargement, independent of its size, monitoring them
regularly, and individuals with such as serious disorder, of course (1,
2). To obtain these results on very large scale, i.e., in men and women,
independent of their age, we need a clinical tool, that allows “general
practitioners” to recognize early and promptly diseased abdominal and
thoracic aorta, before any help of sophisticated semeiotics.
1) Norman PE., Jamrozik K., Lawrence-Brown MM:, et al. Population
based randomised controlled trial on impact of screening on mortality from
abdominal aortic aneurysm BMJ 2004;329 (27 November),
doi:10.1136/bmj.329.7477.0
2) Stagnaro S. Biophysical Semeiotic Diagnosis Of Abdominal Aortic
Aneurism, since early stages. (HONCode 233736, www.semeioticabiofisica.it
URL:
http://www.semeioticabiofisica.it/semeioticabiofisica/Documenti/Eng/Aneu...
3) Stagnaro-Neri M., Stagnaro S., Aneurisma Aortico Addominale: una
Diagnosi clinica con la Semeiotica Biofisica. Acta Cardiol. Medit. 14, 17,
1986
Competing interests:
None declared
Competing interests: No competing interests
I would also like to congratulate the authors with this important
study. I have one comment:
Norman et al. concludes that “it is also important to assess the
current rate of elective surgery for abdominal aortic aneurysm as in some
communities this may already approach a level that reduces the potential
benefit of population based screening”. If there is robust evidence for an
intervention of elective surgery for some of the abdominal aortic aneurysm
then a population based screening for abdominal aortic aneurysm should be
compare with this practice.
It is the opposite case with colorectal cancer screening. When the
Funen and Nottingham RCT were conducted from 1985 to 1995 colonoscopy and
centralisation of the surgical treatment were not normal diagnostic
procedures and treatment (1). So the effects seen in the Funen and
Nottingham studies are overestimated and the expected benefits of
colorectal cancer screening will probably be much less than expected (2).
Reference List
1. Brodersen J. Possible biases when estimating the outcome of
colorectal cancer screening. Cochrane Library 2003.
2. Anonymous. Results of the first round of a demonstration pilot of
screening for colorectal cancer in the United Kingdom. BMJ 2004;329:133-0.
Competing interests:
None declared
Competing interests: No competing interests
The authors are to be congratulated for carrying out this important
study. I have two questions:
In Tables 2 and 3, the operations for rupture plus the fatal ruptures
without surgery do not add up to the number given under “All ruptures”.
What became of the rest of the ruptures?
In Table 5, the difference in cumulative deaths between the two
randomized groups (2232 vs. 2571) is highly significant (p < 0.0001),
yet this difference disappears after age standarization. Can the authors
explain how this could occur in a randomized population with identical
mean age (72.6 years) and presumably identical age distribution?
Competing interests:
None declared
Competing interests: No competing interests
Re: Questions about AAA screening trial
We are grateful to Professor Lederle for drawing our attention to a
problem in Tables 2 and 3 of the published paper. Under the heading
‘Emergency’ in each of these tables the numbers in the columns labelled
“All ruptures” should indeed be the sum of “Operation” + “Fatal rupture
without surgery”, as shown below. However, as the correct numbers of
deaths were used in the calculations of age-standardised rates, this
amendment does not alter our original analyses or conclusions.
Table 2
Invited group Scanned (n=12,203), All ruptures = 3;
Not scanned (n=7,149), All ruptures = 19;
Sub-total invited (n=19,352), All ruptures = 22;
Control group (n=19,352), All ruptures =27.
Table 3
Invited group
Scanned (n=12,203), All ruptures = 3;
Not scanned (n=8,297), All ruptures = 32;
Sub-total invited (n=20,500), All ruptures = 35;
Control group (n=20,500), All ruptures =38.
Professor Lederle also noted an apparent discrepancy in Table 5. In
1996, potentially eligible men having been identified from the electoral
roll, we initially randomised men within five-year strata of age as this
was all that was made available to us by the Electoral Commission. Later,
in 1997-98, when a new version of the electoral roll became available, we
were able to randomise within strata of single years of birth. As a
result of the cruder stratification in 1996, the median (and mean) age
was, by chance, greater in the control group in that year (medians 74 vs
72 years; means 74.1 vs 73.2 years). This has been reflected in the
greater crude mortality in all men allocated to the control group, but the
difference disappears when the data for survival are standardised using
narrow (one year) strata for age.
We have performed exactly the trial that Dr Brodersen describes. Our
study is a test of (systematic) screening vs. usual care. In the course
of the latter, some asymptomatic cases of AAA will be detected through
clinical examination or diagnostic imaging performed for other purposes,
and some of these will proceed to elective surgery. The comment in our
paper was an acknowledgement that a test of a screening strategy might
fail to produce evidence of statistically significant benefits if the
level of detection of AAA in the unscreened group through unsystematic
assessment was already relatively high.
Dr Stagnaro has drawn attention to the dilemma of managing
individuals with relatively small aneurysms, whether detected by screening
or some other means. Although all clinicians should remember to palpate
for an AAA when performing an abdominal examination in older patients, it
is unlikely that this is sensitive enough for screening purposes.
In response to Dr Miles, we inserted Table 4 at the request of
reviewers. As it was a sub-group analysis, we included the data but were
careful not to draw strong conclusions based upon it.
We agree with Professor Michaels that setting policy with regard to
screening programmes is complex. Our observation that a large number of
AAAs had been repaired prior to the trial indicates a high level of pre-
existing detection and treatment. This level will vary from country to
country but if it is high, it will reduce any benefit from systematic
screening. In order to examine critically the predictions of the kinds of
models Professor Michaels mentions, it will be important for all trials of
screening for AAA to continue following up their participants for ruptured
aneurysms and mortality from all causes over considerably longer periods
than those covered by the initial reports from these studies, including
our own.
We were very much aware of the issues canvassed by Dr Silcocks
throughout the design, execution and analysis of our trial. Indeed, we
extended the target age-group upwards during recruitment because of our
concerns about statistical power. Our capacity to boost the size of the
65-74 year stratum was limited by the geography of Perth. Defining a
further secondary analysis mid-trial, once results of certain other
studies were published, struck us as a lot less radical than discarding
our a priori intention-to-analyse.
We agree with Professor Thompson that the totality of the available
Level 2 evidence is the minimum basis on which national policies on
screening for AAA should be set. Indeed, this body of evidence sets a
standard against which the oft-heard calls to establish screening
programmes for other conditions should be judged.
We reported the study in a transparent fashion and deliberately were
cautious in our conclusions. There are significant differences between
Australia and the UK in regard to arrangements for primary care that did
not permit us to undertake a preliminary assessment of the eligibility of
men for screening before we randomised them and issued half invitations to
attend for the ultrasound examination. As such an assessment was possible
in the British trials, direct comparison of response fractions between the
studies undertaken in the two countries is not valid. In the Australian
trial, the response to invitations was on a par with that for mammographic
screening, and there is good, population-based evidence that the latter
programme has had a clinically important impact on the presentation of
breast cancer (Jamrozik K, et al. The impact of mammographic screening on
breast cancer in Western Australia. Med J Aust 2000;172:203-206.)
The MASS trial indicates that screening should be introduced in the
UK. Our results should not undermine this. One of the reasons our
discussion focussed on differences between our trial and MASS was that the
effectiveness of screening may vary according to health care setting. Our
contention is that, having been generated in an importantly different
setting, the Australian results can be used to support the case for
establishment of carefully-designed screening programmes.
Competing interests:
None declared
Competing interests: No competing interests