Papers

Formal measurement of clinical uncertainty: prelude to a trial in perinatal medicine

Institute of Epidemiology and Health Services Research, Leeds LS2 9LN.

Clinical trials are ethical when there is collective and balanced uncertainty about the best treatment among informed clinicians. This uncertainty can be identified either by observing differences in practice or, as explored here, by formal measurement of clinical belief.

Fetomaternal medicine is largely concerned with timing delivery of preterm fetuses that are failing to thrive. Delivery is indicated when prematurity is judged preferable to continued intrauterine life, but often the relative risks of these alternatives are unknown.¹ During the planning of a trial to compare early with delayed delivery under different circumstances 10 specialists in fetomaternal medicine were interviewed about their beliefs.

Methods

and results

The specialists were presented with four scenarios, one practice and three substantive, in each of which the decision between immediate and delayed delivery was difficult (see figure). They recorded their opinions on an analogue dial connected to a microcomputer. For each scenario respondents were asked what they thought the relative risk of permanent morbidity was most likely to be in a hypothetical and infinitely large randomised trial in similar patients. An answer of 1 indicated that their best guess was that such a trial would show no difference between immediate or delayed delivery, an answer of 0.5 that the chance of morbidity would be halved by immediate delivery, and an answer of 2 that it would be doubled. Respondents were then asked what they would regard as a surprisingly good or bad result in the hypothetical trial - that is, when they believed that there was only a one in 40 (2.5%) chance of a result being equally or more extreme. These ranges of expected results gave a measure of the respondents' confidence in their own beliefs. To familiarise participants the answers were displayed during the practice scenario, but they were concealed for the three substantive scenarios.

For each scenario the mean result considered to be most likely was close to 1, with a wide scatter in the individual results - for example, from a 75% decrease to a 25% increase in the risk to a fetus delivered early in the practice scenario (figure). This shows that the scenarios caused collective uncertainty. The mean of all the estimates of surprisingly good and bad results was roughly a reduction and increase in risk of around 50% respectively. All 10 respondents had individual ranges of expected results that included 1 on scenario 1. One was included for six respondents on scenario 2 and for eight respondents on scenario 3.

Singleton Pregnancy, 32 weeks' gestation; blood pressure 160/100 mm hg; proteinuria 1g/l; no maternal symptoms; normal cadiotocogram; no end diastolic flow in umbilical artery on Doppler untrasonography; liqourvolumeslightlyreduced(aminoticindex = 6); other biophysical variables normal; steroids taken for 48 hours; regional neonatal unit available

As in practice scenario but 26 weeks' gestation; cardicotogram showing unreactive fetus; no end diastolic flow on Doppler ultrasonography;liquorvolumeseverlyreduced(aminoticindex = 3); fetal limb movements and breathing reduced; fetal tone abnormal

As in practice scenario but reduced end diastolic flow in umbilical arteryonDopplerultrasonography(systolic/diastolic = 4.5) and liquorvolumes slightly reduced (amniotic index = 6)

View larger version (55K): [in this window] [in a new window]   As in practice scenario but 28 weeks' gestation; reduced end diastolic flow on umbilical artery on Doppler ultrasonography (systolic/diastolic = 4.5); liqourvolumereduced (aminoticindex = 5); fetallimb movements and breathing reduced; fetal tone abnormal

Comment

These results show that experts do not agree about the benefit of delivery for preterm fetuses that are failing to thrive but are not thought to be near to death. For babies such as those described there is collective and reasonably balanced uncertainty² - the main requirement for a randomised trial.³ It is fortunate that clinicians given the same inconclusive information form different views because these differences provide the impetus for clinical trials. Our results also show, however, that some clinicians are in two minds when others have strong expectations of either benefit or harm. This makes agreement on fixed entry criteria unlikely and suggests that individual balanced uncertainty should be the entry criterion, a method already adopted in some major trials.⁴ The measurement of experts' previous belief and their confidence in this previous belief is also suitable for bayesian analysis of clinical trials, which takes account of opinions based on information external to the trial.⁵

Members of the Fetal Compromise Group are S Gudmundsson (Malmo General Hospital, Sweden), D James (Queen's Medical Centre, Nottingham), R Lilford (University of Leeds), G Mason (the General Infirmary, Leeds), K Neales (North Manchester General Hospital), M Pearce (St George's Hospital, London), C Plazzotta (University of Turin), D Spiegelhalter (MRC Biostatistics Unit, Cambridge), J Thornton (the General Infirmary, Leeds), T Todros (University of Turin), and J Walker (University of Glasgow).

The workshop on which this paper was based was funded by the Wellcome Trust and Roussel Pharmaceuticals.

1. Odendaal HJ, Pattinson RC, Bam R, Grove D, Kotze JT. Aggressive or expectant management for patients with severe pre-eclampsia between 28-34 weeks gestation: a randomized controlled trial. Obstet Gynecol 1990;76:1070-5. [Abstract/Free Full Text]
2. Johnson N, Lilford RJ, Brazier W. At what level of collective equipoise does a clinical trial become ethical? J Med Ethics 1991;17:30-4.
3. Collins R, Doll R, Peto R. Ethics of clinical trials. In: Williams CJ, ed. Introducing new treatment for cancer: practical, ethical and legal problems. Chichester: Wiley, 1992:49-65.
4. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group Randomised trial of intravenous streptokinase, oral aspirin, both or neither among 17 187 cases of suspected acute myocardial infarction. Lancet 1988;ii:349-53.
5. Pocock SJ, Hughes MD. Estimation issues in clinical trials and overviews. Stat Med 1990;9:657-71. [Medline]