Review of NICE's recommendations, 1999-2005

NICE recommendations

NICE appraises the clinical and cost effectiveness of health technologies referred to it by the Department of Health. This is different from a scientific assessment and synthesis of the evidence, which is subcontracted to independent academic groups. The two NICE committees appraise this often incomplete evidence. The committees rely on the judgments of their members, who comprise clinicians, health scientists, managers, and patient representatives.

NICE's recommendations are issued in the form of mandatory guidance to the NHS.10 It started controversially by recommending against the use of zanamavir, an antiviral drug for flu. However, reviews of the guidances issued in the first years show that few recommendations can be classified as simply yes or no.11^–14

Despite suggestions to the contrary,11 13 15 NICE has repeatedly stated that it does not have a threshold at which cost effectiveness becomes unacceptable. However, it has clarified that when the cost per quality adjusted life year (QALY) is above £20 000 (€29 000; $37 000), “Judgements about the acceptability of the technology as an acceptable use of NHS resources are more likely to make more explicit reference to factors including the range of uncertainty surrounding the calculation, the innovative nature of the technology, the particular features of the condition and population receiving the technology, and where appropriate the wider societal costs and benefits. Above an incremental cost effectiveness ratio of £30 000/QALY, the case for supporting the technology on these factors has to be increasingly strong.”16

Yes but

Credit: AKG IMAGES

NICE is unique internationally in having a formal appeals process. Stakeholders, including companies and clinical organisations, can appeal against its findings on the grounds of process (due process), perversity (given the evidence), or powers (exceeding its powers). Appeals are heard by a panel composed largely of non-executive members of NICE and industry and patient representatives.

Review of guidance

I reviewed the guidance issued by NICE to the NHS between 1999 and 2005. Some appraisals included more than one technology, and some technologies could be used in different groups of patients. For each guidance, I identified one or more patient technology topics and classified the recommendation for each topic as yes, yes with major restrictions, yes with minor restrictions, and no (box). Restrictions for drugs were relative to licence, and for non-pharmaceuticals, relative to the size of the potential patient group. The classification of each guidance was validated by nine experts (see acknowledgments). I also noted the cost effectiveness (incremental cost per QALY) associated with each recommendation and the results of any appeals.

The 86 guidances published up to April 2005 covered 117 technology or patient topics (see bmj.com). Recommendations were fairly evenly distributed across the four categories, with NICE deciding no for 22 (19%), yes for 27 (23%), yes with major restrictions for 38 (32%) and yes with minor restrictions for 30 (26%).

Of the negative recommendations, almost two thirds were on the grounds of insufficient evidence, the rest were because of unacceptable cost effectiveness. The recommendations for use with major restrictions generally imposed restrictions to improve cost effectiveness. The recommendations with minor restrictions usually specified good clinical practice (monitoring required, use by specialist) but sometimes also recommended use of the lowest cost equivalent technology.

Acceptable cost effectiveness

The highest cost per QALY that NICE has accepted is an estimated £39 000 (range £35 000-43 000) for riluzole to treat motor neurone disease. The guidance noted “the values which patients place on the extension of tracheostomy free survival time.”17 For trastuzumab for advanced breast cancer, NICE cited the company's estimate of £37 500 per QALY but considered that the company's assumptions were unduly pessimistic.18 With imatinib for chronic myeloid leukaemia, NICE initially accepted a cost per QALY from £22 000 to £56 00019 but later reduced this to £26 000, which it calculated using a different comparator.20

Interferon beta (and glatiramer acetate) for multiple sclerosis were deemed not cost effective at an incremental cost per QALY of £35 000-£104 000 (estimated mean £70 000). The government then intervened with a risk sharing scheme with a cost effectiveness threshold “set, for the purpose of this scheme only, at £36 000.”21 Under the scheme, eligible patients who consent have their clinical progress monitored against that required to meet the target cost effectiveness. Drug prices would be reduced for patients whose progress fell below the target. The scheme, established in 2002, had recruited more than 5000 patients by 2005. No reports of its progress have been published, but it will be interesting to see how well the scheme succeeds in achieving its target level of cost effectiveness.

Although NICE does not officially prioritise interventions that save lives over those that improve quality of life, its treatment of some topics suggests the rule of rescue,22 or prioritising life saving therapies, may play a part. With cancer drugs such as imatinib and trastuzumab, which extend life expectancy, NICE accepted relatively poor cost effectiveness. However, the acceptance of riluzole was based on considerations of quality of life rather than on mortality.

Consistency

Some topics have been appraised several times, partly because of appeals. Antiviral drugs for flu have had three appraisals as well as a rapid review in 1999. Each time NICE reiterated its recommendation against use of these drug by healthy people but in favour of their use in vulnerable groups. Three obesity treatments (two medical and one surgical) have been appraised. NICE concluded for each that they should be used only in people with proved determination and appropriate progress on treatment.

Appeals

NICE's 86 guidances have been subject to 25 appeals (29%). Fifteen were dismissed. Of the 10 appeals that were upheld, five resulted in relatively minor changes in the wording of the guidance. But five decisions (interferon beta in multiple sclerosis, drugs for colorectal cancer, flu antivirals, growth hormone in adults, and renal immunosupression in adults) were referred back to the appraisal committee for further appraisal. The appeals process has required NICE to show that it has been comprehensive in its examination of the evidence and consistent in its treatment of each topic.

Discussion

At its current rate of appraisal—around 20 a year—NICE can cover only a minority of new and existing treatments. This led to announcements in late 2005 of a more rapid review process.23 However, a more rapid process is likely to be considerably less intensive. The appraisal of drug treatments for multiple sclerosis, for example, took much of NICE's first two years, with 338 documents listed on its website. It eventually recommended against use of interferon beta and glatiramer acetate because of their high cost per QALY. Despite considerable effort, including additional research, NICE was unable to identify a subgroup of patients in whom these drugs might have a more acceptable level of cost effectiveness. The fact that the government then intervened with a special purchase scheme based on a cost per QALY gained of £36 00021 indicated that the government thought this was an acceptable level, at least for these drugs.

Overall NICE must be judged to have succeeded in surviving some controversial decisions. Its appeal system has imposed consistency and has so far prevented appellants proceeding to legal challenge. Although clinicians have understandably feared blanket restrictions, these have been fairly rare. NICE continues to be best characterised not by saying no, but by saying yes but…