BMJ  2003;327:E96-E98 (4 October), doi:10.1136/bmjusa.02030003 (published 28 August 2002)

BMJ USA: Editorial

Whose health care is more efficient?

Fuzzy math obscures comparison of US and UK

This article originally appeared in BMJ USA

A longstanding debate in quality improvement has been whether providers need more resources to improve the process and outcomes of care or whether they can achieve these goals by using their resources more efficiently. Political and ethical considerations make it unrealistic to expect a randomized study to resolve the issue, and thus observational studies with comparison groups offer the best alternative. When researchers examine this question in the UK, which has only one system of care (the National Health Service, NHS), the comparison group, by necessity, has to be from another country. One such comparison was published recently by Feachem et al in the BMJ.¹ The comparison group they selected for the NHS was the California Kaiser Health Plan, citing similarities in the age of the health systems and the services they provide. (The full text of the study is available at http://bmj.com/cgi/content/full/324/7330/135.

The study findings received considerable publicity, perhaps because they seemingly refuted the popular notion that American health care is less efficient. The authors reported that patients in Kaiser wait on average less time for specialist consultations and for elective procedures as compared with patients in the NHS (see Table 1). Kaiser also appeared to be superior to the NHS in reported performance measures for diabetes and cardiovascular care. Citing their finding that Kaiser uses less than a third of the hospital bed days per capita as the NHS (270 versus 1000 per 1000 persons), the authors suggested that Kaiser achieves these quality benefits by redirecting hospital budgets toward prevention and ambulatory care. These findings, coupled with the comparisons of costs across the two systems (see Table 2), suggested that there are unrealized opportunities to improve the NHS not by investing more funds but by improving how they are spent.

View this table: [in this window] [in a new window]   1 Comparing responsiveness of NHS and Kaiser to consumers, 2001

 

View this table: [in this window] [in a new window]   1 Comparison of costs in the NHS and Kaiser

 

Several methodological problems raise questions about the validity of the results on which the conclusions are based. First, the investigators' attempt to create equivalent comparison groups by adjusting their cost estimates for the greater age and lower income of patients in the NHS than in Kaiser was inadequate. US health plans are known to select healthier subgroups of patients, regardless of their age or socioeconomic class.² The NHS, like public delivery systems in the US, is designed to accept all patients, regardless of their health or ability to pay. Second, the quality of the performance data used to contrast the two health care systems is somewhat suspect. The Kaiser data come from multiple years and from a pool of patients that extends beyond the group that was used to determine costs. The methods for collecting information on quality were neither standardized nor audited across the health systems, nor were the results adjusted for the large measured differences in the patient populations. Third and most importantly, in attempting to standardize the purchasing power of each system's currency in the health sector, the authors adjusted away real purchasing advantages the NHS enjoys as a single purchaser with enormous size and clout in the marketplace. Several rapid responses posted on bmj.com noted that this adjustment erroneously shrank the apparent difference in per capita health care costs between the two systems from $759 to $187. ^3-5 The over-adjusted results suggest that NHS costs are 90% of Kaiser's $1951 per capita, when a more traditional economic comparison reveals the NHS per capita costs to be $1192, or 61% of Kaiser's. The NHS is in reality a much less expensive system than Kaiser.

This study, with its methodological shortcomings, may have the untoward effect of giving ammunition to providers who already feel resistant to having their practice and performance scrutinized. Rather than using cost and performance data to stimulate quality improvement along the lines promoted by Berwick in an editorial that accompanied the study,⁶ providers might instead feel justified in dismissing these sorts of data because of how easily they appear to be manipulated. Health care improvement requires an open examination of practice. The inherent variation in the US health care system provides innumerable natural experiments from which to learn how to make health care better. However, to create an atmosphere of fairness and to promote a process for learning, it is essential for scientific journals to ensure that researchers get the science right before publishing.

The study by Feachem et al carries an important message for clinicians and policymakers: systems of care, even those that are underfunded, have the potential to increase efficiency and make resources go further. That message is easily lost when it is defended with weak data, because methodological problems obscure the larger lesson. Similarly, evidence about ways to apply existing resources more efficiently is posed as a false choice when it is used as a counterargument to investing additional resources. Both strategies must often be blended together to achieve maximal gains in improving effectiveness and efficiency.

Department of Medicine, Epidemiology and Biostatistics, University of California San Francisco, San Francisco General Hospital, 1001 Potrero Avenue, Building 90, Ward 95, San Francisco, CA 94110 (bindman{at}itsa.ucsf.edu)

References

1. Feachem RGA, Sekhri NK, White KL. Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente. BMJ 2002;324:135-143 [Abstract/Free Full Text]
2. Hellinger FJ, Wong HS. Selection bias in HMOs: a review of the evidence. Med Care Res Rev 2000;574:405-439
3. Nashef SAM. Re: The Kaiser vs NHS comparison. January 20, 2002. Available at: http://bmj.com/cgi/eletters/324/7330/135#18858.
4. Blair SD. Kaiser fudge factor. January 22, 2002. Available at: http://bmj.com/cgi/eletters/324/7330/135#18934.
5. Grumbach K. "Getting what you pay for," not "getting more for your money." January 25, 2002. Available at: http://bmj.com/cgi/eletters/324/7330/135#19043.
6. Berwick DM. Same price, better care. BMJ 2002;324:142-143 [CrossRef]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Rapid Responses:

Read all Rapid Responses

Applying Best Practice

Gray Southon
bmj.com, 1 Sep 2002 [Full text]

response to kaiser being better than nhs

evelyn m kerr
bmj.com, 19 Sep 2002 [Full text]

Kaiser's Bill and his Big Apple

Roger K.A. Allen
bmj.com, 21 Sep 2002 [Full text]