BMJ  2005;330:581 (12 March), doi:10.1136/bmj.330.7491.581

Information in practice

Effect of electronic health records in ambulatory care: retrospective, serial, cross sectional study

¹ Clinical Systems Planning and Consulting, Kaiser Permanente, 2101 Webster Street, Oakland, CA 94612, USA

Abstract

Objective To evaluate the effect of implementing comprehensive, integrated electronic health record systems on use and quality of ambulatory care

Design Retrospective, serial, cross sectional study.

Setting Colorado and Northwest regions of Kaiser Permanente, a US integrated healthcare delivery system.

Population 367 795 members in the Colorado region and 449 728 members in the Northwest region.

Intervention Implementation of electronic health record systems.

Main outcome measures Total number of office visits and use of primary care, specialty care, clinical laboratory, radiology services, and telephone contact. Health Plan Employer Data and Information Set to assess quality.

Results Two years after electronic health records were fully implemented, age adjusted rates of office visits fell by 9% in both regions. Age adjusted primary care visits decreased by 11% in both regions and specialty care visits decreased by 5% in Colorado and 6% in the Northwest. All these decreases were significant (P < 0.0001). The percentage of members making 3 visits a year decreased by 10% in Colorado and 11% in the Northwest, and the percentage of members with 2 visits a year increased. In the Northwest, scheduled telephone contact increased from a baseline of 1.26 per member per year to 2.09 after two years. Use of clinical laboratory and radiology services did not change conclusively. Intermediate measures of quality of health care remained unchanged or improved slightly.

Conclusions Readily available, comprehensive, integrated clinical information reduced use of ambulatory care while maintaining quality and allowed doctors to replace some office visits with telephone contacts. Shifting patterns of use suggest reduced numbers of ambulatory care visits that are inappropriate or marginally productive.

Introduction

Widescale development of electronic health record systems has been repeatedly recommended in the United States,^{1 2 3} but only about 5% of US primary care providers use them.⁴ Illegible or unavailable paper records may necessitate conservative management strategies and redundant or marginally productive visits, diagnostic and screening tests, and interventions. Electronic health records provide greater accessibility, accuracy, and completeness of clinical information.³ Two Kaiser Permanente regions separately implemented comprehensive electronic health record systems. We examined their effect on selected measures of use and quality of ambulatory care.

Methods

Kaiser Permanente is the largest, not for profit, integrated healthcare delivery system in the United States. It provides all healthcare needs for adults and children, including preventive, routine, specialty, emergency, and inpatient care, ancillary testing, pharmacy and rehabilitative services, and home care. Two Kaiser Permanente regions, Colorado and the Northwest, implemented electronic health record systems (table).

View this table: [in this window] [in a new window]   Characteristics of Colorado and Northwest healthcare regions

 

Electronic health record systems
Before implementation, individual paper based medical records were manually delivered to multiple sites throughout the integrated Kaiser Permanente system, often several miles apart; availability of records for same day and unscheduled care was unreliable.

Although the regions implemented different systems (an internally developed system in Colorado and an externally supplied system in the Northwest region) they shared important characteristics:

• Integrated documentation and reporting of clinical results reporting, including comprehensive recording of use of primary and specialty care, telephone contact, urgent care, and emergency departments
  
• Computerised entry of physician orders for tests and prescriptions
  
• 24 hour availability of medical records at the point of care
  
• Immediate availability for all potential users—for example, staff in telephone advice centres, pharmacists, and staff reporting clinical results
  
• Easy searching.
  

Design
We conducted retrospective serial cross sectional studies for each region. We assessed usage from administrative data and quality of care from the Health Plan Employer Data and Information Set. This nation-wide information set is a series of standardised performance measures covering all population needs and pressing public health problems. Results are based on statistically valid random samples of members and are rigorously audited.

The electronic health records were introduced progressively over more than a year and in different calendar years in the two regions. We reviewed data for entire calendar years, delimiting the implementation period by the calendar year in which primary care implementation began (year I¹) and the calendar year in which it was completed (year I²). Given natural lags in implementation and impact, the baseline period includes year I¹ and the intervention period begins in year I².

We conducted interviews with roughly 100 clinical and operations leaders and staff. The interviews suggested the electronic system increased the efficiency of ambulatory care visits and reduced redundant or marginally productive healthcare services by allowing patient issues to be resolved during the first contact, enabling more services to be offered per visit and reducing the need for separate health maintenance visits. Informants also reported reduced use of clinical laboratory, radiology, and emergency services and increased effectiveness of scheduled and unscheduled telephone contact with patients.

To evaluate these impressions, we examined total office visits and separated use into primary and specialty care. We included visits to doctors and other health professionals. We examined the frequency distribution of primary care visits. Data on total office visits and use of emergency departments, radiology departments, clinical laboratories, and telephone call centres were obtained from regional data warehouses.

To assess quality of care, we selected three measures from the information set that represented activities in primary and specialty care, care of routine and chronic conditions, and care requiring a cross departmental referral. These were percentages of relevant members receiving advice on stopping smoking, screening for cervical cancer, and retinal examinations in diabetes.

Statistical analysis
We stratified the annual rates of total office visits per region by primary care and specialty care and adjusted to a fixed age distribution over the study period (0-64 years and 65). We calculated the difference between the age adjusted visit rates in year I¹ and the observation period.

Year I¹ was selected as the baseline year for several reasons. In keeping with national trends, use of ambulatory care had been rising at Kaiser Permanente before electronic health records were introduced (figure 1). Reasons include changing reimbursement mechanisms, policies to review usage, and new technologies.⁵ Year I¹ represents maximum capture of this upward trend. It also represents the last point of substantially steady state operational practices in ambulatory care. Although implementation began in year I¹, most of the implementation of electronic records occurred in year I².

View larger version (30K): [in this window] [in a new window]   Rates of office visits adjusted for age before and after introduction of electronic health records in primary care

 

We selected year 2 as the primary comparison year because it was the last point for which consistent data from both regions were available. Regional measures of use of ambulatory care subsequently changed.

Results

At the time of assessment, the Colorado and Northwest regions had two and four years' experience, respectively, with electronic health records.

Use of ambulatory care
Both regions had significant decreases in use of services. The age adjusted number of total office visits per member in year 2 decreased by 9% compared with year I¹ (P < 0.0001, in both regions), and age adjusted primary care visits decreased by 11%. Age adjusted specialty care visits decreased by 5% in Colorado and 6% in the Northwest (both P < 0.0001). In year 4, the total office visit rate in the Northwest region was 8% lower than before electronic records became available (fig 1). Partial implementation had minimal effect during year I¹.

The frequency pattern of ambulatory primary care visits suggested a general decrease in use across all patients. The percentage of members making three or more visits a year decreased by 10% in the Colorado region and 11% in the Northwest region between year I¹ and year 2. In year 4, the rate in the Northwest region had decreased by an additional 2%. Moreover, the percentage of members with 2 visits a year increased. This finding is particularly striking in Colorado, since a disproportionate number of people aged 65 and over were enrolled during the study period, and is consistent with the effects of electronic health records described by clinical and operational leaders.

We reviewed other data that could potentially explain decreased use of ambulatory care. Rates of visits to emergency departments (internal and external to Kaiser Permanente) did not rise over the study, and the ratio of all primary care providers to members and the ratio of referrals to outside providers in both regions throughout the study both remained stable.

To rule out other global influences, we examined the rate of change in office visits, as independently defined by three other Kaiser Permanente regions, for the same period. The data did not show comparable decreases. The rate of ambulatory care visits by people aged 45 or older increased by 14% across the United States between 1992 and 2002, which encompasses our study period.⁶

Telephone contact
In the Northwest region, telephone encounters scheduled at the discretion of physicians increased from a baseline of 1.26 per member per year to 2.09 after two years. In the Colorado region, staffing of call centres briefly shifted from primarily nursing staff to include doctors with access to electronic health records. Appointments needed by patients after telephone contact with access to electronic health records decreased by 7%. Doctors reported being able to resolve more health issues by phone. Rates of appointments after telephone contact rose when staffing reverted to nurses.

Radiology and clinical laboratory services
Age adjusted rates of use of radiology services decreased by 14% in the first two years after introduction of electronic health records in the Northwest region. Despite more recent increases in general use of imaging inside Kaiser Permanente and industry-wide, the age adjusted rate remained 4% lower than before implementation. The chief of radiology in the Colorado region believed strongly that availability of electronic records to all carers improved interpretation of films.

Laboratory usage in the Northwest region had decreased by 18% four years after electronic health record were introduced; rates subsequently increased 5-7% annually. Rates of laboratory usage in the Colorado region remained generally stable, rising 14% before electronic health records were introduced and falling 2.9% in the two subsequent years.

Quality of care process measures
Quality of health care, as assessed by the three predefined measures, remained unchanged or slightly improved after electronic health records were introduced (see bmj.com). This allays any fleeting concerns that decreased usage compromised quality of care.

Discussion

We compared use of ambulatory care before and after the introduction of electronic health records and found decreased use of both primary and specialty services. In addition, the percentage of members with more than three visits a year fell. Telephone contacts supplanted some outpatient visits and we found stable or improved quality of care measures.

Possible explanations
The observed changes have many potential explanations. Efficiency of outpatient care may have been increased by the readily available comprehensive clinical information. This conclusion is supported by the shifting frequency distribution of visits, the increased use of phone contact in lieu of a visit, and doctors' consistent observations across the two regions that electronic health records enabled them to identify and resolve patients' health issues in the first contact or with fewer contacts.

We were able to eliminate several confounding factors as potential causes of reduced use of ambulatory care: changes in age distribution, reduced practitioner availability (as measured by the ratio of all primary care providers to members), and possible shifting of primary care to specialty, emergency, or outside care (as measured by rates of visits and outside referral rates).

Organisational pressure to reduce use of ambulatory care could have caused similar effects. Despite differing operating systems and organisational goals, neither region had an implicit or explicit goal to reduce outpatient visits during the study. No meaningful changes occurred in copayments for primary care or specialty care, and neither the health plan products nor the products offered by employer purchaser groups changed substantially during the study period.

Applicability
Neither region represents an unusual patient population and both have an average or above average illness burden.

Some questions about electronic health records remain unanswered. For instance, informants expected that improved availability of complete laboratory data would eliminate redundant testing, yet we found no conclusive evidence of this. Decreases in use of laboratory services were neither consistent in the two regions nor sustained over time. However, before the availability of electronic health records there may also have been underuse of some tests (such as lipid screening).

Effect on doctors
Some research indicates that electronic health record systems impose a greater burden on clinicians.^7-9 This effect may be temporary¹⁰ or situation dependent.¹¹ However, internal Kaiser Permanente work indicates that the effect is highly variable but time neutral on average (unpublished data). Additionally, Geisinger Health System noted a significant improvement in productivity after introducing electronic records (personal communication, 2003).

Interviews with clinical and operations leaders indicated that strong leadership support for realising potential efficiency gains was key to successful implementation. Organisational structure supporting free flow of information and efficiency gains is key to realising the benefits of electronic health record implementation.¹²

Effect on quality
Despite perceptions of improved quality of care after electronic health records were introduced, we found generally stable and only occasionally improvedperformance on selected measures. Both regions are high performing, which may make it more difficult to identify marginal quality improvements. At a minimum, we can assert that electronic health records and the resulting effects on usage do not reduce the quality of care and may in fact increase appropriate use of healthcare services.

What is already known on this topic Electronic medical records have been shown to improve the quality of health care in specific areas Their effect on overall use and quality of ambulatory care is unknown What this study adds Introduction of electronic records reduced visits to primary and specialist outpatient care in two regions of a US health maintenance organisation No concomitant increase in use of other services was found Selected measures of quality of care were unaffected

---

This is the abridged version. The full version is on bmj.com

We thank the many physicians, operations leaders, and analysts in the Kaiser Permanente Colorado and Northwest regions for their contribution to this study. In particular, we thank Homer Chin, Nan Robertson, and Marianne Gapinski for their sponsorship and insights and Jenni Green for advice and help in writing this article.

Contributors: See bmj.com

Funding: Kaiser Foundation Health Plan and Hospitals.

Competing interests: None declared.

Ethical approval: Not required.

References

1. Bates DW, Ebell M, Gotlieb E, Zapp J, Mullins HC. A proposal for electronic medical records in US primary care. J Am Med Inform Assoc 2003;10: 1-10.
2. Committee on Improving the Patient Record. The computer-based patient record: an essential technology for health care. Washington, DC: National Academy Press, 1997.
3. General Accounting Office. Medical ADP systems: automated medical record systems hold promise to improve patient care. Washington, DC: GAO, 1991.
4. Anderson, JD. Increasing the acceptance of clinical information systems. MD Comput 1999;16(1): 62-5.[Medline]
5. Duffy SQ, Farley DE. Patterns of decline among inpatient procedures. Public Health Rep 1995;110: 674-81.[ISI][Medline]
6. Woodwell DA, Cherry DK. National ambulatory medical care survey: 2002 summary. Adv Data 2004 Aug 26;(346): 1-44.
7. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations: effects on resource utilization. JAMA 1993;269: 379-83.[Abstract]
8. Penrod LE, Gadd CS. Assessing physician attitudes regarding use of an outpatient EMR: a longitudinal, multi-practice study. Proc AMIA Symp 2001: 528-32.
9. Krall MA. Acceptance and performance by clinicians using an ambulatory electronic record in an HMO. Annu Symp Comput Appl Med Care 1995: 708-11.
10. Keshavjee K, Troyan S, Holbrook AM, VanderMolen D. Measuring the success of electronic medical record implementation using electronic and survey data. Proc AMIA Symp 2001: 309.
11. Bates DW, Studer J, Reilly, CA, Cureton, EA, Spurr, CD, Kuperman GJ. Evaluating the impact of a computerized ambulatory record. Proc AMIA Symp 2000: 964.
12. Bates DW, Ebell M, Gotlieb E, Zapp J, Mullins HC. A proposal for electronic medical records in US primary care. J Am Med Inform Assoc 2003;10: 1-10.

(Accepted 31 January 2005)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study

J Tim Scott, Thomas G Rundall, Thomas M Vogt, and John Hsu
BMJ 2005 331: 1313-1316. [Abstract] [Full Text] [PDF]

Electronic health records reduce office visits
BMJ 2005 330: 0. [Full Text]

This article has been cited by other articles:

• Lo, H. G., Newmark, L. P., Yoon, C., Volk, L. A., Carlson, V. L., Kittler, A. F., Lippincott, M., Wang, T., Bates, D. W. (2007). Electronic Health Records in Specialty Care: A Time-Motion Study. J. Am. Med. Inform. Assoc. 14: 609-615 [Abstract] [Full text]  
• Classen, D. C., Avery, A. J., Bates, D. W. (2007). Evaluation and Certification of Computerized Provider Order Entry Systems. J. Am. Med. Inform. Assoc. 14: 48-55 [Abstract] [Full text]  
• Sidorov, J. (2006). It Ain't Necessarily So: The Electronic Health Record And The Unlikely Prospect Of Reducing Health Care Costs.. Health Aff (Millwood) 25: 1079-1085 [Abstract] [Full text]  
• Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., Morton, S. C., Shekelle, P. G. (2006). Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care. ANN INTERN MED 144: 742-752 [Abstract] [Full text]  
• Scott, J T., Rundall, T. G, Vogt, T. M, Hsu, J. (2005). Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study. BMJ 331: 1313-1316 [Abstract] [Full text]  
• (2005). How Do Electronic Health Records Affect Ambulatory Care?. Journal Watch Dermatology 2005: 13-13 [Full text]  
• (2005). How Do Electronic Health Records Affect Ambulatory Care?. JWatch General 2005: 5-5 [Full text]  

Rapid Responses:

Read all Rapid Responses

Overlooked

Mira de Vries
bmj.com, 17 Mar 2005 [Full text]

Re: Overlooked--'Would Ethical Approval be required for this study in UK?

susanne mccabe
bmj.com, 17 Mar 2005 [Full text]

Anonymous clinical data repository technology negates the need for consent from research subjects

Derrick C Attfield
bmj.com, 24 Jun 2005 [Full text]