nejm 0807651

8/18/2019 NEJM 0807651

1/11

special article

T he n e w e n g l a n d j o u r n a l o f medicine

n engl j med 361;4 nejm.org july 23, 2009368

Effects of Pay for Performance

on the Quality of Primary Care in EnglandStephen M. Campbell, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D.,

Bonnie Sibbald, Ph.D., and Martin Roland, D.M.

From the National Primary Care Researchand Development Centre, University ofManchester, Manchester (S.M.C., D.R.,

E.K., B.S., M.R.); and the University ofCambridge General Practice and PrimaryCare Research Unit, Institute of PublicHealth, Cambridge (M.R.) — both in theUnited Kingdom. Address reprint requeststo Dr. Campbell at the National PrimaryCare Research and Development Centre,University of Manchester, Oxford Rd.,Manchester M13 9PL, United Kingdom, orat [email protected].

N Engl J Med 2009;361:368-78.Copyright © 2009 Massachusetts Medical Society.

A b s t r a c t

Background

A pay-for-performance scheme based on meeting targets for the quality of clinicalcare was introduced to family practice in England in 2004.

Methods

We conducted an interrupted time-series analysis of the quality of care in 42 repre-sentative family practices, with data collected at two time points before implemen-tation of the scheme (1998 and 2003) and at two time points after implementation(2005 and 2007). At each time point, data on the care of patients with asthma, diabetes,or coronary heart disease were extracted from medical records; data on patients’perceptions of access to care, continuity of care, and interpersonal aspects of care were collected from questionnaires. The analysis included aspects of care that wereand those that were not associated with incentives.

Results

Between 2003 and 2005, the rate of improvement in the quality of care increased forasthma and diabetes (P

8/18/2019 NEJM 0807651

2/11

Effects of Pay for Performance on the Quality of Primary Care in England

n engl j med 361;4 nejm.org july 23, 2009 369

In 2004, the U.K. government intro-

duced a pay-for-performance scheme with 136indicators for family practices. The indicators

covered the management of chronic disease, prac-tice organization, and patients’ experiences withrespect to care.1 In 2006, revisions to the schemeadded seven new clinical areas, including demen-

tia and chronic kidney disease, and two new indi-cators of patient access to care (see the Supplemen-tary Appendix, available with the full text of thisarticle at NEJM.org).2 Payments make up approx-imately 25% of family practitioners’ income, and99.6% of family practitioners participated in thepay-for-performance scheme, which is voluntary.

We have previously reported on the quality ofclinical care in 2005, the year after the pay-for-performance scheme was introduced.3 We founda modest acceleration in the rate of improvementin the quality of care for asthma and diabetes

but not for heart disease. There had been rapidimprovement in the quality of care for all threeconditions before the introduction of pay for per-formance. This article extends these analyses toinclude performance data in 2007. We used aninterrupted time-series analysis to examine trendsin the quality of clinical care from 1998 through2007, a period spanning the introduction of payfor performance. We also report on trends inpatient reports on communication with their phy-sician, on access to care, and on continuity ofcare across the same period.

Methods

Data Collection

Trained research staff abstracted clinical datafrom the medical records kept by 42 nationallyrepresentative family practices. In each practice,data were collected for nonoverlapping randomsamples of patients (20 in 1998 and 12 each in2003, 2005, and 2007) who had heart disease,asthma, or diabetes; the data were collected with

the use of quality indicators.4,5

The methodsused to collect data in 2007 were consistent withthe methods used in 1998, 2003, and 2005.3

For patient evaluation, a version of the GeneralPractice Assessment Questionnaire (www.gpaq.info) was mailed, with one follow-up reminder,to a random sample of 200 registered adult pa-tients (age, ≥18 years) in each practice.6,7 Rapidaccess to any doctor within 48 hours was associ-ated with an incentive under the pay-for-perfor-

mance scheme, and our questionnaire includedtwo items addressing the patient’s ability to getan appointment within 48 hours with “any doc-tor” and with “a particular doctor.” Because ofconcern that the scheme’s focus on clinical indi-cators might lead practitioners to neglect otheraspects of care,8 we also analyzed communication

with physicians and continuity of care. Commu-nication was assessed by asking seven questions, with the answers scored on a six-point scaleranging from “very poor” to “excellent”; continu-ity of care was assessed with the use of the samesix-point scale and a single question: “How oftendo you see your usual doctor?” All scores wererescaled to range from 0 to 100. The rate of re-sponse to the survey was 38% in 1998, 47% in2003, 45% in 2005, and 38% in 2007. In all cases,higher scores indicate higher quality of care. Theresearch protocol was approved by the North

West Research Ethics Committee.

Statistical Analysis

As we had done previously,3,9,10 we computedan overall clinical quality score for each patientin 1998, 2003, 2005, and 2007, which was basedon the number of indicators for which appropri-ate care was provided, divided by the number ofindicators relevant to that patient. This score rep-resents the percentage, from 0 to 100%, of “nec-essary” or “indicated” care provided to the patient.Practice-level quality scores were computed as themean of individual patient scores in each prac-tice. We computed separate quality scores for thesubgroups of indicators that were assigned incen-tives under the pay-for-performance scheme andfor the subgroups that were not assigned incen-tives.

Data on quality of care had been collected inthe same practices in 1998.9 When a pay-for-performance scheme was announced for com-mencement in 2004, we designed an interruptedtime-series study whereby data on quality of care

would be collected at two points before thescheme was introduced (1998 and 2003) and attwo points after its introduction (2005 and 2007).We use the term “pre-introduction period” to re-fer to the period from 1998 through 2003, “in-troduction period” for 2003 through 2005 (fromthe year before to the year after the implementa-tion of pay for performance), and “post-introduc-tion period” for 2005 through 2007.

We analyzed the data as an interrupted, or

The New England Journal of Medicine

Downloaded from nejm.org on April 20, 2016. For personal use only. No other uses without permission.

Copyright © 2009 Massachusetts Medical Society. All rights reserved.

8/18/2019 NEJM 0807651

3/11

Th e n e w e n g l a n d j o u r n a l o f medicine


segmented, time series. In this model, the within-practice variation was partitioned into three maincomponents to provide independent tests of theslope in scores for the pre-introduction period(test 1); the change in level during the introduc-tion period, allowing for the trend before pay forperformance (test 2); and the change in slope

from before to after pay for performance (test 3).11 Practice was treated as a random effect, and ro-bust standard-error estimates were used (see theSupplementary Appendix).

The analysis for each outcome measure wasconducted in two steps. In step 1, we used theinterrupted time-series analysis to look for evi-dence that pay for performance was having aneffect on the trend in scores over time, as indi-cated by a statistically significant result withrespect to either the change in level or thechange in slope (tests 2 and 3). The results of

these tests determined step 2: if the results ofneither test were significant, there was no evi-dence that pay for performance had affected thepreexisting trend and we conducted no furtheranalyses; if the results of either test were signifi-

cant, there was evidence of an effect and we in- vestigated this further by using the coefficientsfrom the time-series analysis to compare theimmediate- and long-term effects of the scheme(i.e., compare the slope during the introductionperiod with the slope during the post-introduc-tion period) and to estimate the size of the effect

on mean quality scores in 2005 and 2007.We compared the trends in quality scores for

the subgroups of indicators associated with in-centives and indicators not associated with incen-tives by means of interactions between indicatorset and the changes in level and slope (as definedabove) within a regression analysis. If either inter-action was significant, we took this as evidencethat the trends varied by indicator set and nexttested the interaction between indicator set andthe change in slope from the introduction periodto the post-introduction period.

The quality scores based on medical recordsand those based on patient evaluation are subjectto ceilings of 100%, and many practices achievedthis level on at least one indicator. The ceilingnecessarily limits any linear trend in improve-ment, since a score on quality cannot exceed100%. Analyses were therefore conducted onscores transformed to a logit scale, which has noceiling, as described previously.3 The transforma-tion increases the weight given to score changesnear the ceiling or floor — for example, scorechanges from 97 to 98% and from 55 to 65% arenumerically equivalent (0.41) after transformation.However, where possible, results are re-expressedin original units to facilitate interpretation.

To assess the sensitivity of the findings to ourstatistical assumptions, we varied the method ofstatistical inference with the use of a bootstrapmethod, using 1000 bootstrap samples, and weassumed a linear model for the trend by repeatingthe analysis on untransformed scores (for detailssee the Supplementary Appendix). We report anyresults that differ from those of the primary

analysis.

Results

Coronary Heart Disease

The quality of care for coronary heart disease hadbeen improving before the pay-for-performanceincentives were introduced (Tables 1 and 2 andFig. 1). The rate of increase was equivalent to anaverage of 3.5% per annum from 1998 through

Table 1. Mean Clinical-Quality Scores for 42 Family Practices in 1998, 2003,

2005, and 2007.*

Variable Mean Clinical-Quality Score

1998 2003 2005 2007

Clinical care

Coronary heart disease† 58.6±1.4 76.2±1.6 85.0±1.0 84.8±1.3

Asthma 60.2±2.5 70.3±2.5 84.3±1.8 85.0±1.4

Diabetes 61.6±1.8 70.4±1.5 81.4±0.8 83.7±0.7

Patients’ perceptions

Communication with physicians 69.4±1.0 70.5±1.4 69.1±1.6 71.3±1.2

Access to care (appointmentwithin 48 hr)

To see a particular physician 39.0±4.3 33.3±4.0 34.4±3.9 32.1±3.2

To see any physician 67.2±3.3 61.0±3.7 63.9±3.2 64.2±3.2

Continuity of care 70.7±1.7 70.3±1.7 66.2±1.8 66.0±1.6

* Plus–minus values are means ±SE. Data on the care of patients with asthma,diabetes, or coronary heart disease were extracted from medical records, anddata on patients’ perceptions of communication with physicians, access tocare, and continuity of care were obtained from questionnaires. Communi-cation was assessed by asking seven questions, with the answers scored on asix-point scale ranging from “very poor” to “excellent”; continuity of care wasassessed with the use of the same six-point scale and a single question: “Howoften do you see your usual doctor?” Access to care was scored as the per-centage of patients who reported that they were able to get an appointmentwithin 48 hours. All scores were rescaled to range from 0 to 100.

† Scores are shown for 40 of the 42 practices.




8/18/2019 NEJM 0807651

4/11



2003 (95% confidence interval [CI], 2.8 to 4.2;P

8/18/2019 NEJM 0807651

5/11



incentives as compared with those that were not.For heart disease, the scores for aspects of carethat were linked to incentives showed a biggerimmediate increase when the pay-for-performancesystem was introduced (P = 0.05), although thistrend was not significant as calculated in the lin-ear model (P = 0.46). The long-term trends (scores

in the post-introduction period vs. scores in thepre-introduction period) did not differ significant-ly (P = 0.06). However, the difference was signif i-cant when calculated with the use of the boot-strapping method (P = 0.05) or the linear model(P = 0.03), and in absolute terms, the mean qual-ity score for aspects of care for heart disease that

were not linked to incentives declined after 2005, whereas the quality score for care that was linkedto incentives increased. For asthma, the immedi-ate effect of pay for performance did not differbetween care that was and care that was notlinked with incentives (P = 1.00), but the trendssubsequently diverged (post-introduction period vs. pre-introduction period, P = 0.006; post-intro-duction period vs. introduction period, P = 0.05), with the mean score for care that was not linkedto incentives declining after 2005, and the meanscore for care that was linked to incentives in-

creasing. Trends in diabetes care did not differat any time according to whether the care waslinked to incentives.

Communication, Waiting Times,

and Continuity of Care

The percentages of patients able to see a physician within 48 hours, as well as the mean scores onthe physician-communication scale, showed nosignificant changes in trend. Continuity of caredeclined significantly after the introduction ofpay for performance (P

8/18/2019 NEJM 0807651

6/11



T a b l e 3 . M e a n C l i n i c a l - Q u a l i t y S

c o r e s i n 1 9 9 8 , 2 0 0 3 , 2 0 0 5 , a n d 2 0 0 7 , A c c o r d i n g t o I n d i v i d u a l C l i n i c a l I n d i c a t o r s . *

C o n d i t i o n

1 9 9 8

2 0 0 3

2 0 0 5

2 0 0 7

M e a n S c o r e

N o . o f

P r a c t i c e s

M e a n S c o r e

N o . o f

P

r a c t i c e s

M e a n S c o r e

N o . o f

P r a c t i c e s

M

e a n S c o r e

N o . o f

P r a c t i c e s

C o r o n a r y h e a r t d i s e a s e

C l i n i c a l i n d i c a t o r s

F r e q u e n c y o r p a t t e r n o f a n g i n a a t t a c k s ( p a s t 1 5 m o )

4 3

. 4 ± 2 . 2

4 0

6 0 . 3 ± 3 . 9

4 0

7 0 . 1 ± 3 . 4

4 0

5 9 . 7 ± 4 . 4

4 0

B l o o d p r e s s u r e ( p a s t 1 5 m o ) †

8 7

. 0 ± 1 . 7

4 0

9 5 . 3 ± 1 . 2

4 0

9 8 . 8 ± 0 . 5

4 0

9 9 . 0 ± 0 . 5

4 0

E x e r c i s e c a p a c i t y o r a c t i v i t y s t a t u s ( p a s t 1 5 m o )

3 6

. 5 ± 3 . 3

4 0

6 6 . 3 ± 4 . 6

4 0

7 2 . 8 ± 3 . 7

4 0

6 4 . 5 ± 4 . 9

4 0

C h o l e s t e r o l l e v e l ( p a s t 5 y r ) †

6 3

. 5 ± 2 . 6

4 0

8 9 . 7 ± 2 . 0

4 0

9 8 . 7 ± 0 . 5

4 0

9 9 . 4 ± 0 . 4

4 0

D i e t a r y a d v i c e ( p a s t 5 y r )

5 8

. 8 ± 3 . 9

4 0

7 5 . 7 ± 4 . 0

4 0

8 7 . 4 ± 2 . 8

4 0

8 3 . 5 ± 3 . 4

4 0

S m o k i n g s t a t u s ( p a s t 5 y r ) †

8 5

. 1 ± 1 . 9

4 0

8 6 . 6 ± 2 . 8

4 0

9 8 . 3 ± 0 . 6

4 0

9 9 . 2 ± 0 . 5

4 0

R e f e r r a l t o s p e c i a l i s t f o r e x e r c i s e e l e c t r o c a r d i o g r a m ( e v e r ) †

7 6

. 2 ± 2 . 0

4 0

8 8 . 2 ± 1 . 9

4 0

8 9 . 2 ± 1 . 9

4 0

9 3 . 8 ± 1 . 4

4 0

B l o o d p r e s s u r e c o n t r o l l e d ( ≤

1 5 0 / 9 0 m m H g ) †

4 7

. 9 ± 2 . 2

4 0

7 2 . 2 ± 2 . 1

4 0

8 1 . 5 ± 1 . 8

4 0

8 2 . 6 ± 2 . 4

4 0

S e r u m c h o l e s t e r o l c o n t r o l l e d ( ≤ 1 9 0 m g / d l ) †

1 6

. 9 ± 1 . 8

4 0

6 0 . 7 ± 3 . 3

4 0

7 2 . 9 ± 2 . 9

4 0

7 9 . 5 ± 2 . 2

4 0

A s p i r i n p r e s c r i b e d o r r e c o m m e n d e d u n l e s s r e c o r d

o f c o n t r a i n d i c a t i o n o r i n t o l e r a n c e †

7 6

. 2 ± 1 . 9

4 0

8 0 . 7 ± 2 . 4

4 0

9 2 . 5 ± 1 . 5

4 0

8 8 . 9 ± 1 . 9

4 0

C o n d i t i o n a l f a c t o r s ‡

A d v i c e g i v e n t o s m o k e r s ( p a

s t 5 y r ) †

6 7

. 3 ± 4 . 8

3 7

7 7 . 9 ± 6 . 0

3 5

9 7 . 9 ± 1 . 6

3 5

9 5 . 7 ± 2 . 1

2 9

W e i g h t a d v i c e f o r o v e r w e i g h

t p a t i e n t s ( p a s t 5 y r )

6 3

. 7 ± 4 0

4 0

7 8 . 8 ± 4 0

4 0

8 2 . 5 ± 4 0

4 0

9 0 . 0 ± 4 0

4 0

A c t i o n t a k e n o n b l o o d p r e s s u r e i f s y s t o l i c p r e s s u r e > 1 6 0 m m H g

o r i f > 1 4 0 m m H g w i t h c h o l e s t e r o l l e v e l > 1 9 0 m g / d l †

3 5

. 7 ± 5 . 8

3 5

6 0 . 3 ± 7 . 7

2 9

6 7 . 5 ± 9 . 8

2 0

7 8 . 1 ± 9 . 1

1 6

A c t i o n t a k e n i f c h o l e s t e r o l > 2 1 0 m g / d l ( f o r p a t i e n t s < 7 0 y r

o f a g e ) †

5 5

. 1 ± 5 . 9

3 6

7 6 . 1 ± 6 . 4

2 3

7 1 . 7 ± 9 . 3

2 3

8 4 . 6 ± 1 0 . 4

1 3

P r e s c r i b e d b e t a - b l o c k e r a s m

a i n t e n a n c e t h e r a p y i f s o l e t h e r a p y †

4 8

. 2 ± 5 . 0

3 7

5 8 . 4 ± 5 . 4

3 9

8 0 . 8 ± 3 . 5

3 9

6 8 . 7 ± 4 . 5

3 9

A s t h m a


N o r m a l o r p r e d i c t e d p e a k f l o w o r r e c o r d o f d i f f i c u l t y u s i n g m e t e r

( p a s t 5 y r )

7 6

. 1 ± 2 . 9

4 2

8 5 . 7 ± 2 . 3

4 2

9 1 . 6 ± 1 . 9

4 2

9 5 . 6 ± 1 . 2

4 2

R e c o r d o f d a i l y , n o c t u r n a l , o

r a c t i v i t y - l i m i t i n g s y m p t o m s

( p a s t 1 5 m o )

4 3

. 7 ± 3 . 3

4 2

5 8 . 1 ± 4 . 2

4 2

7 6 . 5 ± 2 . 8

4 2

7 1 . 6 ± 3 . 9

4 2


8 2

. 6 ± 2 . 9

4 2

8 4 . 3 ± 2 . 6

4 2

9 6 . 0 ± 1 . 1

4 2

9 8 . 8 ± 0 . 5

4 2

I n h a l e r t e c h n i q u e ( p a s t 5 y r )

5 3

. 1 ± 4 . 1

4 2

5 8 . 7 ± 4 . 5

4 2

7 6 . 0 ± 4 . 1

4 2

8 5 . 7 ± 2 . 8

4 2



s t 5 y r ) †

6 2

. 8 ± 5 . 2

4 1

8 6 . 8 ± 3 . 7

3 9

9 6 . 9 ± 1 . 6

4 1

9 8 . 9 ± 0 . 8

4 0

S e l f - m a n a g e m e n t p l a n f o r p a t i e n t s w h o a r e r e c e i v i n g h i g h -

d o s e c o r t i c o s t e r o i d s o r w

h o h a v e h a d i n p a t i e n t t r e a t m e n t

f o r a s t h m a ( p a s t 5 y r )

3 4

. 6 ± 5 . 7

4 1

5 0 . 5 ± 7 . 9

3 2

8 4 . 0 ± 6 . 9

2 5

8 9 . 4 ± 6 . 8

1 6




8/18/2019 NEJM 0807651

7/11



F o r p a t i e n t s w i t h r e c o r d e d e

x e r c i s e - i n d u c e d b r o n c h o s p a s m ,

s h o r t - a c t i n g b r o n c h o d i l a

t o r s p r e s c r i b e d f o r u s e b e f o r e

e x e r c i s e ( p a s t 5 y r )

4 0

. 5 ± 6 . 4

3 6

8 4 . 3 ± 6 . 2

2 6

9 4 . 7 ± 5 . 3

1 9

8 0 . 7 ± 8 . 0

1 8

P e a k f l o w d u r i n g a c o n s u l t a t i o n f o r a n e x a c e r b a t i o n

( m o s t r e c e n t e x a c e r b a t i o

n )

7 2

. 2 ± 5 . 1

4 0

7 2 . 6 ± 6 . 2

3 4

8 2 . 7 ± 4 . 7

3 3

6 8 . 6 ± 6 . 5

3 3

O r a l c o r t i c o s t e r o i d s p r e s c r i b e d i f p e a k f l o w < 6 0 %

o f n o r m a l v a l u e ( m o s t r e

c e n t e x a c e r b a t i o n )

8 8

. 8 ± 4 . 7

2 7

9 4 . 4 ± 5 . 6

9

1 0 0 . 0 ± 0 . 0

1 0

8 4 . 6 ± 1 0 . 4

1 3

A c t i o n t a k e n i f p a t i e n t h a s n o c t u r n a l s y m p t o m s ( p a s t 5 y r )

6 2

. 7 ± 4 . 9

2 9

7 6 . 2 ± 5 . 1

3 8

9 1 . 8 ± 3 . 6

3 6

8 3 . 3 ± 4 . 4

3 8

A c t i o n t a k e n i f p a t i e n t h a s a c t i v i t y - l i m i t i n g s y m p t o m s ( p a s t 5 y r )

5 6

. 4 ± 7 . 7

2 6

7 8 . 6 ± 6 . 9

2 3

1 0 0 . 0 ± 0 . 0

2 3

5 8 . 8 ± 7 . 1

2 7

R e f e r r a l t o r e s p i r a t o r y p h y s i c i a n w h e n o r a l c o r t i c o s t e r o i d s a r e

u s e d i n m a i n t e n a n c e t r e a t m e n t ( p a s t 5 y r )

4 7

. 1 ± 1 1 . 7

1 7

5 3 . 3 ± 2 2 . 6

5

3 3 . 3 ± 3 3 . 3

3

1 6 . 7 ± 1 6 . 7

6

D i a b e t e s


G l y c a t e d h e m o g l o b i n ( p a s t 1 5 m o ) †

8 7

. 1 ± 2 . 4

4 2

9 3 . 1 ± 1 . 3

4 2

9 9 . 4 ± 0 . 3

4 2

9 8 . 8 ± 0 . 5

4 2

V i s u a l e x a m i n a t i o n o f f e e t ( p

a s t 1 5 m o )

5 7

. 4 ± 4 . 6

4 2

6 9 . 6 ± 3 . 7

4 2

8 8 . 0 ± 2 . 8

4 2

9 1 . 5 ± 2 . 0

4 2

R e c o r d i n g o f p e r i p h e r a l p u l s

e s o r r e c o r d o f a b i l i t y t o f e e l v i b r a t i o n

( p a s t 1 5 m o ) †

6 0

. 0 ± 4 . 6

4 2

6 2 . 9 ± 4 . 2

4 2

9 0 . 2 ± 2 . 1

4 2

9 1 . 3 ± 2 . 0

4 2

S e r u m c r e a t i n i n e ( p a s t 1 5 m

o ) †

7 9

. 6 ± 2 . 9

4 2

8 9 . 9 ± 2 . 1

4 2

9 6 . 4 ± 1 . 0

4 2

9 9 . 0 ± 0 . 8

4 2

P r o t e i n u r i a ( p a s t 1 5 m o ) †

6 6

. 3 ± 4 . 0

4 2

7 4 . 8 ± 3 . 7

4 2

8 2 . 8 ± 3 . 6

4 2

8 9 . 8 ± 1 . 9

4 2

E x a m i n a t i o n o f f u n d i o r v i s u

a l a c u i t y ( p a s t 1 5 m o ) †

6 9

. 4 ± 3 . 2

4 2

7 2 . 2 ± 3 . 2

4 2

8 2 . 7 ± 2 . 3

4 2

8 1 . 1 ± 3 . 7

4 2

W e i g h t ( p a s t 1 5 m o ) †

8 0

. 2 ± 2 . 8

4 2

8 6 . 5 ± 2 . 9

4 2

9 7 . 2 ± 0 . 8

4 2

9 9 . 6 ± 0 . 3

4 2

B l o o d p r e s s u r e ( p a s t 1 5 m o ) †

9 2

. 6 ± 1 . 3

4 2

9 5 . 8 ± 1 . 8

4 2

9 9 . 0 ± 0 . 4

4 2

9 9 . 0 ± 0 . 5

4 2


8 6

. 5 ± 2 . 5

4 2

8 8 . 5 ± 2 . 8

4 2

9 8 . 4 ± 0 . 7

4 2

9 9 . 6 ± 0 . 3

4 2

S e r u m c h o l e s t e r o l ( p a s t 5 y r

) †

7 5

. 1 ± 3 . 1

4 2

9 7 . 6 ± 0 . 8

4 2

9 9 . 4 ± 0 . 3

4 2

9 9 . 8 ± 0 . 2

4 2

B l o o d p r e s s u r e c o n t r o l l e d ( ≤

1 4 0 / 8 5 m m H g ) †

2 1

. 8 ± 1 . 9

4 2

3 5 . 4 ± 2 . 6

4 2

4 9 . 0 ± 3 . 1

4 2

5 1 . 6 ± 2 . 5

4 2

S e r u m c h o l e s t e r o l c o n t r o l l e d ( ≤ 1 9 0 m g / d l ) †

2 1

. 8 ± 1 . 4

4 2

5 2 . 0 ± 2 . 5

4 2

7 2 . 5 ± 2 . 6

4 2

7 8 . 9 ± 1 . 9

4 2

G l y c a t e d h e m o g l o b i n c o n t r o

l l e d ( ≤ 7 . 4 g / d l ) †

3 7

. 8 ± 2 . 8

4 2

3 9 . 8 ± 2 . 4

4 2

5 0 . 6 ± 3 . 1

4 2

5 4 . 9 ± 3 . 4

4 2




8/18/2019 NEJM 0807651

8/11



and 5.5 percentage points in 2007 (95% CI, −1.0to 12.1). For heart disease, pay for performancein 2005 was associated with a nonsignificant im-provement in quality above the levels expected(2.8 percentage points; 95% CI, −0.1 to 5.8), andin 2007, it was associated with a nonsignificantreduction in quality from that expected (0.8 per-

centage points; 95% CI, −4.7 to 3.1). The resultsof patient evaluations of continuity of care were4.1 percentage points lower than expected in 2005(95% CI, −6.1 to −2.0) and 4.3 percentage pointslower in 2007 (95% CI, −6.9 to −1.6).

Discussion

We previously found that there were improve-ments in some aspects of clinical care over andabove the underlying trend after the introductionof a pay-for-performance scheme.3 Our current

findings suggest that although these initial im-provements were maintained, for two of the threeconditions studied (heart disease and asthma),improvements in the quality of care reached aplateau a year after the scheme’s introduction.Allowing for ceiling effects, care for diabetes con-tinued to improve, but it did so at a rate equiva-lent to that of the rate in the pre-introductionperiod.

Within these overall trends for care, we foundsignificant differences between aspects of carethat were linked to incentives and aspects of carethat were not linked to incentives. For asthmaand heart disease, we found a significant differ-ence in the effect of pay for performance onthese two groups of quality indicators; for bothconditions, mean quality scores for aspects ofcare that were not linked to incentives droppedbetween 2005 and 2007, whereas mean scoresfor aspects of care that were linked to incentivescontinued to increase. This widening gap in qual-ity came on top of already lower levels of care forindicators not linked to incentives.

For all aspects of care — whether associated with incentives or not — and for all three condi-tions, rates of quality improvement slowed con-siderably after 2005. There are several possible ex-planations. The f irst is that near-maximal scoreshad been achieved. However, whereas achieve-ment was high for some indicators (e.g., smokingstatus recorded for more than 98% of patientsfor all conditions), the logit transformation the-oretically eliminates ceiling effects, and we ob-


P a t i e n t e d u c a t i o n p r o v i d e d w i t h i n p a s t 5 y r i f d i a g n o s i s m a d e

d u r i n g t h a t p e r i o d

8 5

. 2 ± 3 . 2

4 1

8 8 . 0 ± 3 . 4

4 1

1 0 0 . 0 ± 0 . 0

4 2

9 9 . 0 ± 0 . 7

4 2


s t 5 y r ) †

6 7

. 1 ± 5 . 6

4 0

7 9 . 9 ± 5 . 7

3 6

9 5 . 8 ± 3 . 1

3 6

1

0 0 . 0 ± 0 . 0

3 4

R e f e r r a l t o a s p e c i a l i s t i f c r e a t i n i n e > 2 0 0 m m o l / l i t e r ( p a s t 5 y r )

7 5

. 0 ± 1 3 . 1

1 2

1 0 0 . 0 ± 0 . 0

3

1 0 0 . 0 ± 0 . 0

4

1

0 0 . 0 ± 0 . 0

4

F o r p a t i e n t s < 8 0 y r o f a g e , t r

e a t m e n t o f f e r e d i f a v e r a g e o f p a s t

3 r e a d i n g s s h o w s d i a s t o l i c p r e s s u r e > 1 0 0 m m H g o r s y s t o l i c

p r e s s u r e > 1 5 0 a n d d i a s t o l i c p r e s s u r e > 9 0 m m H g

6 3

. 3 ± 7 . 1

3 2

8 2 . 2 ± 7 . 8

2 2

8 4 . 4 ± 9 . 1

1 5

6 0 . 7 ± 1 3 . 0

1 4

I f p a t i e n t i s b e i n g t r e a t e d f o r h y p e r t e n s i o n a n d h a s p r o t e i n u r i a ,

A C E i n h i b i t o r p r o v i d e d

5 3

. 4 ± 3 . 2

4 2

5 7 . 1 ± 2 0 . 2

7

6 8 . 8 ± 1 3 . 2

8

6 3 . 0 ± 1 6 . 1

9

I f p a t i e n t i s r e c e i v i n g A C E i n

h i b i t o r , c r e a t i n i n e a n d p o t a s s i u m

m e a s u r e d w i t h i n 1 m o a f t e r s t a r t o f t r e a t m e n t ( p a s t 5 y r )

3 7

. 2 ± 4 . 8

4 1

3 0 . 3 ± 5 . 0

4 2

3 9 . 7 ± 4 . 4

4 1

3 4 . 3 ± 5 . 5

4 1

A s s e s s m e n t f o r h y p o g l y c e m

i a s y m p t o m s i f p a t i e n t r e c e i v i n g

s u l f o n y l u r e a ( p a s t 1 5 m o

)

1 8

. 2 ± 3 . 1

4 2

8 . 1 ± 1 . 8

4 2

7 . 8 ± 1 . 5

4 2

1 0 . 6 ± 2 . 1

4 2

F o r p a t i e n t s < 7 0 y r o f a g e , i f

m o s t r e c e n t g l y c a t e d h e m o g l o b i n

v a l u e w a s > 9 g / d l , t h e r a p e u t i c i n t e r v e n t i o n t o i m p r o v e

g l y c e m i c c o n t r o l o f f e r e d

7 7

. 4 ± 4 . 5

3 6

7 3 . 6 ± 5 . 6

3 7

6 9 . 4 ± 6 . 7

3 3

7 3 . 3 ± 6 . 7

3 0

* P l u s – m i n u s v a l u e s a r e m e a n s ±

S E . D a t a o n t h e c a r e o f p a t i e n t s w i t h a s t h m a , d i a b e t e s , o r c o r o n a r y h e a r t d i s e a s e w e r e e x t r a c t e d f r o m m e d i c a l r e c o r d s . S c o r e s a r e b a s e d o n a s c a l e o f 0

t o 1 0 0 . T o c o n v e r t t h e v a l u e s f o

r c h o l e s t e r o l t o m i l l i m o l e s p e r l i t e r , m u l t i p

l y b y 0 . 0 2 5 8 6 . T o c o n v e r t t h e v a l u e s f o r c r e

a t i n i n e t o m i l l i g r a m s p e r d e c i l i t e r , d i v i d e b

y 8 8 . 4 . A C E d e n o t e s a n -

g i o t e n s i n - c o n v e r t i n g e n z y m e .

† T h i s i n d i c a t o r o r a n e q u i v a l e n t c l i n i c a l p r o c e d u r e w a s a s s o c i a t e d w i t h a n i n c e n t i v e i n t h e p a y - f o r - p e r f o r m a n c e s c h e m

e .

‡ C o n d i t i o n a l f a c t o r s a r e i n d i c a t o

r s t h a t d i d n o t a p p l y t o a l l p a t i e n t s .




8/18/2019 NEJM 0807651

9/11



served the same plateau effect for indicators re-flecting lower levels of achievement. A secondexplanation is that once initial gains had beenmade, subsequent gains were more difficult toachieve. A third explanation is that the structureof the pay-for-performance scheme did not rewardfurther improvement once targets had been at-tained. This explanation is supported by the factthat family practices in our study gained, onaverage, 96.9% of available clinical-quality pay-ment points in 2005 and 97.8% in 2007 (which were similar to the average gains of 97.1% and97.5%, respectively, for all family practices inEngland12,13) — that is, there was little financialincentive for further improvement. A fourth ex-planation is that family practitioners had suffi-cient income and had little personal motivationto improve performance and income further (thetarget-income hypothesis); this explanation wouldbe consistent with the 30 to 40% gains in fam-

ily practitioners’ net income from the 2002–2003period to the 2005–2006 period.14

Our data cannot be used to ascertain the rela-tive merit of these explanations. However, gov-ernment negotiators in England appear to en-dorse the third explanation (too many physiciansachieving maximal or near-maximal paymentsfor quality of care). Alterations of the pay-for-performance scheme in 2006 introduced higherthresholds for maximal clinical-quality payments

and a wider range of indicators (see the Supple-mentary Appendix).

This study suggests that continuity of caredeclined after pay for performance was intro-duced. One possible explanation is that practicesfocused on meeting rapid-access targets in whichaccess to any doctor in the practice within 48hours was linked to incentives but access to aparticular physician was not,15 making it morediff icult for patients to see their own doctor. Thiscould be an unintended and perverse effect ofthe scheme and is a concern, since continuity isan aspect of family practice that patients value.16 Another explanation is that there were increasesin the size of practices, and many practices in-troduced nurse-led clinics for management ofindividual chronic diseases. Although this mayhave been an important part of improving thequality of care, it may have made continuity ofcare harder to achieve.

Other studies suggest that financial incentivesresult in small improvements in quality.17,18 Ourdata suggest that the pay-for-performance schemein England attained its quality-improvement goalsbut that the pace of improvement was not sus-tained once these goals had been reached. Theremay be unintended consequences for aspects ofcare other than those studied, which may be in-fluenced by differences in the operational detailsof apparently similar incentive schemes.19 An

100

S c o r e

90

80

60

50

30

70

40

01998 1999 2000 2001 2002 2003

Year

2007200620052004

Asthma, incentive

Diabetes, incentiveCoronary heart disease, incentive

Asthma, no incentive

Diabetes, no incentive

Coronary heart disease, no incentive

l

Figure 2. Mean Scores for Clinical Quality at the Practice Level for Aspects of Care for Coronary Heart Disease, Asthma,and Type 2 Diabetes That Were Linked with Incentives and Aspects of Care That Were Not Linked with Incentives,

1998–2007.

Quality scores range from 0% (no quality indicator was met for any patient) to 100% (all quality indicators were met

for all patients).




8/18/2019 NEJM 0807651

10/11



unanticipated benefit of the scheme in Englandhas been a reduction in sociodemographic in-equalities in the delivery of health care.20

Our study has several limitations. First, the pay-for-performance scheme was introduced through-out the United Kingdom, thereby precluding acontrolled trial and making the use of an inter-

rupted time series the best evaluation methodavailable. The only other time-series analysis ofthe quality of primary care in England suggeststhat pay for performance has had a more modesteffect than that suggested by our results.21 Sec-ond, because practices were observed at only twotime points before the introduction of pay forperformance, we cannot say whether the rate ofimprovement was already accelerating as a resultof earlier but still ongoing initiatives. Third, thestatistical power of our study was such that onlymoderate-to-large differences in trend were de-

tectable between indicators that were and thosethat were not associated with incentives. Fourth,response rates for the patient questionnaire werepoor (38 to 47%), although there is no reason tosuspect any differences in bias at the four studytime points. Finally, we focused on three diseasesfor which substantial efforts had been made toimprove the quality of care before the introduc-

tion of the pay-for-performance scheme. Pay forperformance might have a greater effect on con-ditions with lower profiles, including some in-troduced as the scheme developed (e.g., learningdisabilities).

In conclusion, between 1998 and 2007, there were significant improvements in measurable as-

pects of clinical performance with respect to thecare provided for three major chronic diseases.The initial acceleration in the underlying rate ofquality improvement after the introduction of payfor performance was not sustained. If the aim ofpay for performance is to give providers incen-tives to attain targets, the scheme achieved thataim. There may have been unintended conse-quences, including reductions in the quality ofsome aspects of care not linked to incentives andin the continuity of care.

Supported by the Department of Health for England.

Dr. Roland reports serving as an academic adviser to the gov-ernment and British Medical Association negotiating teamsduring the development of the U.K. pay-for-performance schemein 2001 and 2002. No other potential conf lict of interest relevantto this article was reported.

The views presented here are those of the authors and notthose of the Department of Health.

We thank the staff of all the practices that participated in thestudy and also Nan Bailey, Michele Bohan, Nicholas Burr, EllaGaehl, Nicola Small, and Angela Swallow, who contributed todata collection.

References

Roland M. Linking physicians’ pay to1.quality of care — a major experiment in

the United Kingdom. N Engl J Med 2004;351:1448-54.

Quality and Outcomes Framework:2.guidance for GMS contract, 2009/10. Lon-don: NHS Employers, March 2009. (Accessed June 29, 2009, at http://www.nhsemployers.org/Aboutus/Publications/Documents/QOF_Guidance_2009_final.pdf.)

Campbell S, Reeves D, Kontopantelis E,3.Middleton E, Sibbald B, Roland M. Qual-ity of primary care in England with theintroduction of pay for performance.N Engl J Med 2007;357:181-90.

Campbell SM, Roland MO, Shekelle4.PG, Cantrill JA, Buetow SA, Cragg DK.The development of review criteria forassessing the quality of management ofstable angina, adult asthma and non-insu-lin dependent diabetes mellitus in generalpractice. Qual Health Care 1999;8:6-15.

Campbell SM, Hann M, Hacker J,5.Durie A, Thapar A, Roland MO. Qualityassessment for three common conditionsin primary care: validity and reliability ofreview criteria developed by expert panelsfor angina, asthma and type 2 diabetes.Qual Saf Hea lth Care 2002;11:125-30.

Ramsay J, Campbell JL, Schroter S,6.Green J, Roland M. The General Practice

Assessment Survey (GPAS): tests of dataquality and measurement properties. FamPract 2000;17:372-9.

Mead N, Bower P, Roland M. The Gen-7.eral Practice Assessment Questionnaire(GPAQ) — development and psychometriccharacterist ics. BMC Fam Pract 2008;9:13.(Accessed June 29, 2009, at http://www.biomedcentral.com/1471-2296/9/13.)

Mangin D, Toop L. The Quality and8.Outcomes Framework: what have you doneto yourselves? Br J Gen Pract 2007;57:435-7.

Campbell SM, Hann M, Hacker J, et al.9.Identify ing predictors of high quality carein English general practice: observationalstudy. BMJ 2001;323:784-7.

Campbell SM, Roland MO, Middleton10.E, Reeves D. Improvements in the qualityof clinical care in English general prac-tice 1998-2003: longitudinal observationalstudy. BMJ 2005;331:1121-5.

Wagner AK, Soumerai SB, Zhang F,11.Ross-Degnan D. Segmented regressionanalysis of interrupted time series studiesin medication use research. J Clin PharmTher 2002;27:299-309.

National Quality and Outcomes Frame-12. work statistics for England 2005/06. Lon-

don: National Health Service InformationCentre, 2006. (Accessed June 29, 2009, athttp://www.ic.nhs.uk/webfiles/publications/qof/NationalQualityOutcomesFramework280906_WORD.doc.)

National Quality and Outcomes Frame-13. work statistics for England 2007/08. Lon-don: National Health Service InformationCentre, 2008. (Accessed June 29, 2009, athttp://www.ic.nhs.uk/statistics-and-data-collections/supporting-information/audits-and-performance/the-quality-and-outcomes-framework/qof-2007/08/bulletin.)

GP Earnings and Expenses Enquiry14.2005/06: final report. London: NationalHealth Service Information Centre, 2008.(Accessed June 29, 2009, at http://www.ic.nhs.uk/statistics-and-data-collections/primary-care/general-practice/gp-earnings-and-expenses-enquiry-2005-2006:-final-report.)

Goodall S, Montgomery A, Banks J,15.Salisbury C, Sampson F, Pickin M. Imple-mentation of Advanced Access in generalpractice: postal survey of practices. Br JGen Pract 2006;56:918-23.

Cheraghi-Sohi S, Hole AR, Mead N, et16.




8/18/2019 NEJM 0807651

11/11



al. What patients want from primary careconsultations: a discrete choice experimentto identify patients’ priorities. Ann FamMed 2008;6:107-15.

Rosenthal MB, Frank RG, Li Z, Ep-17.stein AM. Early experience with pay-for-performance: from concept to practice. JAMA 2005;294:1788-93.

Lindenauer PK, Remus D, Roman S,18.

et al. Public reporting and pay for perfor-mance in hospital quality improvement.N Engl J Med 2007;356:486-96.

McDonald R, Roland M. Pay for per-19.formance in primary care in England andCalifornia: comparison of unintendedconsequences. Ann Fam Med 2009;7:121-7.

Doran T, Fullwood C, Kontopantelis E,20.Reeves D. Effect of financial incentives oninequalities in the delivery of primaryclinical care in England: analysis of clini-

cal activity indicators for the quality andoutcomes framework. Lancet 2008;372:728-36.

QRESEARCH and The Information21.Centre. Time series analysis for selectedclinical indicators from the Quality andOutcomes Framework 2001-2006. Notting-ham, United Kingdom: QRESEARCH, 2007.(Accessed June 29, 2009, at http://www.qresearch.org/Public_Documents/Time%20Series%20Analysis%20for%20selected%20clinical.pdf.)

Copyright © 2009 Massachusetts Medical Society.

POSTING PRESENTATIONS AT MEDICAL MEETINGS ON THE INTERNET

Posting an audio recording of an oral presentation at a medical meeting on the

Internet, with selected slides from the presentation, will not be considered priorpublication. This will allow students and physicians who are unable to attend themeeting to hear the presentation and view the slides. If there are any questionsabout this policy, authors should feel free to call the Journal’s Editorial Offices.


Downloaded from nejm org on April 20 2016 For personal use only No other uses without permission