rss 2009 - investigating the impact of the qof on quality of primary care

UK pay-for-performance schemeMethodsResults

Investigating the impact of the Quality andOutcomes Framework on the quality of primary

care

Evan Kontopantelis Tim Doran* David ReevesJose Valderas Martin Roland

National Primary Care Research and Development CentreUniversity of Manchester England

RSS, 9th September 2009

Kontopantelis The QOF impact


Outline

1 UK pay-for-performance schemeThe ideaFramework details

2 MethodsThe designThe General Practice Research DatabaseAnalysis

3 ResultsShort term QOF effect (2004/05)Long term QOF effect (2004/05-2006/07)Summary



The ideaFramework details

Improving quality of care.A (very) juicy carrot...

A P4P program kicked off in April 2004 with theintroduction of a new GP contract.

General practices are rewarded for achieving a set ofquality targets for patients with chronic conditions.The aim was to increase overall quality of care and toreduce variation in quality between practices.

The incentive scheme for payment of GPs was namedQuality and Outcomes Framework (QOF).QOF is reviewed at least every two years.




Quality and Outcomes Framework.Indicator details relate to Year 1.

Estimated cost of £1.8b, over 3 years (escalated to £2.8b).GP income increased by up to 25%, dependent onperformance.146 quality indicators.

Clinical care for 10 chronic diseases (76 indicators).Organisation of care (56 indicators).Additional services (10 indicators).Patient experience (4 indicators).

Implemented simultaneously in all practices (a controlgroup was out of the question).




Performance.Incentivised aspects keep improving but what about the non-icentivised ones?

55

60

65

70

75

80

85

90

%

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007year

Coronary heart disease Asthma Diabetes

40, 42 and 42 practices respectively in total

Clinical performance

70

80

90

100

%

2004/05 2005/06 2006/07 2007/08QOF year

Asthma(3) CHD(9) Mental Health(3) DM(15)

Epilepsy(3) Heart failure(2) Stroke(7) COPD(2)

based on data from all English practicesnumber of indicators used in the averages, displayed in brackets

Clinical domain averages

Overall performance of incentivised care

Quality scores for all QOFclinical indicators have beenimproving.They are only a smallproportion of all clinical care.There are concerns thatquality for non-incentivisedaspects of care may havebeen neglected.How to measure performanceon the non-incentivisedaspects though?



The designThe General Practice Research DatabaseAnalysis

Clinical indicators...and research questions.

Two aspects to clinical indicators:a disease condition (e.g. diabetes, CHD).a care activity (e.g. influenza vaccination, BP control).

ConditionIncentivised(QOF)

Non-incentivised

Incentivised activity (QOF) A: 30 B: 9Non-incentivised activity C: 7 D: 8

We aimed to compare the four classes on changes inquality from pre-QOF to post-QOF.

Would class A indicators show most improvement?Would classes B and C show some ‘halo’ effects since theyinvolve either a QOF condition or activity?Has quality for class D indicators declined?




Enter GPRD...Patient level data!

GPRD holds event data for more than 270 Englishpractices, from 1999 (488 active practices in 2007 andaround 10m patients).Database is broken down to numerous tables, because ofthe volume of data.Final sample was 600,000 patients from 150 practices(over 24GB in size).Allows us to:

Investigate quality of care at the patient level.Model and measure non incentivised aspects of care.




GPRD details.

Event files.Clinical: all medical history data (symptoms, signs anddiagnoses).Referral: information on patient referrals to external carecentres.Immunisation: data on immunisation records.Therapy: data relating to all prescriptions issued by a GP.Test: data on test records.

Lookup files.Medical codes: READ codes, 111,865 available.Product codes: 77,198 available.Test codes: 304 available.




The process.Step 1: searching in the dark...

Size of the tables prohibits looking at codes one by one.Instead we use search terms to identify potentially relevantcodes in the lookup tables and create draft lists.

Example (Search terms for diabetes)String search in Medical codes: ’diab’ ’mell’ ’iddm’ ’niddm’.READ code search in Medical codes file: ’C10’ ’XaFsp’.String search in Product codes file: ’insulin’ ’sulphonylurea’’chlorpropamide’ ’glibenclamide’.




The process.Step 2: clinicians brawl over the codes (statisticians place bets).

Clinicians go through the draft lists and select the relevantcodes.Three sets of codes are created, that correspond to:

QOF criteria.Conservative criteria.Speculative criteria.

Using the finalised code lists we search for events in theClinical, Referral, Immunisation, Therapy and Test files.




The process.Step 3: extracting the events of interest, for each indicator.

For each indicator and year, we used the associatedclinical rules and retrieved events to construct a % score:

(cases where indicator was met / eligible patients)*100%.

Example (C4 indicator, QOF year 06/07: 01Apr06-31Mar07)Percentage of patients with hypertension who have arecord of blood glucose once since diagnosis.Denominator: Patients registered for the full year with thepractice, diagnosed with hypertension at any point until the31st of March 2007 (excluding resolved cases).Numerator: Patients in the denominator who have had atleast one record of blood glucose, postdating the first(unresolved) hypertension diagnosis.




Issues to tackle.Is the analysis ever easy...?

Indicator classes are imbalanced.Three different types of activities:

clinical processes related to measurement (PM/R).clinical processes related to treatment (PT).intermediate outcome measures (I).

ConditionIncentivised (QOF) Non-incentivisedPM/R PT I PM/R PT I

Inc. activity (QOF) 18 8 4 8 1 0Non-inc. activity 4 3 0 2 6 0

Quality of care was already improving (prior to QOF).The ceiling has been reached for certain ‘easy’ indicators.




The approach.Interrupted Time Series.

ITS multi-level multivariate regression analyses, allowed usto compare the four indicator classes on the basis of twooutcome measures:

20

40

60

80

100

Per

form

ance

(%)

00−0

1

01−0

2

02−0

3

03−0

4

04−0

5

05−0

6

06−0

7

QOF years

Observations Regression line, pre−QOF

Regression line, post−QOF pre−QOF line extended

level change

The level differencebetween the estimated pre-and post-QOF trends, in2004/05.The change in slope fromthe pre- to the post-QOFtrend.




The approach.Some details.

The main analysis used logit-transformed scores.Regression models run with xtmixed in STATA (mle).Practices were treated as crossed with indicators andrandom intercepts were used.Created variable combining class & type into 9 ind groups.Analysed 3 models, for each of the outcome measures:

UncontrolledControlled pre-QOF slope & mean of each indicator - RE*.Controlled pre-QOF slope & mean of each indicator - FE.

Using the estimated means for the 9 indicator groups wecompared across classes on: (i) PM/R types only(ii) PT types only (iii) PM/R & PT types combined.



Short term QOF effect (2004/05)Long term QOF effect (2004/05-2006/07)Summary

Still...




Change in level immediately following QOF.Compared to expectations.

Level change - logit, controlled, RE-1

.37

1.23

-0.8

3-0.5

40.

35

0.68

-0.3

0

0.24

0.130.

45

-0.7

4

0.32

-0.3

3

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

A-I

A-PM

/RB-

PM/R

C-PM

/RD-

PM/R

A-PT

B-PT

C-PT

D-PT A-P

B-P

C-P

D-P

Indicator groupLevel change - logit, uncontrolled

-0.1

4

0.49

0.03 -0.0

2 0.41

-0.1

0-0

.04

0.65

-0.0

6

-0.0

5

-0.0

6

0.89

-0.0

8

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

A-I

A-PM

/RB-

PM/R

C-PM

/RD-

PM/R

A-PT

B-PT

C-PT

D-PT A-P

B-P

C-P

D-P

Indicator group

Type PM/R indicators of classA significantly increased inlevel above expectationpost-QOF.Type PM/R indicators ofclasses B & D significantlydecreased in level belowexpectation post-QOF.Similar results for classes A &D (but not B) using thecombined process typeindicators.




Change in level immediately following QOF.Comparing ordered means across classes, using the Newman-Keuls procedure.

A-I 0.32C-PM/R B-PM/R D-PM/R

A-PM/R 1.23 * *** ***C-PM/R 0.45 *** ***B-PM/R -0.74 nsD-PM/R -1.37

A-PT D-PT B-PTC-PT 0.24 ns ns nsA-PT 0.13 ns nsD-PT -0.30 nsB-PT -0.33

C-P B-P D-PA-P 0.68 ns *** ***C-P 0.35 * ***B-P -0.54 nsD-P -0.83

Ordered Means For PM/R indicators all fourclasses were significantlydifferent from each other inmean level change, bar B & D.For PT indicators there wereno significant differences.For combined P indicators,pairs A & C and B & D werenot significantly different, butA & C had significantly greaterlevel changes than B & D.




Change in slope from pre- to post-QOF.Change in slope - logit, controlled, RE

-0.2

1

-0.1

2

-0.2

30.

06

-0.1

8

-0.1

7-0

.21

-0.1

8

-0.0

5

-0.2

2

-0.2

5-0.1

7

-0.2

9

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

A-I

A-PM

/RB-

PM/R

C-PM

/RD-

PM/R

A-PT

B-PT

C-PT

D-PT A-P

B-P

C-P

D-P

Indicator groupChange in slope - logit, uncontrolled

0.03

-0.3

2

0.00

-0.1

0-0

.16

-0.2

1

-0.0

2

-0.0

3

-0.0

9

-0.2

9

-0.1

2-0.0

9

-0.0

8

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

A-I

A-PM

/RB-

PM/R

C-PM

/RD-

PM/R

A-PT

B-PT

C-PT

D-PT A-P

B-P

C-P

D-P

Indicator group

Slope declined significantly:for the A-I group.for all classes of PM/Rindicators, bar class C.for all classes of PTindicators.for all classes of thecombined P indicators, barclass C.




Change in slope from pre- to post-QOF.Comparing ordered means across classes, using the Newman-Keuls procedure.

A-I -0.12A-PM/R B-PM/R D-PM/R

C-PM/R 0.06 *** *** ***A-PM/R -0.17 nsB-PM/R -0.23D-PM/R -0.29

A-PT D-PT B-PTC-PT -0.17 nsA-PT -0.18D-PT -0.21B-PT -0.21

A-P B-P D-PC-P -0.05 ** ** ***A-P -0.18 nsB-P -0.22D-P -0.25

Ordered Means For the PM/R, only class Cdiffered significantly from theother classes (in mean slopechange).For the PT indicators therewere no significantdifferences.For the combined Pindicators, only class Cemerged as significantlydifferent.




Conclusions....results are still being interpreted!

Short term:Only the PM/R indicators within class A showed greaterthan expected improvement.PM/R indicators in classes B and D demonstratedsignificantly lower than expected gains.Class C (QOF-incentivised patient group) PM/R indicatorssignificantly out-performed classes B and D.

Long term:All types and classes, with the exception of C-PM/R,experienced a significant drop in the rate of improvement.

QOF had an immediate ’halo’ effect on non-QOF activitiesfor patient groups targeted by QOF (class C).


Thank you Not too much information, I hope!

Comments and questions:[email protected]


rss 2009 - investigating the impact of the qof on quality of primary care

Health & Medicine