the challenge of designing ‘good enough’ performance measures & results framework targets...
TRANSCRIPT
The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework
Targets & Performance ManagementWorld Bank’s Public Sector Performance
Global Expert Team 9 May
Gwyn Bevan
Models of delivery of public services
Knights / Knaves
Econs / Humans
Prospect theory
Trust & Altruism (T&A)
Knights Humans Reward failure?
Hierarchy & Targets (H&T)
Knights & Knaves
Econs Sack staff?
Public Ranking (PR) Knights Humans Reputation?
Choice & Competition (C&C)
Knights & Knaves
Econs Winners & losers?
UK’s natural experiment: before & after devolution (1999)
NHSs Schools
Before After Before After
EnglandT&A
H&T
PR
C&C
PRWales T&A C&C
PR
C&C
H&T+PR: ‘naming & shaming’ spotlight + prospect theory
Staff in zero-rated hospital ‘devastating …hit right
down to workforce -- bad reports usually hit senior management upwards
nurses demand changing rooms .. because being accosted in streets’
Source: Mannion et al (2005)
‘dirty dozen’ zero-rated CEOs(English NHS)
Hospital waiting times: T&A H&T+PR
Numbers waiting elective admissions (England) (‘000s)
H&T+PR
Hospital waiting times: T&A
0
10
20
30
40
50
60
70
1997 1999 2001 2003 2005
England Wales
No/‘000 waiting > 3 months GP 1st outpatient appointment
Hospital waiting times. Natural experiment: T&A vs H&T+PR
051015202530354045
1997 1999 2001 2003 2005
England Wales
No/‘000 waiting > 3 months GP 1st outpatient appointment
H&T+PR
Ambulances:T&A
45
50
55
60
65
70
75
80
1999 2001 2003 2005 2007
England Wales
Source: Bevan & Hamblin (2009)
% responses life-threatening emergencies < 8 minutes Target
E & W
Ambulances. Natural experiment: T&A vs H&T+PR
45
50
55
60
65
70
75
80
1999 2001 2003 2005 2007
England Wales
Source: Bevan & Hamblin (2009)
% responses life-threatening emergencies < 8 minutes H&T+P
RTarget E
Target W
35
45
55
65
1994 1996 1998 2000 2002 2004 2006 2008
England Wales
% pupils achieving > 5 good grades at 16
Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years
Schools: C&C+PR
35
45
55
65
1994 1996 1998 2000 2002 2004 2006 2008
England Wales
Schools. Natural experiment: C&C+PR vs C&C
Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years
Wales stops PR
% pupils achieving > 5 good grades at 16
H&T & PR: design
Residual domain R
Gages on P
No measures on P
Tin openers on P
Selected domain P
H&T & PR : problems of gaming
Selected domain T: incentives apply to measures M[T]
Excluded domain N: unimportant or cannot be measured
Holmstrom & Milgram (1991)
H&T & PR: types of gaming
N? Worsen
performance? M[T]?
Hit the target & miss the point
Misreporting & cheating
CollusionTarget M[T]
Domain N
Domain T
For discussion
Theory of contingency mix of models
T&A performance measurement no gaming ineffective
PR + H&T powerful gaming Design to counter gaming
C&C choice attractive in principle Characteristics influence effectiveness
For discussion C&C: Requirements
Universities Schools Hospitals
Money follows choice Supply-side flexibility Information Transaction costs Management freedom
COFEPRIS: Commission for Protection against Sanitary Risks
9,000 registrations (value $1bn) required 150 registered in 2010
Elements for transformation of failing organisations
English NHS COFEPRIS
Spotlight effect
Naming & shaming in media
Inform media of failure
‘Broken windows’
Waiting times Registrations
Prospect theory
Sack ‘failing’ CEOs Sack underperforming middle managers
Rewards Name elite organisations
Promote young able managers
COFEPRIS: Sanitary registrations
Source: Mikel Arriola Peñalosa (2012) Federal Commissioner for the Protection against Sanitary Risks