evidence-based management an objective view of …€¦ · balanced scorecard theory –norton and...
TRANSCRIPT
EVIDENCE-BASED MANAGEMENT
AN OBJECTIVE VIEW OF LEADERSHIP IMPACT
JEAN CHENOWETH, SR VICE PRESIDENT
100 TOP HOSPITALS PROGRAMS
“Leadership is a multi-dimensional and ever-changing
phenomenon” Porter-O’Grady & Malloch, 2010
Evidence-based leadership standards are few and far between.
“Books abound with strategies based on experiential knowledge or personal
philosophy, but few studies have successfully linked leadership practices to
measurable outcomes” Lynham & Chermack, 2006
“Evidence-based management is the systematic application of the best
available evidence to business processes, strategic decisions and the
evaluation of managerial practices” Kovner, Fine & D’Aquila, 2009
EVIDENCE-BASED LEADERSHIP
2
THE BEGINNING OF EVIDENCE-BASED
MANAGEMENT
Evidence-based models should be designed to address specific three categories of management questions - Kovner & Rundall, 2006
Business transaction management
Evidence-based medicine initiatives of JCAHO began in1984
Operational management
Evidence-based management engineering standards ( MAPS , 1964)
Strategic management
Malcolm T. Baldrige Program (National Institute of Standards and Technology, 1981)
Measurement of leadership 100 Top Hospitals: National Benchmarks for Success. Wm. M. Mercer Provider
Consulting, HCIA, 1993
Economic Evaluation of the Baldrige Performance Excellence Program, A.N. Link, PhD, University of North Carolina at Greensboro, J.T. Scott, PhD., Dartmouth College, National Institute of Standards and Technology, US Department of Commerce, November 2011
Comparison of Baldrige Award Applicants and Recipients with Peer Hospitals on a National Balanced Scorecard, D.A. Foster, PhD, J Chenoweth, Thomson Reuters, National Institute of Standards and Technology, US Department of Commerce, October, 2011
3
DIFFERENCES IN SCORECARD OBJECTIVES MUST
BE DIFFERENTIATED MORE EFFECTIVELY
Hospital Compare
5
100 TOP HOSPITALS MEASUREMENT OF LEADERSHIP IMPACT AND VALUE:
Goal to measure leadership’s ability to drive the consistency and reliability of organization’s performance versus peers
Not a consumer tool for hospital selection
21 year development and field testing effort
Balanced scorecard theory – Norton and Kaplan, Harvard University
Academic validation of hospital balanced scorecard
Griffith & Alexander, ACHE Hayhow Award for Excellence in Research – Measuring Comparative Hospital Performance, Journal of Healthcare Management 47:1, January/February 2002.
Relative performance on a set of Medicare-based measures (balanced scorecard) can be used by hospital governing boards to identify and rank improvement in achievement of mission.
Objective statistical analysis of public data, updated annually
Peer-reviewed risk and severity adjustment and methodologies
4
The Baldrige criteria represent R&D investments, sets of
performance standards and calibrated bench standards to
achieve predetermined levels of management performance
An economically sound estimate of Baldrige program is
Benefit-to-cost ratio of 820-to-1
The Baldrige Performance Excellence Program value could
not be replicated by private sector actions alone.
ECONOMIC VALUE OF BALDRIGE PROGRAM
Source: Economic Evaluation of the Baldrige Performance Excellence Program, A.N. Link,
University of North Carolina at Greensboro, J.T. Scott, Dartmouth College, National Institute of Standards
and Technology, US Department of Commerce, November 2011
7
MAJORITY USE BALDRIGE PROCESSES BUT HAVE NOT APPLIED FOR AWARD
8
Baldrige award winners 2 times more
likely than peers to become 100 Top
winners 3 years after award.
Baldrige award winners improve more
than 5 times faster than peers as
leadership processes take hold p =
0.007
BALDRIGE WINNER PERFORMANCE ON
OBJECTIVE NATIONAL BALANCED SCORECARD
• NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY,
US DEPT. OF COMMERCE
Study available at
http://www.nist.gov/baldrige/baldrige-102511.cfm
NIST* REQUEST: COMPARE BALDRIGE WINNERS TO PEERS USING
100 TOP BALANCED SCORECARD
6
The impact of leadership on organization’s relative balanced performance CAN
be measured
Organization’s life cycle can be identified and compared
Information reflects leadership impact on organization
Relative success or failure to improve
Rates of improvement
Resultant performance against national benchmarks
Degree of alignment
LEADERSHIP CAN BE MEASURED
9
Journey Not Begun
Early Success Mature Culture Of PI
PI Culture At Risk
LEADERSHIP’S JOURNEY TO EXCELLENCERATE OF IMPROVEMENT AND RESULTANT PERFORMANCE
Composite score
LOW performance
HIGH improvement
HIGH performance
HIGH improvement
HIGH performance
LOW improvementLOW performance
LOW improvement
25
10
10
COMPOSITE SCORE
MORTALITY
COMPLICATIONS
SAFETY
CORE
MEASURES
EXPENSE
PROFIT
TOP 10%
TO
P 1
0%
LOW performance
HIGH improvement
HIGH performance
HIGH improvement
HIGH performance
LOW improvementLOW performance
LOW improvement
LOS
LEADERS IN WEAK ORGANIZATIONS REQUIRE
LASER FOCUS TO DEVELOP CULTURE OF PI
11
LEADERS OF HIGH PERFORMERS MUST RAISE BAROR RISK OF FALLING BEHIND
PI Culture At Risk
MORTALITY
COMPLICATIONS SAFETY
CORE MEASURES
LOS
EXPENSE
Composite score
PROFIT
12
H1
H2
H3
H4H5
H6
H7
H8H9H10
H11
H12
H13
H14
H15H16
H17
0
20
40
60
80
100
0 20 40 60 80 1002011
2009 -
201
1 R
ate
of Top
Top LOS
H1
H2
H3
H4H5
H6
H7H8
H9H10
H11
H12
H13
H14
H15H16
H17
0
20
40
60
80
100
0 20 40 60 80 1002011
2009 -
201
1 R
ate
of Top
Top MORTALITY
H1
H2H3
H4 H5
H6
H7
H8
H9
H10
H11
H12 H13
H14
H15
H16
H17
0
20
40
60
80
100
0 20 40 60 80 100
2007 -
201
1 R
ate
of
Top
Top
2011
Top
Top HCAHPS
H1H2
H3
H4
H5
H6
H7
H8H9
H10
H11
H12
H13
H14
H15
H16
H17
0
20
40
60
80
100
0 20 40 60 80 100
2011
2007 -
201
1 R
ate
of
Top
Top OPERATING PROFIT MARGIN
LEADER’S ACHIEVEMENT OF CONSISTENCY HIGHLY EVOLVED CULTURE OF PERFORMANCE IMPROVEMENT
13
WE HAVE NOT MASTERED CONSISTENCY
H1
H2
H3
H4
H5
H6
H7 H8H9
H10 H11
H12
H13
H14
H15
H16H17
0
20
40
60
80
100
0 20 40 60 80 100
2011
2009 -
201
1 R
ate
of
Top
Top PATIENT SAFETY
H1
H2
H3
H4
H5
H6
H7
H8
H9
H10H11H12
H13
H14
H15
H16
H17
0
20
40
60
80
100
0 20 40 60 80 100
2011
2007 -
201
1 R
ate
of
Top
Top
INPATIENT. EXPENSE/DISCHARGE
H1H2H3
H4
H5
H6
H7
H8
H9
H10
H11
H12
H13H14
H15
H16
H17
0
20
40
60
80
100
0 20 40 60 80 1002011
2009 -
201
1 R
ate
of
Top
Top
H1
H2
H3
H4
H5
H6
H7
H8
H9
H10H11
H12
H13
H14
H15
H16
H17
0
20
40
60
80
100
0 20 40 60 80 1002011
2007 -
201
1 R
ate
of Top
Top
COMPLICATIONS CORE MEASURES
14
Top Leadership
Board
CEO characteristics
Communication
Goals
Infrastructure
Investment
Executives
Clinical performance
Nursing
Pharmacy
HOW ARE LEADERS DIFFERENT
IN 100 TOP HOSPITALS?
MISSION MAKES A SUBSTANTIAL DIFFERENCE
IN FOCUS AND PERFORMANCE
Church-owned NFP Best overall balanced performance
Significant. lower mortality
Significant. shorter lengths of stay
Significant. higher HCAHPS
Second best in Core Meas., Expense
CHURCH NFP - HIGHEST BALANCED PERFORMANCE
FP Corporation
Lowest expense
Highest profit
Highest core Meas. scores
Government-owned (non-federal
Lagged behind on all measures
Significantly worse on
Core measures
Expense
All NFP combined
Significant, better safety
Significant. lower 30 mortality
Strong HCAHPS scores
15
17
FH
IN F
ina
list P
rese
nta
tion
BROADENING LEADERSHIP INSIGHTS
LEADERSHIP MEASUREMENT IS SCALABLE
AT DIFFERENT ORGANIZATIONAL LEVELS
National balanced scorecard based on public data allows
aggregation of performance data at many levels
States
Health Plans, ACOs
Health systems
Service lines
Alignment of the organization
Reliability of performance
Hospitals
Service lines
Non-clinical departments
Alignment of the organization
Physician group practices, PHOs
Service lines
17
Data Year: 2004
LOCUS OF HIGH PERFORMANCE SHIFTED TO
MIDWEST WITH FOCUS ON QUALITYCOLLABORATION TO RAISE ALL BOATS
2006
2008
Quintile Performance Key:
Quintile Percentile Range Performance
1 >80 to 100 Best
2 >60 to 80
3 >40 to 60
4 >20 to 40
5 > 0 to 20 Worst
18
NATIONAL ENVIRONMENT
BENCHMARKS SHIFT SOUTH AND WEST
2011 DATA
Quintile Performance Key:
Quintile Percentile Range Performance
1 >80 to 100 Best
2 >60 to 80
3 >40 to 60
4 >20 to 40
5 > 0 to 20 Worst
19
IMPACT OF POLICY, INCENTIVES, PROVIDER FOCUS
ON STATE HOSPITAL INDUSTRY
H1
H2
H3
H4
H5
H6
H7
H8
H9
H10
H11
H12
H13
H14
H15
H16
H17
H18H19
H20H21
H22H23
H24
H25
H26
H27
H28
H29
H30
H31
H32
H33
H34
H35
H36
H37
H38
H39
H40
H41H42
H43
H44
H45
H46
H47
H48 H49
H50
H51
H52
H53
H54
H55
H56
H57 H58
H59
H60
H61
H62
H63
H64
H65
H66
H67
H68
H69
H70
H71
H72
H73
H74
H75
H76
H77 H78
H79
H80
H81
H82
H83
H84
H85
H86
H87
H88
0
20
40
60
80
100
0 20 40 60 80 100
2009 Level of Achievement
20
05
-2
00
9 R
ate
of
Imp
rove
me
nt
Top H1 H2
H3
H4
H5
H6
H7
H8
H9H10
H11
H12
H13
H14
H15
H16
H17
H18
H19
H20
H21
H22
H23
H24
H25
H26
H27
H28 H29
H30
H31
H32
H33
H34
H35
H36
H37
H38
H39
H40
H41
H42
H43
H44
H45
H46
H47
H48
H49
H50
H51
H52
H53
H54
H55
H56
H57
H58
H59
H60
H61
H62H63
H64
H65
H66
H67
H68
H69
H70H71
H72
H73
H74
H75
H76
H77
H78
H79
H80
H81
H82
H83
H84
H85H86
H87
H88
H89
H90
H91
H92
H93
H94
H95
H96
H97
H98
H99
H100
H101
H102
H103
H104
H105
H106
H107
H108
H109
H110
H111
H112H113
H114H115
H116
H117H118
H119
H120
H121
H122
H123H124
H125
H126
H127
H128
H129
H130
H131
H132
H133
H134
H135
H136
H137
H138
H139
H140H141
H142H143
H144
H145
H146
H147
H148H149
H150
H151
H152
H153
H154
H155
H156
H157
H158
0
20
40
60
80
100
0 20 40 60 80 100
2009 Level of Achievement
20
05
-2
00
9 R
ate
of
Imp
rove
me
nt
Top 10
Top 10%
BALANCED SCORECARD PERFORMANCE
OF NEW YORK HOSPITALS
BALANCED SCORECARD PERFORMANCE
OF MICHIGAN HOSPITALS
2005 – 2009 Longitudinal MedPAR Data
SOLE FOCUS ON FINANCE, PENALTIES FOCUS ON QUALITY, COLLABORATION
20
2009 88 HOSPITALS
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1 2
1 2
1 2
1 2
1 2
1 2
1 2
1 2
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3 4
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
MICHIGAN CEO COMMITMENT TO COLLABORATE HIGHER STATE-WIDE VALUE DELIVERED
2
1 2
1 2
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3 5
1 2 3 4 5
DETROIT
1
1
1 2
1 2
1 2
GRAND RAPIDS
HOLLAND
1
1 2
1 2
KALAMAZOO
BATTLE CREEK
1
1
1
1 ─ 3
ANN ARBOR
JACKSON
1
1
MUSKEGON
FLINT MIDLAND
SAGINAW
2
1 2 3 4 5
3
1 ─ 3
LANSING
OWOSSO
Quintile Performance Key:
Quintile Percentile Range Performance
1 >80 to 100 Best
2 >60 to 80
3 >40 to 60
4 >20 to 40
5 > 0 to 20 Worst21
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
4 5
4 5
4 5
4 5
4 5
4 5
4 5
4 5
4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
3 4 5
2 3 4 5
2 3 4 5
2 3 4 5
1 2 3 4 5
1 2 3 4 5
NYS LONG TERM
PERFORMANCEIMPROVEMENT
2007 - 2011 FFY
1
1
1
1
1 2
1 2 3 5
1 2 3 5
1 2 3 5
1 2 3 5
1 2 3 5
1 2 3 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
NYS PERFORMANCE
ACHIEVEMENT
2011 FFY
CRITICAL FOR BOARDS AND EXECUTIVES:
TWO DIMENSIONAL MEASUREMENT NYS HOSPITAL LEADERSHIP IMPACT: PERFORMANACE VS. IMPROVEMENT
Quintile Performance Key:
Quintile Percentile Range Performance
1 >80 to 100 Best
2 >60 to 80
3 >40 to 60
4 >20 to 40
5 > 0 to 20 Worst
22
MEASURING SYSTEM LEADERSHIP
24
0
10
20
30
40
50
60
Performance Improvement
Av
era
ge P
erc
en
tile
of
Perf
orm
an
ce (
Hig
her
is B
ett
er)
Hospitals in Systems Have Higher Performance and Rates of Improvement on 100 Top Hospitals National Balanced Scorecard (p< 0.0001 — Both Comparisons)
Non-systemmembers
SYSTEM MEMBERSHIP AND PERFORMANCE
SOURCE: DAVID FOSTER, PHD., CHIEF SCIENTIST, 2012 .
TRUVEN WHITE PAPER PENDING PUBLICATION
24
INITIAL VARIATION ACROSS SYSTEMS2009 10 TOP HEALTH SYSTEM WINNERS
PERCENTILESOVERALL MORT COMP PSI
CORE
MEAS
30-DAY
MORT
30-DAY
READMITALOS HCAHPS
ADVOCATE HEALTHCARE 97.3 96.1 100.0 77.3 96.9 79.6 6.1 74.9 21.8
BANNER HEALTH 98.0 92.5 90.2 78.0 47.1 58.8 52.5 95.3 56.9
CATHOLIC HEALTH PARTNERS 98.4 94.1 97.3 86.3 75.7 56.9 52.2 60.0 48.2
FAIRVIEW HEALTH SERVICES 97.6 67.1 63.9 97.6 63.5 78.0 38.0 98.8 66.3
KETTERING HEALTH NETWORK 99.6 96.9 98.4 40.0 98.4 59.2 68.2 96.5 64.1
MAYO FOUNDATION 98.8 50.6 85.1 49.0 85.5 83.1 50.2 91.4 95.3
OHIOHEALTH CORPORATION 96.9 27.1 94.1 95.3 96.5 25.5 16.5 89.4 71.8
SCRIPPS HEALTH 96.5 89.8 18.4 79.2 70.2 97.1 78.4 65.1 92.5
SPECTRUM HEALTH 100.0 69.4 85.9 62.0 93.3 72.5 91.2 88.6 91.8
UNIVERSITY HOSPITALS
HEALTH SYSTEM99.2 79.6 91.0 96.9 91.4 60.4 10.2 90.2 36.1
Quintile Performance Key:
Quintile Percentile Range Performance
1 >80 to 100 Best
2 >60 to 80
3 >40 to 60
4 >20 to 40
5 > 0 to 20 Worst
25
ALIGNMENT
27
Alignment of member hospitals• Distance from Centroid
• Closer to centroid, better the score
• Lower score is better
Alignment Score
Unaligned System: 42.4
Top P & I Median: 21.5
Peer Group Median: 32.8
ALIGNMENT OF SYSTEM HOSPITALS
Alignment Score
Aligned System: 9.57
Top P & I Median: 21.5
Peer Group Median: 32.8
MAJOR HEALTH SYSTEM OPPORTUNITY
IMPROVE CONSISTENCY, RELIABILITY
34.8
22.7
27.4
2013 PERFORMANCE AND 5-YEAR RATE OF IMPROVEMENT
28
DOES OWNERSHIP OF SYSTEMS MAKE A
DIFFERENCE?
Source: Differences in Health System Quality Performance by Ownership. David Foster,
PHD. Truven Research Brief, 2010
28
HEALTH SYSTEM SCORECARD, 2011 STUDY
Quintile Performance Key:
QuintilePercentile
Range
Performance
Level
1 >80 to 100 Best
2 >60 to 80
3 >40 to 60
4 >20 to 40
5 > 0 to 20 Worst
H1H2
H3
H4
H5
H6
0
20
40
60
80
100
0 20 40 60 80 100
2009 Performance
20
05
-2
00
9 R
ate
of
Imp
rove
me
nt
Top 10%
Top 10%
10 Top Health Systems Composite Score
HEALTH SYSTEM KEY
H1. SYSTEM A
H2. SYSTEM B
H3. SYSTEM C
H4. SYSTEM D
H5. SYSTEM E
H6. SYSTEM F
29
FAST FORWARD TO 2013 STUDYFOR PROFITS ARE TARGETING HIGHER QUALITY
HCA MEMBER HOSPITALS
Performance and Rates of Improvement
Quintile Performance Key:
Quintile Percentile Range Performance
1 >80 to 100 Best
2 >60 to 80
3 >40 to 60
4 >20 to 40
5 > 0 to 20 Worst
CORE MEASURES HCAHPS
30
OUR ONGOING PROCESS
We are Continually Learning
31
VARIATION IN CLASS PERFORMANCE CAN REFLECT
LEADERSHIP COMPLACENCY OR BIAS
MAJOR TEACH ING HAS HIGHEST SURVIVAL
HIGHEST COMPLICATIONS & ADVERSE EVENTS
MORTALITY
COMPLICATIONS
SAFETYHCAHPS
OTHER CLASSES OUTPERFORM IN
LOS, PROFIT, HCAHPS
EXPENSE CONTROL
ADJ. LENGTH OF STAY
32
CHALLENGE OF RELIABILITY FOR LEADERSABSENCE OF ALIGNMENT DAMAGES POPULATION MANAGEMENT
MORTALITY SCORES OF 12 HOSPITALS IN HEALTH SYSTEM
HOSPITAL KEY
1. MEMORIAL
2. MEMORIAL COUNTY
3. Memorial WEST
4. MEMORIAL EAST
5. COMMUNITY
6. COMMUNITY NORTH
7. COMMUNITY SOUTH
8. ST. MARK
9. ST MARY
10. POLK
11. MARSHALL
12. FREDRICK COUNTY
33
CHALLENGE TO INSURANCE PLAN LEADERSHIPCOMPARISON OF INSURANCE NETWORK PERFORMANCE ON EXCHANGES
2
2
2 3
2 3
2 3
2 3
2 3
1 2 3
1 2 3
1 2 3
1 2 3 4
1 2 3 4
1 2 3 4
1 2 3 4
1 2 3 4
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
2
2
2 3
1 2 3
1 2 3
1 2 3 4
1 2 3 4
INDIANA
2 3
2 3
2 3
1 2 3 4
1 2 3 4 5
1 2 3 4 5
MICHIGAN
OHIO
2 3
1 2 3
1 2 3 4
1 2 3 4
1 2 3 4 5
HEALTH NETWORK – MIDWESTERN STATES
GEOGRAPHIC AND COST SELECTION - INADEQUATE
34
EMPLOYERS WANT CONSISTENCY ACROSS
INSURANCE NETWORK
3
3 4 5
1 3 4 5
1 2 3 4 5
SEATTLE, WA EMPLOYER’S NETWORK
100 Top Hospital Performance2005 State Level Rankings
Quintile 1-Best (10)Quintile 2 (10)Quintile 3 (10)Quintile 4 (10)Quintile 5-Worst (11)
100 Top Hospitals® Ranking
Hospital Balanced
Excellence
Provider inclusion based on total 2012 active self-insured Medical and Drug Claims
5
2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
CITY OF SEATTLE
WAS SEATTLE EMPLOYER’S NETWORK SELECTED ON PRICE ONLY?
5
2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
SHOULD EMPLOYER’S ASSUME ALL PERFORMANCE IS SAME?
35
LOCAL INSURANCE
NETWORKA SOUTHERN HEALTHCARE
SYSTEM
1
1
1
1
1
1
CONSISTENT PERFORMANCE
ACROSS COMMUNITIES SERVED
UNEVEN PERFORMANCE
ACROSS COMMUNITIES SERVED
2
2
2 3
2 3
1 2 3
1 2 3
1 2 3 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
Excellence In Quality, Cost, Efficiency, Patient Perception Of Care
LEADER’S MISSION AND GOALS ARE KEY
ADVANTAGE OF PROVIDER-BASED HEALTH PLANS
36
REFORM FORCES SINGLE WORLD VIEW
Collaboration brings
faster results,
consistency
Common goals
Common data
37
