William Berry, MDPrinciple Research Scientist,

Harvard T.H. Chan School of Public HealthDeputy Director

Ariadne Labs

Exploring the Relationship Between Managerial

Practices and Quality Improvement

The South Carolina Ariadne Labs Partnership

4+ years to improve OR Safety The work has improved patient care across the state

Some observations- SC hospitals are pioneers- SC hospitals are different from one another- Some hospitals appear more ready to work quality

improvement work than others- And…..

Management Matters

The World Management Survey: A Little History

Developed by Stanford, London School of Economics, and the Harvard Business School

More than 15,000 interviews representing 34 countries

Across multiple industries:- Manufacturing- Schools- Healthcare

Asked if organizations adhere to 3 best practices:

Targets: Does the organization support long-term goals with tough, but achievable short-term performance benchmarks?

People Management: Does the organization properly manage and retain talent while re-training or moving underperformers?

Monitoring: Does the organization rigorously collect and analyze performance data to identify opportunities for improvement?

Improving Management Practices In Textile Manufacturers in India

Randomized study of Indian textile manufacturing firms

28 plants across 17 firms randomized to intervention (diagnostic survey plus four months targeted consulting) or control (survey alone) in 2:1 ratio

17% increase in productivity with ~$300,000 increased productivity at intervention plants

Manufacturing Was Found To Have Better Management Practices Than Hospitals

United States

The Management Survey in Healthcare

2,000 Acute Care Hospitals in 9 Countries, including the U.S.

Researchers cold called hospitals One interview per hospital, targeting a middle

manager (typically nurse managers) Interviews focused on the adoption of management

practices:– Monitoring– Targets– People Management

The Findings In Healthcare

Management correlated with hospital quality of care, productivity, and financial performance

Management practices varied widely across hospitals

Management is correlated with quality of care: Higher management scores associated with lower AMI mortality rates (risk adjusted)

Notes: Based on 324 observations with available AMI information (Canada:29; Sweden: 48; UK: 74; US: 178). We z-score the AMI data within country to take into account differences in the way the AMI rates are calculated across countries, and keep only hospitals with at least 20 AMI cases in a year. For both AMI rates and Management, we take residuals from a regression including country dummies, hospital controls(number of employees, specialty, percentage of managers with a clinical degree), noise controls (13 interviewer dummies, the seniority and tenure of the manager who responded, the duration of the interview, and an indicator of the reliability of the information as coded by the interviewer, interviewee type) and regional dummies. AMI mortality rates data refer to 2009 in the US and UK, to 2008 in Sweden and the average between 2007 and 2009 in Canada. The p-value on the difference between the bottom and the middle tercile is 0.204; the p-value on the difference between the bottom and the top tercile is 0.001. The p-value on the difference between the middle and the top tercile is 0.07.

The Management Survey in South Carolina

South Carolina Management Survey Project launched in 2014 15 participating hospitals to date 84+ of interviews conducted to date Recruiting hospitals that perform inpatient surgery

What is different about the work in SC? Looking at organizations more deeply than ever done before Multiple interviews and across service lines:

- C-suite, Chiefs of Departments, Middle Managers Assessing additional management practices than previous

projects We can explore the relationship that management has to clinical

outcomes

Managerial Practices Assessed In The Interviews

4 practices:– Operations: whether standardized protocols are understood and used by all clinical

staff– Monitoring: whether performance is tracked and reviewed and whether differing

levels of performance lead to different consequences– Targets: whether established goals address long- and short-term time horizons and are

clearly defined and tied to hospital objectives– People: whether high and poor performers are adequately rewarded and penalized and

talent is properly managed and retained

Key Performance Indicators

Management believed to matter on multiple levels - Financial and clinical outcomes- Process improvements (e.g. uptake of surgical safety program across

hospitals in Safe Surgery 2015 Collaborative)

Massachusetts is following in the footsteps of South Carolina

Some preliminary findings from South Carolina and Massachusetts

C-suite managers believe that the management practices are better than middle managers

C-Suite

Middle Managers

C-suite scores are not aligned with middle managers’ scores

Note that in only one hospital are middle managers more optimistic than C-suite managers.

Line of agreement

Better key performance indicator tracking by C-suite is associated with better management scores

Note: Based on a sample of 147 interviews across 26 hospitals in South Carolina and Massachusetts. Higher values = better use of KPIs

C-Suite

Middle Managers

Key Performance Indicator (KPI) Use

We want to expand this work to more hospitals in South Carolina

Benefits of participation

Individualized, benchmarked report for each hospital that participates- Understand management variation within your

hospital- Benchmark management practices in your

hospital to others in your state and beyond- Identify areas for improvement

Help build knowledge around management and clinical quality improvement

Sample Report

22

The overall management score is calculated using using all interviews conducted within each hospital. A management score of 5 = full adoption of processes, 0 = No adoption of processes

Alignment Across Management Areas

Four Management Areas:1.Operations management – whether standardized protocols are understood and used by all clinical staff2.Monitoring management – whether performance is tracked and reviewed and whether differing levels of performance lead to different consequences3.Target management – whether established goals address long- and short-term time horizons and are clearly defined and tied to hospital objectives.4.People management – whether high and poor performers are adequately rewarded and penalized and talent is properly managed and retained.

A management score of 5 = full adoption of processes, 0 = No adoption of processes23

Joining The Project

Your Organizations Commitment:7-10 one-hour interviews with managers across your organizationDesignate a contact for your organization to work with our team to schedule the interviews

How To Join The Project:Information about this project is included in your meeting materialsTalk to Alex Haynes or myself at this meetingContact our team via email for more information:

Alex Haynes, MD abhaynes@ariadnelabs.org Brooke Huskey bhuskey@ariadnelabs.org

Thank you

Extra Slides In Case You Need Them

Hospital distributions typically shifted the the left relative to manufacturing

US Italy

India

C-suite managers are more optimistic about their organization

C-suite managers tend to report higher rate of adoption of basic management practices relative to middle managers

C-suite managers scores are much more compressed relative to middle managers. In particular, C-suite managers tend to have a much thinner left tail in the distribution (and no scores below 2).