public defence: realist evaluation of capacity building programme of health managers in tumkur,...

Capacity building and performance in local health systemsA realist evaluation of a local health system strengthening intervention

in Tumkur, India

Prashanth Nuggehalli Srinivas

Public defense UCLApril 21, 2015

Promoter: Jean MacqCo-promoter: Bart Criel

Scheme

• What is it like to be a health manager?

• Getting from capacity to performance

• Questions, methods and their evolution

• Results

• Lessons learned, limitations and relevance

Background

Methods

Results & Discussion

21/04/15Public defence, Prashanth N S,

Universite Catholique de Louvain, Brussels2

What is it to be a health manager?Policies & programmes

People

District

Background



Regional improvements, but disparities remain

“Accelerated progress to reduce mortality during the neonatal period and at ages 1–59 months is needed in most Indian districts.”

- Usha Ram et. al. 2013



Background

Management matters

“Systemic failure” as a cause? (Asha George, 2007 & 2009; Gita Sen, 2006)

Nanjundappa Committee Report, Government of Karnataka



Background

What is it to be a health manager?Policy

People

District

• Dynamic environment, decentralisingdistricts

• Decades in clinical specialty

• Falling faith in public services “only for the poor”, “only preventive care”, “frustrating…”

• Private sector

• Researcher and academia focus

….the fall guys!



Background

From knowledge to action

• Knowledge

• Skills….competencies

• Capacity

• Performance of health managers/workers

• Improvement/ “good” management

• Utilisation and coverage

• Better health services

• Better health status



Background

Transformation

Photo: Steve Greer & Getty images/stevegreerphotography.com



Kirkpatrick framework



Background

First put forward by Donald Kirkpatrick in his PhD dissertation in 1954 at the University of Wisconsin titled “Evaluating a Human Relations Training Program for Supervisors”

Capacity and performance

Brown, L., LaFond, A., & Macintyre, K. (2001). Measuring capacity building. Chapel Hill: MEASURE Evaluation.



Capacity Performance

Environment

Background

Socio-culturalEffortTimeCulture-oriented change & “new way of doing things” (shifting norms, powers, values)

TechnicalProgrammaticQuick(er)TangibleTask-oriented nature of change (changing procedures and activities)

Potter & Brough 2004

Organisational change



Where it began for me?



The Tumkur capacity building intervention



Background

3 million population 80% rural

Mentoring

Contact classes & Assignment

Health managers (medical & non-medical) at district level

Health team at taluka level

Primary health centremedical officers & local government reps

2-3 days per month, residential contact classes

At least 5 mentoring days/month

1 assignment/month

Intervention



Background

Questions evolved, so did methods

From controlling the context allowing experimentation to understanding howcontext shapes outcomes



Methods

Outcomes pattern: what are the practical effects produced by causal mechanisms being triggered in a given context?

Context: what conditions are needed for an intervention to trigger mechanisms to produce particular outcomes patterns?

Mechanism: what is it about an intervention which may lead it to have a particular outcome in a given context?

Pawson & Tilley, 1997. Illustration from Marchal 2013



Methods

Same intervention, different settings



Methods

Study design



Methods

Elicited programme theory- 1

Contact classes could work through commitment and efficacy of

health managers who bear an intention to make positive change

by providing them resources in the form of knowledge and/or

skills; they are likely to apply these knowledge and skills in

talukas where local team environment supports such change

and the change agenda aligns with the local government and

district/state expectations




Mentored participants are more likely to seek opportunities to

improve their local health systems to make positive change in the

organisation’s performance wherever there is no hindrance (or there is

an alignment) to such moves either from above or from the

PRI/community structures

Elicited programme theory - 2




Local health systems could be improved in decentralising health

systems if teams have the ability to negotiate with various actors

about their change proposals and if they claim decision-spaces for

preparation and implementation of action plans and local

decision-making at district and lower levels; if the capacity

building programme could work at multiple levels to ensure

better alignments between opposing elements across various

actors and levels in the health system.

Elicited programme theory - 3




Study design



• Perceptions were aligned

• Receptive to technical guidance

• Open to involving “lower levels” and local government reps




But structural problems…

Nature of the teams at taluka and district level



The National Rural Health Mission (NRHM) appointed “managers” were contractual appointees within teams with very senior clinically trained doctor-managers


“What is the use of putting my time into the action plan, if they will change it anyway at the state (level)?”

a district level health manager

Perceptions across the health bureaucracy




Tumkur health status report (2011), SwasthyaKarnataka & Institute of Public Health, Bangalore

Narrow perceived decision-spaces

…in spite of National Rural Health Mission’s on-paper decentralisedplanning and management since 2005




Study design



Explaining organisational change

• Identified case studies based on diversity of context and/or outcome after scanning context, mechanism and outcome elements

• Confronted the reformulated PT and first round of context-mechanism-outcome-based change scenarios to these cases




Case selection

• Committed and mentored teams with low-moderate intention to

make change

• Poorly resourced teams with varying commitment levels/types &

high intention for change




21/04/15 30

“In my taluka for example, I think we can make big change. It is not that everybody in my taluka want to make changes. Only one-third of them are motivated to make changes. And that is enough. I think I can make a lot of improvement by motivating these people.”

- a Gubbi taluka helath manager

“More resources mean more opportunities to make change. If they slowly give more and more power to us at taluka level, we can make many more improvements. Right now, very little is possible at taluka level. “

- another taluka health manager from Gubbi

What action plan are you talking about? What decentralisation? I sent so many requirements for staff and proposals for improvement. Only thing I got is more work, less staff and zero solutions. On one hand, I have to answer the local government members’ complaints and on the other hand, I have to just keep implementing plans and schemes coming from above. Nothing can be done without more staff.

- a health manager from CN Halli

We felt that we have to do it. So many mothers were just being referred to Tumkur. The delivery load is high and for several months, we had only one obstetrician, but somehow we managed. I know how the pressure is at the district hospital, so having caesarean facility at Sira decreases the burden at the district hospital. It’s not easy, but somehow it is happening.

- a Sira health manager

Public defence, Prashanth N S, Universite Catholique de Louvain, Brussels


Refined programme theory



Committed health management teams could utilise new opportunities

for organisational improvement presented by decentralising health

systems wherever their change agenda aligns with the expectations of

higher levels of the bureaucracy.


Refined programme theory



Tapping commitment for organisational change could be frustrating in

low-resource local health systems where health managers working in

poorly resourced talukas, in spite of their improved management

capacities and intentions to make change, could get frustrated by the

lack of facilitating action from above.


Lessons learned – organisational change




• Application of insights from organisational and social sciences

in health systems strengthening

• Post-hoc reconstruction of programme theory; Interventions

need to invest in understanding why they expect change in the

given context

• Using programme theory refinement and realist evaluation as

an operational tool for implementation

Lessons learned - methods




Source: Nick D Kim/lab-initio.com

Limitations – comprehensiveness & lens



Need for more case diversity and further iterations could improve the final refined theory


Limitations - complexity



Fran/cartoonstock

Relevance

Teaching PolicyResearchPractice

Karnataka HR strategy

Theory-driven evaluation inspired research projects




Thank you

Tendencies may be possessed unexercised, exercised unrealised, and

realized unperceived (or undetected) by men; they may also be

transformed.

- Roy Bhaskar in “A Realist Theory of Science”, Introduction, p.18



public defence: realist evaluation of capacity building programme of health managers in tumkur,...

Healthcare

prashanth n s

universite catholique

health outcomes

com210415public defence

public services

indian states

subregional disparities

lot of regional disparities