An Impact Assessment of Stakeholder Engagement Interventions in Ugandan Oil

Extractives

Protea Hotel, Kampala Uganda

December 6, 2017

Outline

• Background

• Impact Assessment of MSFs

• Study Design

• Treatment Effects

• Limitations

Background

Analysis of Categorised Barriers

4

Multi-Stakeholder Forums (MSFs)

•District-based

•Process and Outcome-oriented: 1. identify priority barriers 2. describe the cause of the barriers 3. develop actions to address them 4. jointly monitor their implementation

•Priorities and actions reviewed regularly

5

MSFs Composition

MSFs

Communities (30 villages; 2 reps each)

Local Governments

(Districts & Sub-Counties)

Private Sector

(JVPs & AUGOS)

Central Government

(Ministries, Departments, Agencies)

6

MSFs Operating Principles

1. Recognizing the unique role of each stakeholder

2. Understanding that all stakeholders are equally important

3. Engaging each other with mutual respect and trust

4. Listening to one another and speaking in an open and effective way

5. Working towards mutually beneficial outcomes and leaving no one behind

6. Documenting and sharing of information in a transparent manner

7. Responding and following through on commitments effectively

7

Observed Impacts of MSFs • Community-driven accountability

• Communities and local governments leading advocacy actions they previously expected others (e.g. CSOs) to lead

• Communities seeking national-level action when local-level efforts are not working

• Communities recognizing the importance of documentation of their grievances and maintaining records

• Multi-directional accountability • Central government and companies interacting directly and responding regularly to the concerns of

communities and local governments

• Formation of broad-based associations for entrenchment of best practice and galvanising voice (e.g. AGODA, community associations within districts)

• Communities and local governments are making financial commitments towards jointly developing actions to address common barriers and implement them

• Local governments proposing legal and administrative measures to address barriers identified with the oil and gas industry

8

Impact Assessment of MSFs

Background on Stakeholder Engagement in the O&G Industry

• Historical Background: Social Performance standards post MMSD (Mining Minerals and Sustainable Development) and EIR (Extractive Industries Review)

• Many international and corporate standards

• Expectations of study context for oil development

• Effectiveness studies private

• No systematic peer reviewed public evaluations

10

Theory of Change

11

No MSF

MSF Transparency Accountability

• Better understand the oil sector project cycle

• More realistic expectations

• Priorities identified

• Some community initiative to address concerns (maybe misdirected)

• Community representatives effectively advocate for priorities to the right duty bearer

• Greater empathy and trust between stakeholders• Stakeholders make commitments• Community representatives relay information

back to their villages

• People observe the state of their concerns

• Communities identify those responsible for their concerns

• Communities punish or reward the responsible duty bearers

Co

ntr

ol G

rou

p

Acc

ess

to In

form

atio

nLegend

Outcome

Intervention

Assumption

Tre

atm

en

t G

rou

p

Methodology and Approach

• Mixed Methods: Randomized Controlled Trial supplemented with Qualitative and Gender Analysis

• Four qualitative questions – issue importance, issue satisfaction, and attribution of blame and credit on the issues of land management, social services and local economic development

• Modelling transparency through prompt feedback to data providers

12

Evaluation questions, outcomes • Key Questions

• Are communities likely to seek more transparency and accountability as a result of engagement with governments and corporations through MSFs?

• Are governments and corporations more accountable when they engage with community-based stakeholders through MSFs?

• Outcome Measures

• Transparency: Access to information about oil company and government actions in the region.

• Concerns and Levels of Satisfaction: Index of Satisfaction - concerns weighted by the importance each household places on each concern.

• Causal Attribution. Index of how the household attributes responsibility for their concerns among stakeholders.

• Civic Action. Activities the household or community engages in to hold stakeholders accountable.

13

Study Design

Project Area

15

Arua Buliisa

Hoima Moyo

Nebbi Ntoroko

Nwoya Yumbe

8 Districts

107 Villages

Close to existing exploration blocks

Randomized Controlled Trial

• Causality • What “works”?

• Counterfactuals—what would have happened?

• Over-pessimism? Over-optimism?

• Comparisons • Treatment and Control Groups

• Gold standard=Random assignment

Design of this Study

• Unit of Analysis • Villages

• Households within villages

• Random assignment • Control group: Information only

• Treatment group: Information +

Participation in multi-stakeholder forums

(Baseline) Time 1 Time 2

(Endline) Time 3

Control Observe Observe

Treatment Observe Intervention Observe

Assessing Impact

Comparisons

• Between Treatment and Control Group at Baseline

(should be about the same)

• Between Treatment and Control Groups at Endline

(should be different if treatment works)

Example

Treatment Effects

• Change in an outcome that is due to the intervention

• Difference-in-Differences • Differences in outcomes at baseline b/n treatment and control

• Differences in outcomes at endline b/n treatment and control

• A single measure of the total changes due to treatment

• For most of the measures, something above 0 means the intervention had a positive impact

• Groupings: (1) everyone, subgroups of (2) women and (3) men

Treatment Effects

The program increases transparency for almost all measures

The intervention increases civic activity by households and communities

The treatment improves overall satisfaction for those issues that households care most about

• Index of Issues • Local Economic Development

• Land Management

• Social services

• Composed of relative… • Issue Importance

• Issue Satisfaction

The intervention does not appear to significantly improve specific outcomes in issue areas

Qualitative Responses on Issue Satisfaction

Blame/credit not concentrating on any specific actor; blame remains diffuse

Reasons for blame and credit

0 200 400 600 800 1000 1200 1400

No corruption

Good leadership

Competence

Community cohesion

Good engagement

Sufficient Information

Social benefit

Economic benefit

Laws & policies

Credit

0 200 400 600 800 1000 1200 1400

Corruption

Neglect

Incompetence

Lack of Comm Cohesion

Inadequate Engagement

Inadequate Information

Inadequate Soc. Benefit

Inadequate Econ. Benefit

Laws & Policies

Other

Blame

Limitations

• Short time period between treatment and end line

• Translation and language issues

• Preparation of treatment village representatives

• Exposure to only one MSF

• Exclusion of sub-county and district leaders in the MSF

THANK YOU