Using Outcome Mapping as a Using Outcome Mapping as a Research Methodology for Information Research Methodology for Information Communication Technology ProjectsCommunication Technology Projects

Chris Morris African Advanced Institute for Information & Communications Technology (AAIICT)

Comparative study of “first mile” and Comparative study of “first mile” and “first inch” technology in different low “first inch” technology in different low

density contexts.density contexts.

Project funded by the IDRC

MozambiqueMozambique

AngolaAngola

South AfricaSouth Africa

ZimbabweZimbabwe

FMFI Projects & PartnersFMFI Projects & Partners

Angonet (Huambo)

Wireless Mesh (Peebles Valley)

CSIR Telehealth (Eastern Cape)

UWC MuTi (Eastern Cape)

CSIR HLT (Openphone)

Translate.org (HTML Pub)

Power Line Comms (Rooiwal)

SchoolNet Mozambique (Inhambane)

MICTI (Maputo)

UCM (Pemba)

ISPU (Quelimane)Wireless

(Harare)

ObjectivesObjectives

The objectives of this project are: To develop innovative ICT technologies and to implement “first

mile” solutions To develop business models for first mile solutions To research behaviour in the use of ICTs – how the use of ICTs

has changed community life Scalability and replicability of technologies – the viability of

rolling out the solution• Influence on policy and regulation – demonstration of project

benefits to the regulator To publish a reference book for “first mile” and “first inch”

implementation in rural Africa.

Research Question? Research Question?

• What is the sector you want to contribute to through your project? How does it link to the overall vision of the First Mile/First Inch Network?

• What is it that you want to make known through your project? (What is the research question?) Why?

• Who are you going to influence through your project? • What kinds of influence are you intending to have on your boundary

partners’ behaviours? • What are the main methods by which you are going to answer your

research question? Why did you select these methods? (relates to issue of validity)

• What are the main strategies you are going to use to influence your boundary partners? Why did you select these strategies? (relates to issue of relevance)

• How would you propose to monitor and report on the project process as it unfolds?

Choice?Choice?

• research methodology that was simple to apply• robust in being able to provide an implementation framework but

flexible enough to adapt• we needed to be participatory with our project partners• to understand the development process and learn more about

human behaviour and how that influences technology choice• we are a learning organisation, we wanted to look at the change

taking place, not simply make a claim to it. • a tool that helped us document the journey & understand the

processes - what worked and what did not work and why• one that would develop a space for social learning and creative

dialogue

Choice contd. ?Choice contd. ?

• we needed to be able to reflect and review our plans and make changes when necessary.

• at the end of the project we needed to know where we have been and what were the lessons learned.

• we needed to know how to improve next time. • to be accountable to our stakeholders and donors.• we needed to undertake comparative research across projects,

across countries.• we needed a dissemination strategy - identify Primary Intended

Users and Primary Intended Uses of research results.

Why Outcome Mapping?Why Outcome Mapping?

Outcome Mapping methodology for planning, monitoring, and evaluation in order to document, learn, and report results

Development is accomplished by, and for, people

Focuses on outcomes as behavioural change

Not impact but contributions to outcomes

Boundary partners control change

Outcomes MappingOutcomes Mapping

OM provides a method for development programmes to plan for and assess the capacities that they are helping to build in the people, groups, and organizations that will ultimately be responsible for improving the well-being of their communities.

Outcomes are defined as changes in the behaviour, relationships, activities, or actions of the people, groups, and organizations with whom a program works directly

Boundary partners are those individuals, groups, and organizations with whom the program interacts directly and with whom the program anticipates opportunities for influence

Outcome Mapping…Cont.Outcome Mapping…Cont.

Outcome Mapping asks the following questions :

Why? (What is the vision to which the program wants to contribute?);

Who? (Who are the program's boundary partners?);

What? (What are the changes that are being sought?); and

How? (How will the program contribute to the change process?).

Why?Why?

• VISION This inspired ambitious action and commitment, - but was not an

yardstick by which the project would be measured

Who? - Boundary Partner MapWho? - Boundary Partner Map

Tsilitwa Health Project

Doctor(Hospital)

Clinic sister

DistrictManager

ProvincialDoH

NationalDoH

MEC

Community(Patients)

USAICASA

CSIRResearchers

WorldBank IDRC

Outcome ChallengesOutcome ChallengesBoundary partner – Clinic sister

The project intends to see the clinic sister empowered by the use of the technology and sharing her knowledge with other clinic sisters in the area and seeking advice from specialist. The clinic sister will build a trust relationship with the doctors and be recognised by other health officials for inputs into decision-making. The sister is able to clearly articulate a vision of telehealth activities and goals that is relevant to her community needs. She acts as a champion for the roll-out of telehealth in the province and motivates others.

Boundary partner – hospital doctor

The project intends to see the doctors supporting and participating in the use of the technology. The doctors will join the online community of health specialists and share case studies and lessons learnt. The doctor will build a trust relationship with the clinic sister and help build her skills and knowledge.The doctor will access health information on the internet and continue with his professional training online. The doctor will use the internet to keep in professional and social contact with colleagues and friends around the world thereby reducing some of the feelings of isolation experienced from being in a deep rural location.

Boundary partner – East London hospital specialist

The project intends to see the dermatologist based at the East London hospital responding to the clinic sister’s e-mails and providing advice. on a timely basis. One would also like to see a proactive stance from the doctor in terms of devolving first line diagnostic decisions to the clinic level, based on the learning experience of clear-cut cases. The dermatologist will build up case studies and share and promote the technology at conferences and medical events.

Boundary partner – Provincial Department of Health

The project intends to see Dr Rajeev become an enthusiastic supporter for the telehealth technology. He will influence the Department to adopt the Tsilitwa telehealth project and seek to replicate it throughout the Eastern Cape Province..The rural telehealth solution will be included in the Department’s strategy and budget. The Department will develop training modules for the clinic sisters and provide ongoing support. Dr Rajeev will become a champion for rural connectivity and inspire other provinces and the National Department of Health to replicate the technology.

What? - Progress markersWhat? - Progress markers

Outcomes have been assigned to each boundary partner & progress markers are used to monitor behavioural changes of boundary partners in achieving the desired outcome. The Progress Markers for the Outcome Challenges are categorised according to the current status where:

• Expect to see = Behaviours the project expects to see

• Like to see = Behaviours the project would like to see

• Love to see = The behaviours associated with achieving the best outcome

Progress Marker: Clinic SisterProgress Marker: Clinic Sister

Clinic Sister

Expect to See Like to See Love to See

Clinic sister not afraid to use the system and

feels like it is a benefit Sister becoming proficient in use of the technology and suggests improvements

Sisters becomes a champion for rural telehealth, she develops self-confidence - talking at conferences, , motivating others, advocacy and training

Sister feels confident to communicates with the Nessie Knight hospital doctor using the technology

Some communication with other provincial doctors and specialists (Umtata, East London)

Sister builds up experience and a body of case studies, reading and ongoing skills development on the internet (Consulting medical documents for improved health care) sharing new knowledge with other health workers

The sister’s experience grows and she performs simple diagnoses of patients with some support by the doctor

The clinic sister feels confident and empowered showing off and training other healthworkers.

Sister gives inputs into provincial telehealth

discussions and influences telehealth plans

The community are not afraid to be diagnosed through the use of the technology

Sister accesses the internet for information and uses it for ongoing skills development

Sister articulates a vision of telehealth to the community

 

How? - Strategy MapHow? - Strategy Map

Strategy Causal I-1 Persuasive I-2 Supportive I-3

Strategies and activities aimed at a specific individual or group

Supply ICT equipment

Deliver training to the

sister

Make sure the system

is reliable

Provide technical

support

E-mail from doctors

requesting cases

Regular contact with

sister from CSIR

Brief the DoH

(Hospital super,

matron, DM, Dr

Rajeev, MEC with

regard to the system

to ensure ongoing

support for the sister

E-1 E-2 E-3

Strategies and activities aimed at individual’s or group’s environment

Include Telehealth in

sisters KRAs

Telehealth ICT policy

Frequent visitors to see

telehealth UNDP country visits

Promote the project through the media

Collaboration with other organisations

OM 3 STAGESOM 3 STAGES

• Intentional Design• Why? Who? What? How?

• Outcome & Performance Monitoring• Outcome journals• Strategy journals• Performance journal

• Evaluation Planning

Organisational PracticesOrganisational Practices

• Prospecting for new ideas• Opportunities and resources• Seeking feedback from key informants• Obtaining support from your next highest power• Assessing and redesigning products, services, systems and procedures• Checking on those already served to add value• Sharing your best wisdom in the world• Experimenting to remain innovative• Engaging in organisational reflection

Evaluation Evaluation

• Assess contribution to development outcomes• Understand linkages between behavioural changes and

strategies• User uptake, benefits of ICTs• Influence on regulatory policy

• Documentation of lessons learned• Understand intended uses and intended users• By using OM as a framework it allows comparison

across cases, across countries

Dissemination StrategyDissemination Strategy PI Users PI Uses PI Users PI Uses

Influence on regulatory policy : Department of Communication• Create license exemption for use of wifi in research

• Create license exmption for use of wifi for non-commercial use e.g. health & education

• Dissemination strategy – inner circle, LINK, DoC, NGOs, University

Influence on health policy : Department of Health Eastern Cape• Support for the use of wifi for telehealth in rural clinics

• Demonstrate value of eHealth training module

• Focus group key DoH decision makers,Provincial Head, Supply Chain Manager

ConclusionConclusion• OM as a research tool enabled us to answer the research questions – what is it that

you want to make known that was not known before? The research produced primary data in discovering something new. OM provided the route map and data gathering tools to produce evidence based results. The evaluation of these research results enabled us to answer the research question user uptake, influence on policy

• OM provided a methodology for the comparative analysis between projects, between countries

• OM provided the framework for planning, documenting the journey, and capturing the lessons learned during the life of the project. It helped the project team document “where it had been”, understand the development process, social learning & creative dialogue

• OM enabled the project team to review progress and adjust strategies when necessary recognising development does not have fixed goals but is a moving target.

• OM provides a framework for understanding who are the boundary partners with whom the project seeks to influence and can influence, what are the uses and who are the intended users of the research results.

Thank YouThank You

Chris Morris ICTs for Development

Tel: +27 (12) 841-2509

Cell: 082 892 9349

E-mail: cmorris@csir.co.za

URL: http://www.csir.co.za

URL: http://www.fmfi.org.za