data demand & use (ddu) why we collect health-related data
TRANSCRIPT
Data Demand & Use(DDU)
Why we collect health-related data
Session Overview Understanding the need for data
Importance of improving data informed decision making
Explain the context of decision-making Role of monitoring and evaluation data in decision making
Highlight determinants of data use
List potential barriers to data use Assessment of Barriers to Data Use Tool
Explain the relationship of stakeholders to the data use in decision making cycle Stakeholder Assessment and Engagement Tools
Linking decisions and questions with potential data sources Identify priority decisions and programmatic questions
Create a time-bound plan for using data in decision making
Framework for Linking Data with Action Tool
Understanding the need for data
Decisions Within Programs
Ensure program activities are executed as planned & services are delivered as intended Are we doing the right things, doing them right, and
reaching those we intended?
Decisions about: employment & manpower
mobilization & allocation of resources
needed information and developing feedback channels
Factors other than evidence-based information influence decisions
Power relationships
Timing
Competing priorities
Public opinion
Political ideology
Arbitrariness
Local culture of decision-making
Other information sources
Understanding Decision-Making
What is the decision to be made?
Who makes it?
When and why is the decision made?
How is the decision made?
What information is needed?
What is my role in decision making?
Why Improve Data Informed Decision Making?
HIV epidemic
Resurgence of TB
Continued prevalence of malaria
Pockets of stalled fertility decline
Population burden
Shortage of health care workers
Context
Pressing need to develop health policies, strategies and
interventions
Why Improve Data Informed Decision Making?
Increased financial investments for service delivery
Increased accountability requirements
Improved national HMIS
Increased demand for evaluation and other research
Why address data demand & use?
Evidence-based Decision Making Process
Level of Dissatisfaction that Policy is Based on Scientific Evidence
Per
cent
dis
satis
fied
Overseas Development Institute, Jones et al., 2008.
Challenges
Integrated HMIS still not fully functioning Little or no communication between data
producers and data users Low capacity to collect, analyze & interpret data Limited or no culture of data use
Data collection and use not a priority
The Response
Monitoring & Evaluation Systems
Better Health
Outcomes
Data Informed Decisions
Group Participation
How do you and your organization use data and information?
We can use information to…
Inform health policies and plans
Raise additional resources
Strengthen programs and improve results
Ensure accountability and reporting
Improve quality of services provided
Contribute to global lessons learned
“Making Data Speak” in Thailand
Need: Strengthen commitment of policy makers to HIV Prevention
Data: Behavioral and epidemiological data Response:
Analyzed data with Asian Epidemic Model and Goals model Determined responses and resources needed Communicated data to stakeholders
Decision/Action: Successfully emphasized prevention agenda in national
strategic plan and developed an operation plan to guide prevention programming
Key Messages
Decisions based on evidence lead to better
health outcomes
We all have a role in M&E – partners in progress
High quality information is needed for decision-
making at policy, planning and program levels
Purpose of M&E is not just to produce more
information but to inform action
Determinants of DDU
What Determines Data Demand & Use?
ORGANIZATIONALTECHNICAL
BEHAVIORAL
* Based on PRISM analytical framework (LaFond, Fields et al. 2005 The PRISM: An Analytical Framework for Understanding Performance of Health Information Systems in Developing Countries. MEASURE Evaluation).
Data are often underutilized because of…
Technical constraints
Individual technical skills,
Availability of computers,
Data system design,
Definition of indicators,
Lack of data quality assurance protocols
Data are often underutilized because of…
Organizational constraints
Structural – roads, telecommunication
Organizational – clarity of roles, support, flow of information
Political interference
Data are often underutilized because of…
Individual constraints
Decision maker attitudes,
Staff motivation,
Lack of “data culture”
What Determines Data Demand & Use?
ORGANIZATIONALTECHNICAL
BEHAVIORALPOLITICS
CULTURE
SOCIETY
* Based on PRISM analytical Fields et al. 2005 The PRISM: An Analytical Framework for Understanding Performance of Health Information Systems in Developing Countries. MEASURE Evaluation).
What barriers have you faced to using or getting others to use data and
information?
Discussion
Assessment of Data Use Constraints Tool
Purpose To improve understanding of the demand for data and the
constraints to data use
Description Key informant interview guide designed to identify
constraints
Identifies effective practices in data use
Two versions - Facility level assessment & national and sub-national assessment
Assessment of Data Use Constraints ToolTechnical Constraints
Technical constraints are related to the ability to generate high-quality data and analyses.
RA8 Have you ever had an experience while making a policy or program related decision when you were concerned about the quality of the information being used?
RA9 Are there multiple sources of information or statistics for issues of importance to you, and have you experienced any problems caused by having different estimates?
RA10 I am interested in knowing about technical capacity for collecting and using information. Does your agency have the technical capacity to produce reliable information without a lot of external technical assistance?
RA11 Does your agency have the technical capacity to ensure access to and availability of reliable data?
RA12 Has there been an occasion when data quality or local technical capacity made it difficult for you to use information in making a decision?
RA13 How would you have gone about preventing this situation?
Barrier: Steps involved
PersonResponsible
Other stake-holders
General timeline
Addressing Barriers to Using Data and Information in Decision-making
Activity:Assessment of Data Use Constraints ToolChoose a note taker
Discuss barriers to data use experienced in your work. Here are some questions to start your discussion:
Have you ever had an experience while making a policy or program related decision when you were concerned about the quality of the information being used?
Does your agency have the technical capacity to ensure access to and availability of reliable data?
What specific challenges have you experienced among your staff when it comes to using data?
How does your organization support having the necessary information to make decisions?
Time for activity: 45 minues
Barrier: Lack of technical capacity in M&E
Steps involved
PersonResponsible
Other stake-holders
General timeline
Proposed Intervention: Train all program managers in X organization on basic monitoring and evaluation (indicators, developing M&E plan, documenting results, managing towards results, etc.)
Identify funding
Director Deputy, program mgr
January2008
Seek out trainers and curricula
M&E Specialist
Deputy director, Training coordinator
February 2008
Addressing Barriers to Using Data and Information in Decision-making
Report Back on the Assessment of Data Use Constraints Tool
Share priority barriers
Discuss solutions crafted
Present action plan for two priority barriers
Report back: 2 groups, each 10 minutes
Context of decision-making
Discussion
How can we ensure that information is being used to make diagnoses
and inform decisions?
Stake-holders
Decisions
Context of Decision-making
Data
Stake-holders
Decisions
Context of Decision-making
Data
What is a Stakeholder?
Any person, group or organization with a particular interest in a policy or program Government agencies Beneficiaries Policymakers Funding agencies Providers / Implementers Civil society Researchers M&E Specialists
Stakeholders
Non-governmental organizations
Professional associations
Religious leaders
Journalists/media
Private sector/business
Data Producers vs. Data UsersData producers think that decision-makers:
• Value “political” considerations over evidence
• Are unprepared to measure or evaluate the consequences of their decisions
Decision makers or data users think that health researchers and M&E specialists:
Lack responsiveness to priorities
Favor numbers / jargon to transparent communication
Prefer written reports to face-to-face conversation
Importance of Knowing Your Stakeholders
View activities from different perspectives
Have different degrees of understanding
Need/want different information
Need information at different levels of complexity
Have different intensities of interest
Have different roles in the decision making process
Results of Involving Stakeholders in Data Use Process
Relevance of data
Ownership of data
Appropriate dissemination of data
Use of data
Stakeholder Analysis Matrix & Engagement Plan
Clarifies who has interest in a program
and what that interest is
Identifies who can help a program and
how, and who can hurt it
Helps you use this information for the
success of the planning effort
Stakeholder Analysis Matrix
The Stakeholder Analysis tool is a matrix framework and process for:
Identifying stakeholders
Defining their roles and resources
Identifying dynamics among stakeholders
Setting the optimum stakeholder group
Stakeholder Analysis Matrix
Name of stakeholder organization, group or individual
Stakeholder descriptionPrimary purpose, affiliation, funding
Potential role in the issue or activity
Level of knowledge of the issue
Level of commitmentSupport or oppose the activity, to what extent, and why?
Available resourcesStaff, money, technology, information, influence
Name of stakeholder organization, group or individual
Stakeholder descriptionPrimary purpose, affiliation, funding
Potential role in the issue or activity
Level of knowledge of the issue
Level of commitmentSupport or oppose the activity, to what extent, and why?
Available resourcesStaff, money, technology, information, influence
National AIDS Control Committee
Involved in planning, implementation, M&E of all HIV/AIDS programs in the country; approves donor and NGO-funded HIV/AIDS programs
Facilitate the stakeholder meeting, prepares for meeting by identifying data sources and preparing an agenda
High – receives reports on PMTCT activities from MCH division at MOH; Medium level of knowledge of int’l guidelines and studies
Strongly support the activity, but hesitant to use international data sources. NACC opposes use of the DHS and most recent international estimates as they consider these sources to overestimate HIV prevalence
Staff available to facilitate; Room and computers available for meetings at NACC headquarters
Stakeholder Analysis Matrix Program issue Develop plan (inc. M&E plan) to scale-up PMTCT programs throughout systemProposed activity Convene stakeholders to identify priorities based on available data and develop action planDate November 2006
How to Involve Stakeholder
Quarterly program management meetings Quarterly meetings to interpret RHIS data
Involvement of facility staff to interpret program data
M&E System improvement Indicator planning and/or harmonization
Data quality review meetings
Stakeholder Engagement Plan
Stakeholder organization, group
or individual
Potential role in the activity
Engagement strategy
How will you engage this stakeholder in the activity?
Follow-up strategyPlans for feedback or
continued involvement
Program issueProposed activityDate
Stakeholder Engagement Plan
Stakeholder organization, group or individual
Potential role in the activity
Engagement strategyHow will you engage this
stakeholder in the activity?
Follow-up strategyPlans for feedback or
continued involvement
National AIDS Control Committee
Facilitate the stakeholder meeting, prepares for meeting by identifying data sources and preparing an agenda that allows for the sources to be discussed
The NACC is the lead in this activity. It will be important for the NACC to involve more specifically the PMTCT coordinator, clinical care coordinator and National AIDS Program Coordinator
The NACC is responsible for following up with the stakeholders prioritized
Program issue Develop plan (inc. M&E plan) to scale-up PMTCT programs throughout systemProposed activity Convene stakeholders to identify priorities based on available data and develop action planDate November 2006
Stake-holders
Decisions
Context of Decision-making
Data
Decision Areas
Program design and evaluation Program management and improvement Strategic planning Advocacy and policy development
Program Design & Evaluation
Design Select messages for prevention campaigns
Evaluation
Determine if new program approaches are needed to ensure that health impact objectives are met
Program Management and Improvement
Management Determine if the program is meeting its
process objectives
Improvement Develop new strategies to increase
coverage
Strategic Planning
Identify geographic areas of highest need
Determine human resource allocation
Determine which of offered services is making the greatest impact
Advocacy and Policy Development
Identifying and quantifying underserved populations
Identifying focus areas for new policies
Stake-holders
Decisions
Context of decision-making
Data
Data and Information Census Vital events data Surveillance data Household surveys Facilities level service statistics Financial and management information Modeling, estimates and projections Health research
Stake-holders
Decisions
Factors Other than Data that Influence Decisions
Data
Political Ideology
Competing Priorities
Power Relationships
Public Opinion
Arbitrariness
Stakeholders Involve new counterparts
Strengthening the Decision-Making Process
Decisions Understand service delivery realities
Data May require additional data
Activity: Stakeholder matrix & engagement plan
Break into small groups
Each group should select a decision that they make in their work settings and complete the stakeholder analysis around that decision.
A minimum of 7 stakeholders should be identified
Complete the matrix across the columns for 1-2 stakeholders.
Select one stakeholder and complete the stakeholder engagement plan for that stakeholder
Time for activity: 45 minutes
Group Work Report Back
Have note taker transfer your final Stakeholder Analysis Matrix onto flip chart paper
Share the decision your group chose
Share the priority stakeholders selected
Choose 1 stakeholder and share the entire row from the matrix for that stakeholder
Share the Engagement Plan for the same stakeholder
2 groups report back: 10 minutes each
Linking decisions and questions with potential data sources
Building Data Use into Your Work
PLAN PLAN PLAN !
Regularly review your data – schedule time
Use the Framework for Linking Data With Action
Engage in dialogue with stakeholders to fully understand the
decisions they make
information they need
best way to present that information
Elements of the Framework
Decision makers and stakeholders with potential interest in your data
Decisions / Actions that the stakeholder makes (possible uses of data)
Questions to which the stakeholder requires answers
When the decision will be made
Elements of the Framework cont’d
Indicators and/or data of interest (to respond to stakeholder need)
Source of data
How will data be presented (what types of analyses, graphs, formats)?
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker(DM) Other Stakehold-ers (OS)
Indicator/data
Data Source
Timeline(Analysis) (Decision)
Commu-nication Channel
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker(DM) Other Stakehold-ers (OS)
Indicator/data
Data Source
Timeline(Analysis) (Decision)
Commu-nication Chanel
What Are Decisions?
Choices that lead to action
All decisions are informed by questions
All questions should be based data
Decisions Allocation of resources across countries/
states / districts/facilities
Revising OVC program approaches to emphasize fostering and adoption
Develop and institute workplace policies on HIV/AIDS in all institutions in state X
Hire and allocate staff to facilities
Programmatic Questions What percentage of HIV+ pregnant women in
care are actually delivering in health facilities?
What percentage of clients starting ART are lost to follow up?
Are the number of family planning clients decreasing?
What percentage of pregnant patients who are HIV+ are actually receiving ART?
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication ChanelDMOS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:Client?
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication ChanelDMOS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:Client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication ChanelDMOS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
Framework For Linking Data With Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication ChanelDMOS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
711 form indicators K41, B73, B 91
Service statistics Logistics manage-ment system
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication ChanelDM OS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
711 form indicators K41, B73, B 91
Service statistics Logistics manage-ment system
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication ChanelDMOS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
711 form indicators K41, B73, B 91
Service statistics Logistics manage-ment system
Dec. 2010,March 2011,June 2011, September 2011,December 2011
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication Chanel DMOS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
711 form indicators K41, B73, B 91
Service statistics Logistics manage-ment system
Dec. 2010
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication Chanel DMOS
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
711 form indicators K41, B73, B 91
Service statistics Logistics manage-ment system
Dec. 2010 Short summary presented to facility manager at weekly clinic meeting
Framework For Linking Data With Action
Decision/ Action
Program/Policy Question
Decision Maker (DM) Other Stakeholders (OS)
Indicator/data
Data Source
Timeline(Analysis)(Decision)
Commu-nication Chanel DMOS
Hire more PMTCT counsel-ors
Are we reaching testing targets in PMTCT?Do we have sufficient test kits?What is nurse:client?
DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training
711 form indicators K41, B73, B 91
Service statistics Logistics manage-ment system
Dec. 2010 Short summary presented to facility manager at weekly clinic meeting
Framework for Linking Data with Action
Creates a time bound plan for information informed decision making
Encourages greater use of existing information
Monitors the use of information in decision making
Activity:Framework for Linking Data with Action
Select a note taker
On flip chart paper create the Framework table
Brainstorm 3 decisions or questions in columns
1 & 2
Complete the remaining columns
Time: 1 hour
Small Group Activity - Report Back
Remaining group will have 10 minutes to present their completed Framework
Group discussion - are there other data sources that might have been used in this decision? Were there other stakeholders that should have been considered? (10 minutes)
MEASURE Evaluation is a MEASURE project funded by the
U.S. Agency for International Development and implemented by
the Carolina Population Center at the University of North Carolina
at Chapel Hill in partnership with Futures Group International,
ICF Macro, John Snow, Inc., Management Sciences for Health,
and Tulane University. Views expressed in this presentation do not
necessarily reflect the views of USAID or the U.S. Government.
MEASURE Evaluation is the USAID Global Health Bureau's
primary vehicle for supporting improvements in monitoring and
evaluation in population, health and nutrition worldwide.