using information for health management; part i
DESCRIPTION
Using Information for Health Management; Part I. - Health Information Systems Strengthening. Learning objectives . the information cycle ; tools and processes for turning data into action the relationship between data use and data quality hierarchy of standards / essential data set - PowerPoint PPT PresentationTRANSCRIPT
Using Information for Health Management; Part I- Health Information Systems Strengthening
2
Learning objectives
– the information cycle; tools and processes for turning data into action
– the relationship between data use and data quality
– hierarchy of standards / essential data set– common reasons for compromised data quality,
and various counter measures– different information products for
communicating different meanings
• Reflecting on the data that you have been working with …. What do you think are the steps that have been taken to get the data into the Kenya HMIS?
3
Client record cardsCollection
Processing
Presentation
Action
Tally sheetsEasy way of counting identical events that do not
have to be followed-up (e.g. headcounts, children weighed)
Collection
Processing
Presentation
Action
6
RegistersRecord data that need follow-up over long periods such as ANC,
immunisation, Family Planning, Tuberculosis (TB)
Collection
Processing
Presentation
Action
Collection
Processing
Presentation
Action
Key issues
Registers for CoC
Tally sheets
Tick registers
Reports
weekly,monthly,quarterly
Collection
Processing
Presentation
Action
• And now that the data is there – what now?
10
Information cycle; from data to action
11
Collection
Processing
Presentation
Action
12
Collection
Processing
Presentation
Action
• Data set based on minimum indicator set• Standard definitions• Data sources & tools
• Data quality checks • Data analysis: indicators
• Tables• Graphs• Reports
• Interpret information: comparisons trends• Decisions based on information• Actions
• Why do you think we need this information?
13
Planning cycle INDICATORS
Linking Planning with Information
Information cycle
Collection
Processing
Presentation
Action
Data Collection and Collation in a Health Facility (Zambia HMIS Procedure Manual)
15
Data collection – at the source of data creation (point of care)
Service data collected by nurses and doctors in-between attending to patients
Usually several (manual) steps before it is in any database/storage– Tally sheets– Tally sheet totals at end of month– Monthly summary forms which are
reported to the next levelOften too much to collect for already
overworked staff16
Collection
Processing
Presentation
Action
What data elements should be collected?
Cannot be obtained elsewhere (e.g. survey) Are easy to collect (cost vs usefulness) Do not require much additional work or time Can be collected relatively accurately Is part of one or more indicators
Collection
Processing
Presentation
Action
Essential data sets (EDS)Collection
Processing
Presentation
Action
Hierarchy of standards
Example of a National Data Dictionary
• ZA National Data Dictionary
19
Key issues
Top-down vs bottom-up approaches
Who to involve in discussions
Maximalist vs minimalist approaches
EDS Example: vaccination data
Input (community and facility levels)– Staff attendence, vaccines, to whom, when, where
Process (district)– # Children Vaccinated
Output (province)– Coverage of child immunization
Outcome (national)– Decreased incidence of vaccine preventable diseases
Impact (international)– Decreased mortality, healthier children
Collection
Processing
Presentation
Action
Prioritising data in the EDS:
Finagle’s Law:
The information you have is not what you want;The information you want is not what you need;The information you need is what you can get;The information you can get costs more than
you want to pay!
22Balancing varying information needs
Collection
Processing
Presentation
Action
Comparability of collected data
Stable standardised definitions– To ensure spatial comparability between different facilities,
districts, provinces and nations– To ensure comparability over time
What do you think about this statement:“Revising poor indicators /data sets /data elements may not be advisable due to cost and loss of backward comparability”
Collection
Processing
Presentation
Action
Characteristics of the aggregated anonimised DHIS data
24
Org.Unit Immunization Coverage May 2012
Whatever 70
Notsogood 40
Verybad 15
Superduper 98
Spatial /organizational dimension
Org.Unit A May 2012
Data Element 1 70
Data Element 2 65
Data Element 3 62
Data Element 4 98
Phenomenological dimension
OrgUnit AMonth 2012 Immunization Coverage
Jan 83Feb 80Mar 70Apr 52May 64Jun 60Jul 54
Aug 43Sep 37Oct 39
Temporal dimension
Where do we get data from?Routine data collection
– Routine health unit and community data• Activity data about patients seen and programmes run,
routine services and epidemiological surveillance• Semi-permanent data about the population served, the facility
itself and staff that run it– Civil registration (vital events being integrated with health e.g.
India)Non-routine data collection
– Surveys– Population census (headcounts proportion/facility catchment’s
area)– Quantitative or qualitative rapid assessment methods
Collection
Processing
Presentation
Action
Data Sources in the HMN data warehouse concept
Collection
Processing
Presentation
Action
In Summary: Data collection
32
Collection
Processing
Presentation
Action
Input: Using data sources and tools to collect quality data
Common problems:- Too much to collect- Poor understanding of data
collection tools- Timeliness of reporting- Low data quality
Output: relevant data
Data Processing:• What
observations can you make about your experience in processing the data so far?
33
Processing; assuring data quality and calculate indicators
- Turning data into information- How to assess data quality?- What are indicators, and why do we need
them?
37
Collection
Processing
Presentation
Action
Why checking data is vital?• Use of inaccurate data leads to
– Wrong priorities (focus on the wrong data)– Wrong decisions (not applying the right actions)– Garbage in = garbage out
• Producing data is expensive– Waste of resources to collect poor data
Collection
Processing
Presentation
Action
Routine data should be..
Reliable: Correct, Complete, Consistent
Timely: fixed deadlines for reporting
Actionable: no action = throw data away
Comparable:same numerator and denominator definitions used by all data processers BUT striving for comparability can compromise local relevance
Collection
Processing
Presentation
Action
Complete data?
Spatial: submission by all (most) reporting facilities Timely: is the data available within the required time Temporal: can you do analysis over time?
Collection
Processing
Presentation
Action
41
Timely data?• Late reports weaken the potential for comparison,
action can be too late,but still useful for documenting trends;
• Better to use the data that you have even if incomplete: “Perfection is the enemy of good”
Collection
Processing
Presentation
Action
Correct data? Are we collecting the data
we need? The data values seems
sensible/plausible? The same definition
applied uniformly? Are there any preferential
end digits used?
JAN FEB MARCH APRIL MAY JUNE JULY
250 230 245 225 230 240 250
Collection
Processing
Presentation
Action
Consistent data?
Data in the similar range as this time last year or similar to other organization units
No large gaps or missing data
No multiplicity of data (same data from multiple sources –which one to trust?)
Collection
Processing
Presentation
Action
What are the causes of poor data quality?
- Too many forms to fill out that are not useful to health workers- Absent data collection tools (Nigeria)- Data collection tools are poorly designed and hard to understand- Too many steps of manual aggregation and transfer of figures (next slide)- Limited feedback on data quality to those who collect it- Data is not used
45
Collection
Processing
Presentation
Action
46
Mate KS, Bennett B, Mphatswe W, Barker P, Rollins N (2009) Challenges for Routine Health System Data Management in a Large Public Programme to Prevent Mother-to-Child HIV Transmission in South Africa. PLoS ONE 4(5): e5483. doi:10.1371/journal.pone.
ZA_Auditor General Findings on DQ:DQ Improvement Assessment
Doctor or nurse interacts with patient
Patient record
Data transcribed to
Sub-set of data recorded in register and/or tally sheet
Data capture in DHIS
Step 1
Step 2 Manual recording
Data quality affected by
Monthly summaries collated
Step 5
Monthly summary report compiledStep 3
Step 4
Data analysis and feedbackStep 6
Incomplete, illegible, undated data
Multiplicity of DCT’s, duplicated,
non-standardised
Inability to collate data accurately
Inability to collate data accurately
Data capture errorsIncorrect data elements
activatedValidation not done
No feedbackLittle data analysis by
program managers
Doctor or nurse interacts with
patient
Patient record
Sub-set of data recorded in register and/or tally sheet
Data capture in DHIS
Step 1
Step 2
Strategies to improve DQ
Monthly summaries
collated
Step 5
Monthly summary report
compiledStep 3
Step 4
Data analysis and feedbackStep 6
Incomplete, illegible, undated data
Multiplicity of DCT’s, duplicated,
non-standardised
Inability to collate data accurately
Inability to collate data accurately
Data capture errorsIncorrect data elements
activatedValidation not done
No feedbackLittle data analysis by
program managers
Training and skills development
Financial
Technology
In-service training and
formal courses
Supervision
Supervision
1) Use of DHIS daily data capture, eTools
2) Electronic sign-off of data
3) Facility level capture of ART & TB data
Supervision
Supervision
Supervision
1)Improve printing of DCT
2) Hardware & software at facilities
3) HIS Staffing
Formal courses:Data validation,
feedback, check it, etcSupervision
Data capture formsCorrect data
element activation
In-service training and
formal courses
In-service training and
formal courses
In-service training and
formal courses
Auto-reports as “push” feedback
Doctor or nurse interacts with patient
Patient record
Data transcribed to
Sub-set of data recorded in register and/or tally sheet
Data capture in DHIS
Step 1
Step 2Manual recording
eTool Scenarios: Excel Aggregation
Monthly summaries collated
Step 5
Monthly summary report compiledStep 3
Step 4
Data analysis and feedbackStep 6
Excel Aggregation Tool in facilities
Electronic data transfer
Easy to install and scale across facilities with computers on
site
Doctor or nurse interacts with patient
Patient record
Data transcribed to
Sub-set of data recorded in register and/or tally sheet
Data available in DHIS
Step 1
Step 2 Manual recording
eTool Scenarios: DDC in DHIS14
Monthly summaries collated
Step 5
Monthly summary report compiledStep 3
Step 4
Data analysis and feedbackStep 6
Daily data capture on DHIS14 in facilities
Electronic data transfer
• Already available for Midnight Census in hospitals
• Requires DHIS in facilities – useful for some larger PHC facilities
Doctor or nurse interacts with patient
Patient record
Data transcribed to
Sub-set of data recorded in register and/or tally sheet
Data available in DHIS
Step 1
Step 2 Manual recording
eTool Scenarios: DDC in DHIS2
Monthly summaries collated
Step 5
Monthly summary report compiledStep 3
Step 4
Data analysis and feedbackStep 6
Daily data capture on DHIS2 on central server
Electronic data transfer • Revolutionises the
availability of data and feedback processes;
• Aligns NIDS with DC tools immediately
• Use of tablets could replace paper registers
Doctor or nurse interacts with patient
Electronic Patient record
Data transcribed to
Sub-set of data recorded in register and/or tally sheet
Data capture in DHIS
Step 1
Step 2
Electronic data transfer
eTool Scenarios: EPR systems
Monthly summaries collated
Step 5
Monthly summary report compiledStep 3
Step 4
Data analysis and feedbackStep 6
• Potential expansion of the EPR systems to accommodate all kinds of chronic illnesses
What can be done to improve data quality?
1. Assess the cause by using theInformation Cycle as the basis
2. Programmatic Issues– Essential dataset– Feedback routines– Use of Information
3. Database validation mechanisms– Min/Max rules in software– Data validation rules, check for consistency in logic of data– Completeness and timeliness reports
56
Minimum and Maximum Values
0
500
1000
1500
2000
2500
3000
Jan Feb March April May June July
Num
ber
Minimum and Maximum Values
Maximum
Minimum
Collection
Processing
Presentation
Action
Indicators
Calculated by combining two or more pieces of data, so that– They can measure trends over time– They can provide a yardstick whereby facilities /
teams can compare themselves to others (spatial, organizational)
– monitor progress towards defined targets– Good for measuring change
To do this, indicators need to have a numerator and denominator
57
- measure service COVERAGE and QUALITY
Collection
Processing
Presentation
Action
58
Indicator typesCollection
Processing
Presentation
Action
Type Description ExampleCount Indicator
Number of events without denominator
Number of new HIV+ cases
Proportion Indicator
Numerator is contained in denominator
Immunisation coverage of children under 1 year of age
Ratio Indicator
Numerator is not contained in denominator
Number of maternal deaths per 100,000 live births in same time period
Rate Indicator
Frequency of the event in a specified time in a given population
Number of maternal deaths per 1,000 women of reproductive age in the population
– Maternal mortality ratio?“the number of maternal deaths per 100,000 live births in same time
period.”Numerator: Number of deaths assigned to pregnancy-
related causes during a given time intervalDenominator: Number of live births during the same time intervalMultiplier: 100,000
Millenium Development Goals have a set of proposed indicators [weblink]
denominatorindicator = numeratorX 100 = %
Collection
Processing
Presentation
Action
60
Atop the line – numerators(activities / interventions / events / observations / people)
a count of the event being measured
How many occurrences are there:
morbidity (health problem, disease)
mortality (death)
resources (manpower, funds, materials)
Generally raw data (numbers)
Collection
Processing
Presentation
Action
61
Under the line – denominators (population at risk)
size of target population at risk of the event
What group do they belong to:
- general population (total, catchment, target)
- gender population (male / female)
- age group population (<1, >18, 15-44)
- cases / events – per (live births, TB case)
Collection
Processing
Presentation
Action
An ideal indicator RAVES !!!
62
Collection
Processing
Presentation
Action
63
Indicators should beRELIABLE gives the same result if used by
different people
APPROPRIATE fits with context, capacity, culture and the required decisions
VALID truly measures what you want to measure
EASY feasible to collect the data
SENSITIVE immediately reflects changes in events being measured
Collection
Processing
Presentation
Action
Indicator OperationalizationDefining the sources of the data – both numerator & denominator (how is it to be collected?)
Determining the frequency of collection and processing of the indicator (How often should it be collected, reported, analyzed?)
Determining appropriate levels of aggregation(To where should it be reported and analyzed?)
Setting levels of thresholds and target
What will be the nature of the action (decision) once the indicator reaches the threshold?
Collection
Processing
Presentation
Action
Processing
65
Collection
Processing
Presentation
Action
Input: Relevant data
Processes:- Quality checks- Aggregation to relevant levels- Calculation of indicators- Analysis of data =>
information
Common problems:- Much irrelevant data- Low data quality- Limited knowledge of data
needs and analysis
Output: data converted to information