john q. wong, md, msc justine joyce alim, jose lorenzo angeles, pia cerise creencia,

John Q. Wong, MD, MScJustine Joyce Alim, Jose Lorenzo Angeles, Pia Cerise

Creencia,Arnel Christian Dy, Raymond Joseph Escalona

Health Sciences Program10th DOH National Health Research for Action Forum26 June 2009Heritage Hotel

Introduction.

Targets:• Financing• Service Delivery• Governance• Regulation

*based on Roberts’ five health system control knobs

Outputs:AccessEfficiencyQuality

FOURmula One (F1) Program (2006)

Background Information

Outcomes:Financial Equity

Health StatusCustomer Satisfaction

Background Information

Tie-up with the DOH’s Monitoring and Evaluation for Equity and Effectiveness (ME3) Project

Develop a baseline analysis of health reform efforts in the different levels of the local health system:• Provincial• Municipal Level

Propose an improved, more efficient, and less costly means of obtaining and processing health data

Significance of the Study

One of the issues in the Philippine health system is providing a clear representation of health status in various areas with the quality of

available secondary data…

StakeholdersStakeholdersGovernment

• Mayor• Governor

Department of Health

Outside institutions• Grants providers• Pharmaceuticals

The Filipino People• the health of the family

To determine and compare health reform status of selected F1 and non-F1 provinces

To determine the costs of an online data collection system and compare it with the currently-implemented methodology

Objectives

Methodology.

Study Design

• Variables used are specified in the LGU Scorecard variables list

• Secondary data collection from:o PHOo PHIC provincial officeso Provincial and district hospitalso MLGUso RHUs

Study Design

Error minimized through:• Decomposition of indicators

• Verification of definitions duringo Data collectiono Post-data collection

• Checking if other forms have the same data

• Verification of data from local sourceso Phone callso Email

Study Design

• Performed on 2007 data• Measure of central tendency: median (w/ range)

Descriptive Statistics

Analytical Statistics• Bivariate Analysis: Moses Test for Extreme

Reactions• Regression: Multiple Regression

Cost Comparison Analysis• Comparison of costs

Region VIIIRegion VIIIIV-B MIMAROPAIV-B MIMAROPA

Study Population

F1F1

Non-F1Non-F1

Pretest

28 LGU Scorecard Indicators

1 Percent coverage of target population in endemic provinces with mass treatment for Filariasis

2 Percent coverage of target population in endemic provinces with mass treatment for Schistosomiasis

3Annual parasite incidence for malaria

4TB Case Detection Rate

5TB Cure Rate

6Percentage Fully Immunized Child (FIC)

7Percentage of newborns initiated breastfeeding within one hour after birth

8 Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement

9Percentage of Facility Based Deliveries


10 Contraceptive Prevalence Rate

11 Percentage of Households with access to safe water

12Percentage of households with access to sanitary toilet facilities

13 Average length of stay in hospitals in days

14Average occupancy rate for 1st to 3rd level public and private hospitals

15 Average hospital gross death rate from maternal causes

16Basic Emergency Maternal & Obstretric Care (BEMOC) to population ratio

17Percentage of RHUs accredited by Philhealth for OPB, MCP, and TB-DOTS package

18 Botika ng Barangay (BnB) to barangay ratio

19Percentage of families enrolled In National Health Insurance Program (NHIP)


20Percentage of poor families enrolled in NHIP

21Percentage of provincial budget allocated to health

22Percentage of Municipal Budget allocated to health

23 Percentage of Maintenance and Other Operating Expenses (MOOE) to total health budget

24RHU/Health Center Physician to population ratio

25RHU/Health Center Midwife to population ratio

26Percentage of procurement packages completed through competitive bidding in PWHS

27Percentage of annual financing utilized

28Percentage of audit objections raised within the year that have been cleared

• descriptive statistics per indicator in all four provinces• based on 2007 health data• includes mean, median, range, % incomplete

basis for the trimming down of indicators and municipalities for analysis



Ranking• used for determining levels of performance within a criterion and for comparison of multiple criteria with varied units• 1 – highest rank• Higher quotient value would have a higher rank except for

#8: Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement

#13: Average length of stay in hospitals in days

#14: Average occupancy rate for 1st to 3rd level public and private hospitals

#15: Average hospital gross death rate from maternal causes

#23: Percentage of Maintenance and Other Operating Expenses (MOOE) to total health budget

• Data segregated into provinces.• Maintained municipalities were ranked based on their quotient values for each indicator.

Rank exactly at the middle of the range of all the indicator ranks attained by each municipality =

median rank• Median ranks of municipalities were again ranked (overall rank) and organized into quintiles.

Vertical Analyses

Vertical Analysis: Comparison of Municipalities’ Overall Performances

Table 3.5 Occidental Mindoro Vertical Analysis Results

Municipality Overall Rank Quintile

Median Rank

Lubang 1 1 3

Rizal 2 1 4.5

Magsaysay 3 2 5

San Jose 4 2 5

Calintaan 5 3 5

Paluan 5 3 4.5

Santa Cruz 7 4 6

Abra de Ilog 8 4 5.5

Mamburao 9 5 7

Looc 10 5 8.5

• Data segregated into provinces.• Indicator performance based on quintile values

ranked grouped into quintiles

• Indicators are classified under health reform pillars

Pillar Score = average of indicator rankings Ranking of pillar performance

Horizontal Analyses

Horizontal Analysis: Comparison of Performances in Each Municipality and Province

Table 3.6 Northern Leyte Indicator Performance Ranking

Indicator Overall Rank Quintile V21 1 1 V2 2 1

V16 3 1 V15 4 1 V6 5 1

V23 6 2 V14 7 2 V22 9 2 V 9 10 2 V11 11 3 V12 11 3 V8 13 3

V13 14 3 V19 15 3 V7 16 4 V1 17 4 V4 17 4

V25 19 4 V18 20 4 V24 21 5 V5 22 5

V17 23 5 V10 24 5 V20 25 5

Table 3.7 Northern Leyte Pillar Score Ranking

Pillar Pillar Score Rank Financing 11.2 1 Service Delivery 11.73333333 2 Governance 20 3 Regulation 21.5 4

*Pillar scores were computed as average of overall ranks of indicators per pillar.

• Moses Test for Extreme Reactions• non-parametric (convenience sampling)• determined which indicators had significant effects on health outcomes (F1 vs. non-F1)

• Multiple Regression• measured the magnitude of the effects of the significant variables from bivariate analysis

Bivariate Analysis & MR

Cost Comparison Analysis

• Head-to-head comparison of two cost categories– Workshop cost– Data Collection Cost

• There was a constant display of:– Professional and Appropriate conduct– Proper etiquette in dealing with persons of higher

authority and public office workers

Ethical Considerations

Results.

Descriptive AnalysesDescriptive Analyses

1. Descriptive Statistics2. Vertical Analysis3. Horizontal Analysis

PurposeTo analyze the data set as a whole in regards to the mean, median, and range of values, as well as the skewness, kurtosis, and percentage of missing values for each indicator

1. Descriptive Analysis– Based on the quotients from the raw data

• the minimum and maximum • mean, median, and mode • standard deviation, skewness, and kurtosis

– Provided a count of the entries that were problematic

– Total completeness of data: 95.34%

5 Variables were dropped, <20%

2. Vertical AnalysisPurposeTo compare the performances of the province’s

municipalities for each LGU indicator (representative of health projects and programs)

To show the overall best and worst performing municipalities for each province

F1 Vertical Analysis ResultsOriental Mindoro Southern Leyte

Municipality QuintileLimasawaMalitbog

LiloanPadre Burgos

1

BontocMacrohonLibagon

2

HinundayanTomas Oppus

SogodSan Ricardo

3

PintuyanSan Juan

Maasin City4

Saint BernardSan Francisco

SilagoBaybay

5

Municipality Quintile

Puerto GaleraVictoria

1

Baco 2

BongabongCalapan

3

Pola 4

MansalayGloria

5

Occidental Mindoro Leyte Municipality QuintileTolosa, Tunga, San

Isidro, Tabon-Tabon, Alang-

Alang, Santa Fe, Tabango

1

Leyte, Leyte, Julita, Merida, Hindang,

Lapaz, Palo, Barugo

2

Albuera, Isabel, Pastrana, Dulag,

Bato, Villaba, Tanauan

3

Mayorga, Matag-ob, Hilongos, Dagami,

San Miguel, Matalom, Calubian

4

Capoocan, Palompon, Abuyog,

Kananga, Jaro, Burauen, Carigara,

Babatngon

5

Municipality QuintileLubang

Rizal1

MagsaysaySan Jose

2

CalintaanPaluan

3

Santa CruzAbra de Ilog

4

MamburaoLooc

5

Non-F1 Vertical Analysis Results

PurposeTo compare health reform programs within a

municipality

To show the overall performance of the health programs with in the province

To show the overall F1 pillar performance within the province

3. Horizontal Analysis

Oriental Mindoro Southern Leyte

Indicator Overall Rank Quintile

V2 1 1V25 2 1

V16 3 1

V24 4 1

V1 5 1

V10 6 2V7 7 2

V17 8 2

V11 9 2V12 9 2

V14 11 3

V23 12 3V9 14 3

V15 15 3

V5 16 4

V19 17 4

V20 17 4V18 19 4V4 20 4V8 21 5

V22 22 5V13 23 5V21 24 5V6 25 5

Pillar Pillar RankGovernance 1Service Delivery 2Regulation 3Financing 4

Indicator Overall Rank QuintileV2 1 1V16 2 1V18 3 1V10 4 1V1 5 1V17 6 2V7 7 2V6 8 2V8 9 2V21 10 2V23 11 3V9 12 3V24 13 3V13 15 3V11 16 4V12 16 4V22 18 4V15 19 4V25 20 4V5 21 5V14 22 5V19 23 5V4 24 5V20 25 5

Pillar Pillar RankRegulation 1Governance 2Financing 3Service Delivery 4

F1 Horizontal Analysis Results

Occidental Mindoro Leyte

Indicator Overall Rank QuintileV21 1 1V2 2 1

V16 3 1V15 4 1V6 5 1

V23 6 2V14 7 2V22 9 2V 9 10 2V11 11 3V12 11 3V8 13 3

V13 14 3V19 15 3V7 16 4V1 17 4V4 17 4

V25 19 4V18 20 4V24 21 5V5 22 5

V17 23 5V10 24 5V20 25 5

Pillar RankFinancing 1Service Delivery 2Governance 3Regulation 4

Indicator Overall Rank QuintileV2 1 1

V13 2 1V16 3 1

V14 4 1

V5 5 1

V18 6 2

V8 7 2

V10 8 2V22 9 2

V15 11 3V17 12 3

V6 13 3

V24 14 3

V7 15 3

V1 16 4

V9 17 4

V11 18 4

V12 18 4

V25 18 4V4 21 5

V19 25 5

V20 22 5

V21 23 5

V23 24 5

Pillar Pillar RankRegulation 1Service Delivery 2Governance 3Financing 4

Non-F1 Horizontal Analysis Results

1. Bivariate Hypothesis TestingMoses TestMonte Carlo Simulation

2. Multiple Level Regression Modeling

Statistical AnalysesStatistical Analyses

1. Bivariate Hypothesis Testing

PurposeTo determine whether F1 and non-F1

municipalities differ from each in terms of the 25 variables

– Moses Test for Extreme Reactions– Non parametric– Non random sampling

Bivariate Hypothesis Testing

Preparation• All variables taken from 2007 data• Missing replaced with mean

Results: 14 Significant Variables

Significant Variables from Bivariate Analysis# Definition # Definition

1 % coverage of target pop’n. in endemic provinces w/ mass treatment for Filariasis

16 BEMOC to pop’n ratio

2 % coverage of target pop’n. in endemic provinces w/ mass treatment for Schistosomiasis

17 % RHUs accredited by PHIC for OPB, MCP, TB-DOTS

5 TB Cure Rate 20 Poor families enrolled in NHIP

7 % newborns initiated breastfeeding w/in 1hr after birth

21 % Provincial health budget allocated to health

8 % PEM among 0-5 yrs. Old based on wt. for age anthropometric measurement

24 RHU health center physician to pop’n. ratio

10 Contraceptive Prev. Rate 25 RHU health center midwife to pop’n ratio

15 Ave. hospital gross death rate from maternal causes

Bivariate Hypothesis TestingTable 3.15 Statistically Significant Variables based on the Results of Bivariate Analysis

# The statistically significant variable Who performed better?

1Percent coverage of target population in endemic provinces with mass treatment for Filariasis

F1

2Percent coverage of target population in endemic provinces with mass treatment for Schistosomiasis

F1

5 TB Cure Rate F1


F1

8Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement

Non-F1

10 Contraceptive Prevalence Rate F1

15 Average hospital gross death rate from maternal causes Non-F1

16Basic Emergency Maternal & Obstetric Care (BEMOC) to population ratio

F1


F1

20 Poor families enrolled in NHIP F1

21 Percentage of provincial budget allocated to health Non-F1

24 RHU/Health Center Physician to population ratio F1

25 RHU/Health Center Midwife to population ratio F1

# The statistically significant variable Who performed better?

1Percent coverage of target population in endemic provinces with mass treatment for Filariasis

F1

2Percent coverage of target population in endemic provinces with mass treatment for Schistosomiasis

F1

5 TB Cure Rate F1


F1

8Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement

Non-F1

10 Contraceptive Prevalence Rate F1

15 Average hospital gross death rate from maternal causes Non-F1

16Basic Emergency Maternal & Obstetric Care (BEMOC) to population ratio

F1


F1

20 Poor families enrolled in NHIP F1

21 Percentage of provincial budget allocated to health Non-F1

24 RHU/Health Center Physician to population ratio F1

25 RHU/Health Center Midwife to population ratio F1

2. Enter Multiple Regression Modeling

Preparations • Non-random sampling controlled

– province population and the municipality populations– ratio of muni popln to prov popln

• After Pearson’s correlations: all were maintained

Interpretation• Higher rank in LGU Scorecard significantly associated with

– F1 status– Presence of a PHIC-accredited RHU

• These two factors account for 34% of an LGU’s performance in the scorecard

2. Enter Multiple Regression Modeling

Cost Comparison Analysis

Discussion.

Analogy

• LGUs as patients• Dysfunctional health

system as the disease• Scorecard as a screening

or diagnostic test• Health reforms as

treatment

Indicators in LGU Scorecard

• Indicators measure health and health-related events (signs and symptoms)

• Indicators are diagnostic tests• Indicators lead to

– Diagnosis and treatment

Wilson and Jungner’s

Criteria for Disease Screening

• Disease• Diagnostic test• Diagnosis and treatment

-Wilson JMG and Jungner G, Principles and Practice of

Screening for Disease. WHO, Geneva: 1968.

Disease

• The condition sought should be an important health problem

– Broken health system as a disease • There should be a recognizable latent or early

symptomatic stage– Not applicable since disease is already present

Disease

• The natural history of the condition, including development from latent to declared disease, should be adequately understood– Indicators measure intermediate health outputs

that lead to the F1 health outcomes– However, many unknowns in process of health

reform

• There should be a suitable test or examination

– Performance of LGU Scorecard needs to be tested by time

• The test should be acceptable to the population

– Routine data

Diagnostic Test

• There should be an accepted treatment for patients with recognized disease

–Interventions (reforms) have been defined and are available

Diagnosis & Treatment


• Facilities for diagnosis and treatment should be available– Need to build local capacity to

• Utilize the LGU scorecard• Innovate and implement reforms


• There should be an agreed policy on whom to treat as patients– ME3 defines actions for each level of attainment

of the scorecard


• The cost of case-finding (including diagnosis and treatment of patients diagnosed) should be economically balanced in relation to possible expenditure on medical care as a whole– Does intervening early lead to savings? – Difficult to determine


• Case-finding should be a continuing process and not a ‘once and for all’ project– LGU Scorecard meant to be an iterative process

Limitations and Recommendations.

Limitations of the Study

• LGU scorecard gave a picture of the health status only of public health system

• Convenience sampling• Presence of confounding variables that couldn’t be

statistically controlled– Geography

Methodology


• Incompleteness of data at source• Lack of training of health workers

– Led to personal interpretations of indicators– Omission of indicator

• Lack of data consistency

Data Collection


• Bureaucracy issues– Lack of endorsement– Some LGUs declined to share data

Other issues..


• Indicator list– Hard to satisfy conditions for multiple

regression and Moses test

Data Analysis

Recommendations

• Data collectors training– More practical tests and data checking exercises

• Clear briefing of performance-based payment scheme– Digressive system with regard to set deadline– Amounts to be based on specific situations

• Preparation of necessary permission documents

Methodology

Recommendations

• Monitoring scheme– Constant Updating and Coordination

• Data collector’s kit content– Include extensive sample data set– Reference materials

Recommendations

• Use interval-ratio scale for indicators– Use variables in their original interval or ratio scale rather

than transforming them into dichotomous variables

• Have measurable and quantifiable background indicators– To control for confounding

Statistical Analyses

Recommendations

• More sophisticated statistical analysis– Multiple-level regression analysis accounts

for different levels of local health system: provincial, district (ILHZ’s), and municipality

john q. wong, md, msc justine joyce alim, jose lorenzo angeles, pia cerise creencia,

Documents