1. John Q. Wong, MD, MSc Justine Joyce Alim, Jose Lorenzo Angeles, Pia Cerise Creencia, Arnel Christian Dy, Raymond Joseph Escalona Health Sciences Program 10 th DOH National Health Research for Action Forum 26 June 2009 Heritage Hotel

2. Introduction.

3. Targets: Financing Service Delivery Governance Regulation *based on Roberts’ five health system control knobs Outputs: Access Efficiency Quality FOURmula One (F1) Program (2006) Background Information Outcomes: Financial Equity Health Status Customer Satisfaction

4. Background Information Tie-up with the DOH’s Monitoring and Evaluation for Equity and Effectiveness (ME3) Project Develop a baseline analysis of health reform efforts in the different levels of the local health system: Provincial Municipal Level Propose an improved, more efficient, and less costly means of obtaining and processing health data

5. Significance of the Study One of the issues in the Philippine health system is providing a clear representation of health status in various areas with the quality of available secondary data…

6. Stakeholders  Government Mayor Governor  Department of Health  Outside institutions Grants providers Pharmaceuticals  The Filipino People the health of the family

7.  To determine and compare health reform status of selected F1 and non-F1 provinces  To determine the costs of an online data collection system and compare it with the currently-implemented methodology Objectives

8. Methodology.

9. Study Design Variables used are specified in the LGU Scorecard variables list Secondary data collection from: o PHO o PHIC provincial offices o Provincial and district hospitals o MLGUs o RHUs

10. Study Design Error minimized through: Decomposition of indicators Verification of definitions during o Data collection o Post-data collection Checking if other forms have the same data Verification of data from local sources o Phone calls o Email

11. Study Design Performed on 2007 data Measure of central tendency: median (w/ range) Descriptive Statistics Analytical Statistics Bivariate Analysis: Moses Test for Extreme Reactions Regression: Multiple Regression Cost Comparison Analysis Comparison of costs

12. Region VIII IV-B MIMAROPA Study Population F1 Non-F1 Pretest

13. 28 LGU Scorecard Indicators 1 Percent coverage of target population in endemic provinces with mass treatment for Filariasis 2 Percent coverage of target population in endemic provinces with mass treatment for Schistosomiasis 3 Annual parasite incidence for malaria 4 TB Case Detection Rate 5 TB Cure Rate 6 Percentage Fully Immunized Child (FIC) 7 Percentage of newborns initiated breastfeeding within one hour after birth 8 Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement 9 Percentage of Facility Based Deliveries

14. 28 LGU Scorecard Indicators 10 Contraceptive Prevalence Rate 11 Percentage of Households with access to safe water 12 Percentage of households with access to sanitary toilet facilities 13 Average length of stay in hospitals in days 14 Average occupancy rate for 1st to 3rd level public and private hospitals 15 Average hospital gross death rate from maternal causes 16 Basic Emergency Maternal & Obstretric Care (BEMOC) to population ratio 17 Percentage of RHUs accredited by Philhealth for OPB, MCP, and TB-DOTS package 18 Botika ng Barangay (BnB) to barangay ratio 19 Percentage of families enrolled In National Health Insurance Program (NHIP)

15. 28 LGU Scorecard Indicators 20 Percentage of poor families enrolled in NHIP 21 Percentage of provincial budget allocated to health 22 Percentage of Municipal Budget allocated to health 23 Percentage of Maintenance and Other Operating Expenses (MOOE) to total health budget 24 RHU/Health Center Physician to population ratio 25 RHU/Health Center Midwife to population ratio 26 Percentage of procurement packages completed through competitive bidding in PWHS 27 Percentage of annual financing utilized 28 Percentage of audit objections raised within the year that have been cleared

16. descriptive statistics per indicator in all four provinces based on 2007 health data includes mean, median, range, % incomplete  basis for the trimming down of indicators and municipalities for analysis Descriptive Statistics

17. Ranking used for determining levels of performance within a criterion and for comparison of multiple criteria with varied units 1 – highest rank Higher quotient value would have a higher rank except for #8: Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement #13: Average length of stay in hospitals in days #14: Average occupancy rate for 1 st to 3 rd level public and private hospitals #15: Average hospital gross death rate from maternal causes #23: Percentage of Maintenance and Other Operating Expenses (MOOE) to total health budget

18. Data segregated into provinces. Maintained municipalities were ranked based on their quotient values for each indicator.  Rank exactly at the middle of the range of all the indicator ranks attained by each municipality = median rank Median ranks of municipalities were again ranked ( overall rank ) and organized into quintiles. Vertical Analyses

19. Vertical Analysis: Comparison of Municipalities’ Overall Performances

20. Data segregated into provinces. Indicator performance based on quintile values  ranked  grouped into quintiles Indicators are classified under health reform pillars  Pillar Score = average of indicator rankings  Ranking of pillar performance Horizontal Analyses

21. Horizontal Analysis: Comparison of Performances in Each Municipality and Province

22. Moses Test for Extreme Reactions non-parametric (convenience sampling) determined which indicators had significant effects on health outcomes (F1 vs. non-F1) Multiple Regression measured the magnitude of the effects of the significant variables from bivariate analysis Bivariate Analysis & MR

23. Cost Comparison Analysis Head-to-head comparison of two cost categories – Workshop cost – Data Collection Cost

24. There was a constant display of: – Professional and Appropriate conduct – Proper etiquette in dealing with persons of higher authority and public office workers Ethical Considerations

25. Results.

26. Descriptive Analyses 1.Descriptive Statistics 2.Vertical Analysis 3.Horizontal Analysis Purpose To analyze the data set as a whole in regards to the mean, median, and range of values, as well as the skewness, kurtosis, and percentage of missing values for each indicator

27. 1. Descriptive Analysis – Based on the quotients from the raw data the minimum and maximum mean, median, and mode standard deviation, skewness, and kurtosis – Provided a count of the entries that were problematic – Total completeness of data: 95.34% 5 Variables were dropped, <20%

28. 2. Vertical Analysis Purpose To compare the performances of the province’s municipalities for each LGU indicator (representative of health projects and programs) To show the overall best and worst performing municipalities for each province

29. F1 Vertical Analysis Results Oriental MindoroSouthern Leyte MunicipalityQuintile Limasawa Malitbog Liloan Padre Burgos 1 Bontoc Macrohon Libagon 2 Hinundayan Tomas Oppus Sogod San Ricardo 3 Pintuyan San Juan Maasin City 4 Saint Bernard San Francisco Silago Baybay 5 MunicipalityQuintile Puerto Galera Victoria 1 Baco 2 Bongabong Calapan 3 Pola 4 Mansalay Gloria 5

30. Occidental MindoroLeyte Municipality Quintile Tolosa, Tunga, San Isidro, Tabon- Tabon, Alang-Alang, Santa Fe, Tabango 1 Leyte, Leyte, Julita, Merida, Hindang, Lapaz, Palo, Barugo 2 Albuera, Isabel, Pastrana, Dulag, Bato, Villaba, Tanauan 3 Mayorga, Matag-ob, Hilongos, Dagami, San Miguel, Matalom, Calubian 4 Capoocan, Palompon, Abuyog, Kananga, Jaro, Burauen, Carigara, Babatngon 5 MunicipalityQuintile Lubang Rizal 1 Magsaysay San Jose 2 Calintaan Paluan 3 Santa Cruz Abra de Ilog 4 Mamburao Looc 5 Non-F1 Vertical Analysis Results

31. Purpose To compare health reform programs within a municipality To show the overall performance of the health programs with in the province To show the overall F1 pillar performance within the province 3. Horizontal Analysis

32. Oriental MindoroSouthern Leyte IndicatorOverall RankQuintile V2 11 V25 21 V16 31 V24 41 V1 51 V10 62 V7 72 V17 82 V11 92 V12 92 V14 113 V23 123 V9 143 V15 153 V5 164 V19 174 V20 174 V18 194 V4 204 V8 215 V22 225 V13 235 V21 245 V6 255 PillarPillar Rank Governance1 Service Delivery2 Regulation3 Financing4 IndicatorOverall RankQuintile V2 11 V16 21 V18 31 V10 41 V1 51 V17 62 V7 72 V6 82 V8 92 V21 102 V23 113 V9 123 V24 133 V13 153 V11 164 V12 164 V22 184 V15 194 V25 204 V5 215 V14 225 V19 235 V4 245 V20 255 PillarPillar Rank Regulation1 Governance2 Financing3 Service Delivery4 F1 Horizontal Analysis Results

33. Occidental MindoroLeyte IndicatorOverall RankQuintile V21 11 V2 21 V16 31 V15 41 V6 51 V23 62 V14 72 V22 92 V 9 102 V11 113 V12 113 V8 133 V13 143 V19 153 V7 164 V1 174 V4 174 V25 194 V18 204 V24 215 V5 225 V17 235 V10 245 V20 255 PillarRank Financing1 Service Delivery2 Governance3 Regulation4 IndicatorOverall RankQuintile V2 11 V13 21 V16 31 V14 41 V5 51 V18 62 V8 72 V10 82 V22 92 V15 113 V17 123 V6 133 V24 143 V7 153 V1 164 V9 174 V11 184 V12 184 V25 184 V4 215 V19 255 V20 225 V21 235 V23 245 PillarPillar Rank Regulation1 Service Delivery2 Governance3 Financing4 Non-F1 Horizontal Analysis Results

34. 1.Bivariate Hypothesis Testing Moses Test Monte Carlo Simulation 2.Multiple Level Regression Modeling Statistical Analyses Statistical Analyses

35. 1. Bivariate Hypothesis Testing Purpose To determine whether F1 and non-F1 municipalities differ from each in terms of the 25 variables – Moses Test for Extreme Reactions – Non parametric – Non random sampling

36. Bivariate Hypothesis Testing Preparation All variables taken from 2007 data Missing replaced with mean Results: 14 Significant Variables

37. Significant Variables from Bivariate Analysis #Definition# 1 % coverage of target pop’n. in endemic provinces w/ mass treatment for Filariasis 16BEMOC to pop’n ratio 2 % coverage of target pop’n. in endemic provinces w/ mass treatment for Schistosomiasis 17% RHUs accredited by PHIC for OPB, MCP, TB-DOTS 5TB Cure Rate20Poor families enrolled in NHIP 7% newborns initiated breastfeeding w/in 1hr after birth 21% Provincial health budget allocated to health 8% PEM among 0-5 yrs. Old based on wt. for age anthropometric measurement 24RHU health center physician to pop’n. ratio 10Contraceptive Prev. Rate25RHU health center midwife to pop’n ratio 15Ave. hospital gross death rate from maternal causes

38. Bivariate Hypothesis Testing Table 3.15 Statistically Significant Variables based on the Results of Bivariate Analysis #The statistically significant variableWho performed better? 1 Percent coverage of target population in endemic provinces with mass treatment for Filariasis F1 2 Percent coverage of target population in endemic provinces with mass treatment for Schistosomiasis F1 5TB Cure RateF1 7 Percentage of newborns initiated breastfeeding within one hour after birth F1 8 Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement Non-F1 10Contraceptive Prevalence RateF1 15Average hospital gross death rate from maternal causesNon-F1 16 Basic Emergency Maternal & Obstetric Care (BEMOC) to population ratio F1 17 Percentage of RHUs accredited by Philhealth for OPB, MCP, and TB-DOTS package F1 20Poor families enrolled in NHIPF1 21Percentage of provincial budget allocated to healthNon-F1 24RHU/Health Center Physician to population ratioF1 25RHU/Health Center Midwife to population ratioF1 #The statistically significant variableWho performed better? 1 Percent coverage of target population in endemic provinces with mass treatment for Filariasis F1 2 Percent coverage of target population in endemic provinces with mass treatment for Schistosomiasis F1 5TB Cure RateF1 7 Percentage of newborns initiated breastfeeding within one hour after birth F1 8 Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement Non-F1 10Contraceptive Prevalence RateF1 15Average hospital gross death rate from maternal causesNon-F1 16 Basic Emergency Maternal & Obstetric Care (BEMOC) to population ratio F1 17 Percentage of RHUs accredited by Philhealth for OPB, MCP, and TB-DOTS package F1 20Poor families enrolled in NHIPF1 21Percentage of provincial budget allocated to healthNon-F1 24RHU/Health Center Physician to population ratioF1 25RHU/Health Center Midwife to population ratioF1

39. Bivariate Hypothesis Testing In only three out of the 13 variables does F1 fail to match up to its non-F1 counterparts— Average hospital gross death rate from maternal causes in all four provinces, the cases of maternal death was so few, it usually numbered from zero to three at the most, meaning to say having a case or two may be a result of chance the definition for maternal death was not standardized at the time of the study, thus the figures provided by the sources could have been erroneous.

40. Percentage of provincial budget allocated to health – majority of F1 municipalities had external sources of financing, hence they did need the same chunk of the province’s budget for health – more efficient use of resources by F1 provinces, having better results in other KRA’s

41. Percentage of Protein Energy Malnutrition among 0-5 years old based on weight for age anthropometric measurement (variable 8)

42. 2. Enter Multiple Regression Modeling Preparations Non-random sampling controlled – province population and the municipality populations – ratio of muni popln to prov popln After Pearson’s correlations: all were maintained

43. Interpretation Higher rank in LGU Scorecard significantly associated with – F1 status – Presence of a PHIC-accredited RHU These two factors account for 34% of an LGU’s performance in the scorecard 2. Enter Multiple Regression Modeling

44. Cost Comparison Analysis

45. Discussion.

46. Analogy LGUs as patients Dysfunctional health system as the disease Scorecard as a screening or diagnostic test Health reforms as treatment

47. Indicators in LGU Scorecard Indicators measure health and health-related events (signs and symptoms) Indicators are diagnostic tests Indicators lead to – Diagnosis and treatment

48. Wilson and Jungner’s Criteria for Disease Screening Disease Diagnostic test Diagnosis and treatment -Wilson JMG and Jungner G, Principles and Practice of Screening for Disease. WHO, Geneva: 1968.

49. Disease The condition sought should be an important health problem – Broken health system as a disease There should be a recognizable latent or early symptomatic stage – Not applicable since disease is already present

50. Disease The natural history of the condition, including development from latent to declared disease, should be adequately understood – Indicators measure intermediate health outputs that lead to the F1 health outcomes – However, many unknowns in process of health reform

51. There should be a suitable test or examination – Performance of LGU Scorecard needs to be tested by time The test should be acceptable to the population – Routine data Diagnostic Test

52. There should be an accepted treatment for patients with recognized disease – Interventions (reforms) have been defined and are available Diagnosis & Treatment

53. Facilities for diagnosis and treatment should be available – Need to build local capacity to Utilize the LGU scorecard Innovate and implement reforms

54. Diagnosis & Treatment There should be an agreed policy on whom to treat as patients – ME3 defines actions for each level of attainment of the scorecard

55. Diagnosis & Treatment The cost of case-finding (including diagnosis and treatment of patients diagnosed) should be economically balanced in relation to possible expenditure on medical care as a whole – Does intervening early lead to savings? – Difficult to determine

56. Diagnosis & Treatment Case-finding should be a continuing process and not a ‘once and for all’ project – LGU Scorecard meant to be an iterative process

57. Limitations and Recommendations.

58. Limitations of the Study LGU scorecard gave a picture of the health status only of public health system Convenience sampling Presence of confounding variables that couldn’t be statistically controlled – Geography Methodology

59. Limitations of the Study Incompleteness of data at source Lack of training of health workers – Led to personal interpretations of indicators – Omission of indicator Lack of data consistency Data Collection

60. Limitations of the Study Bureaucracy issues – Lack of endorsement – Some LGUs declined to share data Other issues..

61. Limitations of the Study Indicator list – Hard to satisfy conditions for multiple regression and Moses test Data Analysis

62. Recommendations Data collectors training – More practical tests and data checking exercises Clear briefing of performance-based payment scheme – Digressive system with regard to set deadline – Amounts to be based on specific situations Preparation of necessary permission documents Methodology

63. Recommendations Monitoring scheme – Constant Updating and Coordination Data collector’s kit content – Include extensive sample data set – Reference materials

64. Recommendations Use interval-ratio scale for indicators – Use variables in their original interval or ratio scale rather than transforming them into dichotomous variables Have measurable and quantifiable background indicators – To control for confounding Statistical Analyses

65. Recommendations More sophisticated statistical analysis – Multiple-level regression analysis accounts for different levels of local health system: provincial, district (ILHZ’s), and municipality

66. End.