1. KMC Workshop Group E Monitoring and Evaluation

2. Clarification of Concepts Monitoring: vigilance of a process Evaluation: assessment, value judgment about a process and its results. The group decided that it was going to focus on monitoring, rather than on the specifics of evaluation

3. Clarification of Concepts Monitoring in KMC can have several meanings: – Vigilance of the implementation of KMC at different levels Global (WHO) Regional National Local, etc. – Vigilance of an ongoing program, again: Global Regional, etc.

4. Clarification of Concepts Prior to defining what to monitor during implementation, goals and plans for implementation should be stated. Main purposes of monitoring then, would be: – Surveillance of the compliance with implementation tasks and steps, timetable, etc. – Quantification of performance (e.g. number of trained health workers, etc.) – Quantification of achieved goals (e.g. mortality reduction)

5. Clarification of Concepts Monitoring of an ongoing program can also have several purposes and objectives The group identified monitoring a as a health care quality assurance tool as a very important purpose. Most of the following discussion was centered then on “evidence-based quality assurance” for KMC programs The importance of monitoring for implementation was nevertheless acknowledged as well as the need for address it a next step

6. Objectives To identify elements and domains relevant for developing appropriate monitoring tools for quality assessment-improvement (quality assurance) of an ongoing KMC program. To generate a series of statements which can help to define good practices for monitoring KMC programs To develop the process, a hypothetical program for delivering KMC in a health facility was sometimes used. Extrapolation to other settings and to broader scopes can be made.

7. The EB-Quality improvement cycle Evaluate current practice, identify problems – Separation of mother and infant after birth – Low breast feeding rates – Undesired variability in practices and/or in outcomes Plan and implement interventions – Set quality standards for practice (e.g. clinical practice guidelines, evidence- based) – Set quantitative goals – Implement practices Monitor – Compliance with requirements – Performance Compliance with guidelines Frequency of selected outcomes Adjust performance according to monitoring Evaluate results (close the cycle, go back to 1 st step) and start all over

8. Delivering KMC implies A KMC program: – Resources, administration, planning, management, put together to deliver The KMC intervention – Set of specific processes (interventions ) for caring for the health care of newborn infants involving and empowering their mothers-families – Using a specific method or technique The KMC method, a complex non-pharmacological intervention clearly standardized, defined and supported by scientific evidence. – The method is defined as EB recommendations, usually in the form of structured detailed protocols.

9. Components of a KMC program as related to health care quality Structure Processes Outcomes (Donabedian)

10. Structure Stable part of the Health care system, that provides the support and setting in which health KMC is delivered: – Physical structure – Administrative structure and processess – Norms – Resources Technical Human Capital

11. Process Actual specific health care interventions and procedures employed for providing care – Delivered interventions should be backed by scientific evidence supporting that they do more good than harm – They should be feasible: Available Affordable Proficiently performed – Properly trained personal – Technically appropriate equipment Acceptable for – Target population and – Health care personnel Ethically appropriate

12. Outcomes Changes in different aspects of health associated with previously delivered interventions – Disease (condition) centered Mortality Morbidity, complications and sequels Time to event – Patient centered Growth and development Satisfaction Health related quality of life

13. Monitoring KMC programs for Quality assurance Structure: – Standards should be set Rate of nurses to patients Locative facilities for mothers to provide kangaroo position Etc. – Verification that minimum acceptable standards are met (Basic part of certification? Accreditation?) Process – Recommendations (evidence-based guidelines) have identified processes known to do more good than harm – Compliance with recommendations is monitored Outcomes – Given that there should be evidence showing that recommended processes do more good than harm not every specific health outcome needs to be monitored – Nevertheless, given that despite having evidence-based recommendations, thing can go wrong (poor performance, other quality issues) some important and selected health outcomes should be monitored.

14. Proposed framework to identify what to monitor StructureProcessCondition centered outcome Patient centered outcome Method (Technique) Intervention Program

15. An example ElementStructure Method Does the program has Guidelines or Protocols? Explicit, witten? Clear identification of components? Standardized definitions of components? Identification of therapeutic goals for each component? Evidence-based recommended courses of action? Explicitly defined setting? Explicitly defined target population? Clear inclusion / exclusion criteria Clear role definition for health care personnel?

16. An example (cont.) ElementStructureProcessOutcomes InterventionProperly trained health care personnel Compliance with InitiationMother recruitment processes and inclusion criteria Infant selection criteria KPKangaroo adaptation including monitoring Kangaroo nutrition strategy initiation KNApropiate support for breast feeding Apropiate use of feeding methods Approriate monitoring of feeding Maintenance of position Follow up policiesDischarge - Follow up procedures mortality KMC-related events Hospital stayClinic visits Mother mood depresion Bonding indicators Attachment indicators Parents satisfaction

17. Examples of specific indicators Performance indicators of a KMC program – LBW Infants in KP first visit to the clinic/total LBW infants discharged from hospital Minimum acceptable 90% Optimal 95% (there are infants discharged after leaving KP) Measures compliance with KP between discharge and first visit to out KMC clinic – Hours a day a parent can stay at the NICU Minimum acceptable 12 /24 Optimal 24/24 Measures appropriateness of structure: – Regulations and norms to remove access barriers – Amenities, comfort, space to allow parents to stay

18. Examples of specific indicators: the “SKIND” score Item012 SSC in 1st hour<50>5060 ‘ KP duration in the 1 st 6 hours < 4 h<5.5>5.5 h Initiate BF (Self attachment observed) No1-2 h1 st hour Nutrition, Artificial feeding given? AIFClear fluidsNone Delayed Procedures for 6 h Nonebath all

19. Recommendations Setting Standards for structure and making sound evidence-based recommendations which guide processes is indispensable prior to proposing any reasonable plan for monitoring Identification of “key indicators” amenable for “universal” use can be attempted afterwards A group within INK should work on a guide for developing good monitoring practices for KMC quality assurance An inventory of available instruments for monitoring and assessing implementation and for quality assurance on ongoing KMC should be assembled – The group already identified several research and monitoring instruments both general and specific – Instruments for monitoring the progress of Implementation – Instruments for monitoring quality of care