1. Nurse Education Webinar Series
Susan Hoffmann MSN, RN, NCSN
May 8, 2019

2. Care Coordination Session 6 Evaluation and Measuring outcomes

3. Objective
Participants will be able to evaluate potential strategies to help students and their families reach their identified health and academic goals through care coordination.

5. Evaluation Measurement Contents Documentation
Evaluating Outcomes of Care Coordination
Measuring Outcomes of Care Coordination
Documenting Outcomes of Care Coordination

6. Evaluation

7. Polling Question:
Is there a systemic way that care coordination is currently being evaluated in your school? YES/NO

8. Evaluation
Evaluating care coordination is related to quality improvement.
Focuses on:
Process and outcomes of care for the individual student
Process and outcomes of care for aggregates of students.
Inclusion of interdisciplinary team members.
System outcomes (school and community).
Evaluation of care coordination can be a component of the evaluation of the entire school health program and can be used in quality improvement.
It requires similar skills related to data collection and analysis as those used in quality improvement.
The evaluation will rely on the school nurse’s leadership skills and collaborative communication with members of the interdisciplinary team.
Public health principals are needed to be sure that the evaluation of individual students is done from a holistic approach and that it relates to the larger community.

9. Begin with the end in mind
Planned goals & outcomes provide structure for the evaluation plan.
Plan to evaluate the care coordination process.
Identify data sources and/or data points.
Plan a timeline and process for data collection.
Although we often think of evaluation as the last part of the nursing process it really begins with the plan. During the planning process it is important to consider how you will evaluate the process of care coordination (i.e. what you are doing) as well as the student outcomes and system outcomes. The timeline for evaluation should be included in the plan.

10. Evaluation of Individual Students
Begins during planning and based on SMART Goals
A good outcome evaluation requires a good process evaluation.
Evaluation of individual students should be able to be aggregated into groups of students. i.e all student receiving medications
Emphasize the need to begin during planning and to track the process at specific intervals. Process data is necessary for outcome data. It is better to track and evaluate fewer outcomes when beginning and get good data rather than tracking many outcomes poorly. The evaluation is directly linked to the SMART goals.

11. Evaluation (based on SMART Goals)
To evaluate outcomes, track program activities.
(Process before outcome)
Sources of data
Quantitative
Qualitative (stories)
Discuss the different types of qualitative and quantitative data and the strengths and weaknesses of each.
Process data is necessary for outcome data. Better to track and evaluate fewer outcomes when beginning and get good data rather than tracking many outcomes poorly.
This Photo by Unknown Author is licensed under CC BY-NC-ND

12. Types of Evaluation Data
Quantitative
Numbers which can be aggregated and analyzed with statistical procedures if resources are available.
Qualitative
Words and stories. Can provide a powerful illustration of the numbers.
Both types of data are important and need to be included in the plan and gathered systematically. They often can be obtained together. For example, you can develop a brief parent/teacher questionnaire with Likert type items such as “on a scale of 1 (not helpful) to 5 (very helpful) how would you rate the care coordination program?” These can be summarized and reported (means, frequencies, etc.) Another part of the survey could ask for an example describing how a student improved. They next two slides describe the differences in the data.

13. Evaluation: Quantitative Data
Quantitative data can be simple or complex. This slide is very busy and impossible to read, which is a mistake that people make when presenting quantitative data. It is taken from a Journal article and is appropriate for that audience.
How could the school nurse make this more understandable in a presentation in front of the school board? For example, bolding key numbers, splitting it into more than one slide, simplifying statistics. If it is a lay audience just mention the statistically significant relationships rather than giving all the details.

14. Evaluation: Qualitative Data
Our daughter, was diagnosed with Type I Diabetes. This came as quite a shock to her and to us. …our school nurse played a huge part in our daughter’s success. She met with each of her teachers, her bus driver and the front office staff. …. Our nurse gave us the confidence that our daughter would be able to handle all of these changes at school. We cannot imagine having to go through this without our nurse.
Stories can be powerful. Too often we neglect to systematically gather this type of information. Remember to get parent permission before sharing it with others.

15. Student /Family outcomes
Were the goals attained?
Does the student report a higher quality of life and improved self- management?
Are clinical indicators better (HgA1c, BMI, etc.)?
Is the student doing better academically (improved grades, attendance)?
Does the family feel that the student manages their illness better?
Is the family more supportive of the student?
Consider using surveys of parents, testimonials, etc.
Try to get data on outcomes that can be aggregated. For example, you might use a brief assessment before and after care coordination that includes items appropriate for any condition. For example, “Fifty students with diabetes or asthma reported that they felt more comfortable managing their illness after the school nurse provided care coordination.”

16. Economic Outcomes How much does the care coordination program cost?
How much does the care coordination program save in parent time, teacher time, and decreased use of the hospital and emergency department?
Consider brief end of year surveys to gather this information.
Economic impact is one of the hardest for the school nurse to evaluate and would likely require external consultation.
The reference below describes data from a collaborative evaluation completed by CDC (first author), NASN (Maughan), and a school district (Sheetz).
Wang, LY; Vernon-Smiley, M; Gapinski, M; Desisto, M; Maughan, E; Sheetz, A. (2014). Cost-Benefit Study of School Nursing Services JAMA Pediatr. 2014;168(7): doi: /jamapediatrics Published online May 19, 2014
Wang, LY; Vernon-Smiley, M; Gapinski, M; Desisto, M; Maughan, E; Sheetz, A. (2014). Cost-Benefit Study of School Nursing Services JAMA Pediatr. 2014;168(7): doi: /jamapediatrics Published online May 19, 2014

17. Education system Outcomes
Do teachers/principals feel that the program contributes to the educational mission of the school?
How much does the care coordination program cost?
Does the teacher recognize the value of care coordination in improving their ability to teach?
How much does the care coordination program save in parent time, teacher time?
It’s important to get evaluations from other members of the health care team and families in order to have an idea of how care coordination affects the system and how the system affects care coordination.
Consider brief end of year surveys to gather this information.

18. Health Care Coordination/System Outcomes
Does the primary care provider report improved outcomes ?
Did the school nurse help the family obtain health insurance or a primary care provider?
How much does the care coordination program save in decreased use of the hospital and emergency department?
Discuss the need for school nurses to collaborate with NASN, CDC, and others to demonstrate the value of school nursing.

19. Resources for Gathering, Managing, and Presenting Evaluation Data
Quantitative Date
Hardware/
Software
Resources in the School System
Qualitative Data
Resources external to the school system
What are examples of existing student /school health data - resources - that can be used as a benchmark when evaluating care coordination or to build the case for care coordination programming at school?
Examples:
School Health Profiles
Youth Risk Behavior Survey
State health data
District health data
Consider a request for funding to hire a consultant for program evaluation. Outline “deliverables” if the funds are received. i.e. better data that can be used to compare with external benchmarks.
Hardware and software-This may be within the school system (IT department) or contracted with an external source).

21. NURSE DAY SCHOOL NURSES: MAKING THE GRADE ON SCHOOL HEALTH
MAY 8, 2019

22. Works cited
Wang, LY; Vernon-Smiley, M; Gapinski, M; Desisto, M; Maughan, E; Sheetz, A. (2014). Cost-Benefit Study of School Nursing Services JAMA Pediatr. 2014;168(7): doi: /jamapediatrics Published online May 19, 2014