1. Lessons Learned from Working with Absenteeism Data: A Local Evaluation informing positive community change Presented by: Dr. L. Shon Bunkley, CRP October 16, 2008

2. CRP’s local evaluation of a pilot truancy reduction project  Program background  6 months in operation (Dec ‘07 – May ‘08)  Collaboration between school district, police, county prosecutor, job and family services, juvenile court  Targets elementary and middle school students  Program components  3 unexcused absences = Warning letter from Prosecutor  5 unexcused absences = Mediation session  10 unexcused absences = Truancy board hearing  15 unexcused absences = Referral to Juvenile Court

3. CRP’s local evaluation of a pilot truancy reduction project (cont’d)  Major measures of success  All pilot schools meet or exceed the attendance standards set by CCS  Reduce by 50% the number of referrals to juvenile court from pilot schools  Reduce by 75% the rate of unexcused absences for students who participate in a mediation session  Several secondary measures, like 30 consecutive days of no absences

4. CRP’s local evaluation of a pilot truancy reduction project (cont’d)  Evaluation methodology  Literature review  Interviews Staff Stakeholders Parents and youth  Analysis of secondary data Student absences School attendance rates School referrals to juvenile court

5. CRP’s evaluation findings  Between December and February, 199 students were mediated  Unexcused absences among mediated students at the pilot schools declined anywhere from 36% to 76%  Nearly 100 (50%) mediated students had 30 days of no unexcused absences following mediation  Too early to gauge school level improvements in attendance and referrals to juvenile court

6. Lesson 1: Define measures and outcomes  Definition of truancy  Excused vs. unexcused absences  Definition of referral  Eligible vs. charges filed

7. Lesson 2: Caution against the assessment of impact of a pilot project  Assessing impact in 1 st year is made difficult by:  Project being operational for a limited amount of time Not enough time to iron out the “kinks” of the project Not enough time to expect to see marked change Limited amount of data for analyses  Influence of several other district initiatives aimed at reducing truancy, such as truancy sweeps  High student mobility within the district  Seasonal/time-specific attendance patterns  Moderating factors (e.g., substance abusing parent)

8. Lesson 3: Be aware of competing interests  Client need for data to support effectiveness of the project  In need of additional funding  Desire for replication  Evaluator’s responsibility to keep the evaluation objective and balanced

9. Lesson 4: Clarify data elements and collection procedures  What data elements are needed  Student vs. school-level data  Program stats vs. school/district stats  Ancillary indicators/facilitators of success, like GPA  Who will be responsible for collecting what data  School district vs. program staff  Timelines will be dependent upon when the district releases data  Reconciliation of attendance data

10. Lesson 5: Get clarity and agreement on how data will be analyzed  Rate of absences vs. number of absences  Average rate vs. median rate of absences  Inclusion/exclusion of students mediated at the end of the school year

11. Up Front… 1. Define measure and outcomes 2. Caution against the evaluation of the impact of a pilot project 3. Be aware of competing interests 4. Clarify data elements and collection procedures 5. Get clarity and agreement on how data will be analyzed

12. For more information Dr. L. Shon Bunkley, Senior Research Associate Community Research Partners (614) 224-5917, ext. 104 sbunkley@communityresearchpartners.org