1. Supplemental Educational Services in the State of North Carolina: Evaluation Findings and Activities Steven M. Ross & Martha J Alberg Center for Research in Educational Policy The University of Memphis http://www.memphis.edu/crep

2. Effectiveness (Student Achievement) Service Delivery and Compliance Customer Satisfaction Provider Survey District Coordinator Survey Principal/Site Coordinator Survey Teacher Survey Parent Survey Additional Tests State Tests Figure 1. Components of a Comprehensive SES/Evaluation Modeling Plan Overall Provider Assessment

3. SES Participation Percentages based upon approved providers and eligible districts. 2006-20072007-2008 Active Providers82%94% Districts Served68%71% Students Served5,2978,943

4. SES Survey Response Rates Percentages based upon approved providers and eligible districts and schools. Stakeholder2006-072007-08 Providers36%87% District Coordinators8%38% Principals/Site Coordinators 50%33% Teachers49%26% Parents71%69%

5. Do providers communicate regularly with school personnel and parents?

6. Are providers working with districts, schools, and parents to develop instructional plans geared to students needs?

7. Are you satisfied with the providers services?

8. Matched SES-Control Student Design The matched design is more rigorous and yields results that are more valid as compared to other available designs. As such, this design was chosen specifically for NC after great deliberation between CREP and DPI. Student-level NCEOG scores from the prior year are gathered for each SES student and matched control student. Control students are drawn from a pool of demographically similar students (prior achievement, ethnicity, gender) from the same schools. Control students are students who were eligible to receive SES, but were not served during the current year.

9. Does SES Raise Student Achievement? 2006-2007 READING RESULTS Directional Effects: % of 23 Providers 2007-2008 READING RESULTS Directional Effects: % of 26 Providers

10. Does SES Raise Student Achievement? 2006-2007 MATH RESULTS Directional Effects: % of 17 Providers 2007-2008 MATH RESULTS Directional Effects: % of 28 Providers

11. Considerations Provider and teacher communication is vital and has been lacking in both evaluation years. Collaboration between all stakeholders is necessary to increase focus on the individual needs of each SES student. School personnel and providers could meet to discuss ways to adapt and integrate tutoring services with classroom activities. Consistent attendance at tutoring sessions by students is vital for SES to yield results.

12. Considerations One can reasonably expect with the limited amount of tutoring (20-40 hours) a child receives, limited impact will be seen on his or her state assessment results. The results from these studies can be utilized to require accountability and improvement by providers. Policies or criteria should be defined for determining what constitutes satisfactory provider performance. A classification system for providers should be clearly defined such as: full status, probation, etc.