1. September 12, 2006 RM-MSMSP Evaluation Presentation for Leadership Team Meeting Presented by Bonnie Walters Organized by Barbara P. Heath

2. Evaluation and Research Collaboration Provides opportunity to –Extend the evaluation process into research –Share data –Increase efficiency –Create project-level database

3. Refer to timeline handout Timeline: Year 3 Data Collection Combines the evaluation and research sources (denoted by shading) Includes description of: –Instruments –Data collection process –Random sample cycle –Summer camp outline

4. Summer 2006 Data Collection TCI Course Packs Participant Post-Course Survey Development Team Survey Instructional Team Survey RTOP Observations Camp Surveys and Observations

5. Analysis In process –Individual reports for each instructional team –Case studies for each summer camp –Summary report for Management Team

6. Preliminary TCI Results, SCI (results are statistically significant at the 0.001 level) CourseMax ScoreNormalized Gain Effect Size BIO I400.481.41 BIO II440.351.25 ESC I750.581.89 ESC II430.913.8 PHY I250.431.5 PHY II200.662.23

7. Preliminary TCI Results, MATH (results are statistically significant at the 0.001 level) CourseMax ScoreNormalized Gain Effect Size Discrete120.491.69 Modeling1390.491.68 History450.743.35 Geometry1000.451.18 Stats400.420.7

8. Participant Post-Course Survey: Response Rates Physics I63.64 Biology I52.63 Earth Science I65.00 Geometry68.42 History of Math61.54 Chemistry I58.33 Physics 246.67 Biology 259.09 Algebra83.33 Math Modeling63.16 Chemistry 260.87 Statistics66.67 Discrete Math70.59 Math of Change75.00 WY Earth Science52.00

9. Refer to handout Participant Post-Course Survey

10. 0=Poor --> 5=Excellent Participant Post-Course Survey The overall quality of this course was…

11. Participant Post-Course Survey: % of Respondents who would recommend the course to others BIO I100% BIO II85% PHY I100% PHY 286% CHM I86% CHM II86% ESC I86% ESC II100% ALG95% Math Chng100% Discrete100% GEOM100% Hist. Math88% Modeling100% Stats71%

12. 0=Strongly Disagree --> 5=Strongly Agree Participant Post-Course Survey The course material was presented at an appropriate level.

13. 0=Strongly Disagree --> 5=Strongly Agree Participant Post-Course Survey The course stimulated my thinking about the content of (subject).

14. 0=Strongly Disagree --> 5=Strongly Agree Participant Post-Course Survey Interaction with classmates assisted with my learning.

15. Refer to handout Participant Post-Course Survey

16. Preliminary Conclusions All courses demonstrate increase in participant content knowledge on the individual Teacher Content Inventories (TCI). Participants report high level of satisfaction with the individual courses they completed Higher percentage of participants report they enroll in the courses to improve their teaching, lower percentage to earn stipend and graduate credit (compared to 2005 data)

17. Acknowledgements Dr. Aruna Lakshmanan, Statistician Aaron Perlmutter, EMEC Project Coordinator Alex Cramond, EMEC Summer Staff Brandi Whaley, EMEC Staff

18. Contact Information Barbara P. Heath, Ph.D. East Main Educational Consulting, LLC Wilmington, NC bpheath@bizec.rr.com (910) 620-4089