September 12, 2006 RM-MSMSP Evaluation Presentation for Leadership Team Meeting Presented by Bonnie Walters Organized by Barbara P. Heath

Evaluation and Research Collaboration Provides opportunity to –Extend the evaluation process into research –Share data –Increase efficiency –Create project-level database

Refer to timeline handout Timeline: Year 3 Data Collection Combines the evaluation and research sources (denoted by shading) Includes description of: –Instruments –Data collection process –Random sample cycle –Summer camp outline

Summer 2006 Data Collection TCI Course Packs Participant Post-Course Survey Development Team Survey Instructional Team Survey RTOP Observations Camp Surveys and Observations

Analysis In process –Individual reports for each instructional team –Case studies for each summer camp –Summary report for Management Team

Preliminary TCI Results, SCI (results are statistically significant at the level) CourseMax ScoreNormalized Gain Effect Size BIO I BIO II ESC I ESC II PHY I PHY II

Preliminary TCI Results, MATH (results are statistically significant at the level) CourseMax ScoreNormalized Gain Effect Size Discrete Modeling History Geometry Stats

Participant Post-Course Survey: Response Rates Physics I63.64 Biology I52.63 Earth Science I65.00 Geometry68.42 History of Math61.54 Chemistry I58.33 Physics Biology Algebra83.33 Math Modeling63.16 Chemistry Statistics66.67 Discrete Math70.59 Math of Change75.00 WY Earth Science52.00

Refer to handout Participant Post-Course Survey

0=Poor --> 5=Excellent Participant Post-Course Survey The overall quality of this course was…

Participant Post-Course Survey: % of Respondents who would recommend the course to others BIO I100% BIO II85% PHY I100% PHY 286% CHM I86% CHM II86% ESC I86% ESC II100% ALG95% Math Chng100% Discrete100% GEOM100% Hist. Math88% Modeling100% Stats71%

0=Strongly Disagree --> 5=Strongly Agree Participant Post-Course Survey The course material was presented at an appropriate level.

0=Strongly Disagree --> 5=Strongly Agree Participant Post-Course Survey The course stimulated my thinking about the content of (subject).

0=Strongly Disagree --> 5=Strongly Agree Participant Post-Course Survey Interaction with classmates assisted with my learning.

Refer to handout Participant Post-Course Survey

Preliminary Conclusions All courses demonstrate increase in participant content knowledge on the individual Teacher Content Inventories (TCI). Participants report high level of satisfaction with the individual courses they completed Higher percentage of participants report they enroll in the courses to improve their teaching, lower percentage to earn stipend and graduate credit (compared to 2005 data)

Acknowledgements Dr. Aruna Lakshmanan, Statistician Aaron Perlmutter, EMEC Project Coordinator Alex Cramond, EMEC Summer Staff Brandi Whaley, EMEC Staff

Contact Information Barbara P. Heath, Ph.D. East Main Educational Consulting, LLC Wilmington, NC (910)