Principal Investigator: Dr. Ann Robinson Project Director: Ms. Kristy Kidd Evaluator: Dr. Jill Adelson In collaboration with: Museum of Science, Boston

PROJECT DESCRIPTION STEM Starters+ is designed to serve all students in Grade 1, identified talented students in Grades 2 through 5, and their teachers. Teachers are served through professional development on implementing engineering and science curricula and on talent spotting low-income and/or culturally diverse students. Grade 1 students are served through an engineering unit and a STEM biography linked to the unit content. Grades 2-5 students are served through engineering and science units and multiple STEM biographies linked to the unit content.

Teacher Baseline RESULTS ScaleGroup N Mean (SD)Range Engineering Pedagogical Content Knowledge Self-Efficacy (KS) Experimental (1.14)1.11 – 5.67 Comparison (1.19)1.00 – 5.33 Engineering Engagement Self- Efficacy (ES) Experimental (1.00)2.00 – 6.00 Comparison (1.62)1.00 – 6.00 Engineering Disciplinary Self-Efficacy (DS) Experimental (1.10)1.00 – 6.00 Comparison (1.32)1.00 – 6.00 Engineering Outcome Expectancy (OE) Experimental (0.93)1.80 – 6.00 Comparison (1.24)1.00 – 6.00 Teaching Engineering Self-Efficacy (TESS) Experimental (0.84)2.31 – 5.78 Comparison (1.01)1.00 – 5.73 TESS: Teaching Engineering Self-Efficacy Scale (possible range 1 – 6) ScaleGroup N Mean (SD)Range Teaching Scale Experimental (10.93)50 – 94 Comparison (9.50) Student Learning Scale Experimental (9.67) Comparison (8.73) PASTeL: Perceptual Assessment of Science Teaching and Learning (possible range 0 – 100) * There was not a statistically significant difference between the groups on either of these scales. * There was a statistically significant difference between the groups on all of these scales except for DS.

Educator RESULTS Overarching Themes Positives - Very strong communication from Ms. Kidd - Excellent training - Engagement and excitement by teachers and students - Hands-on learning and higher achievement - Teachers recognized that their instructional behaviors as changed based on what they did at the summer institute Concerns - Time (for professional development, to teach, to organize materials, for assessments) Suggestions - is the best form of communication, although a few indicated a hard copy news bulletin would be nice. Principals would appreciate a news bulletin (electronic or hard copy). - Ms. Kidd visiting schools in person once a month to answer questions - Consider a longer summer professional development - Consider doing the same unit for 2nd and 3rd grade and for 4th and 5th grade STEM Starters+ Year 1 Formative Evaluation Interviews with Teachers, Principals, and Central Office Administrators Conducted October, 2015

ChallengesLessons Learned Teacher administered assessments are not collected systematically or thoroughly Amount of data overwhelms other aspects of the project Modified research design to focus on open- ended science process instrument during years 3 and 4 of the project Lack of engineering instrumentation with adequate psychometric properties for elementary students and teachers Collaborated to review, adapt, and develop instrumentation School instability due to changes in district management Added extra district to account for possible attrition due to school instability Lack of communication across various district levels (classroom, office, school, and central office) Communicated project opportunities and successes early and often at district, building, and classroom levels. Perceived lack of instructional time to teach the engineering units in Grade 1 Drilled down to daily schedule to suggest “instructional slots” negotiated with district literacy directors Scheduling professional development institutes during a time teachers are available and agreeable Offered professional development multiple times during summer 2015 with make-up sessions available during the fall 2015