1. Tim Brower Professor & Chair Manufacturing & Mechanical Engr. Oregon Institute of Technology MSP Regional Meeting, San Francisco, February 14 & 15, 2008 Edith Gummer Director Classroom-Focused Research & Eval. Northwest Regional Educational Lab

2. 2 Partners Include.... 2 Oregon Institute of Technology Northwest Regional Educational Laboratory

3. 3 Project Goals The vision for this partnership is to introduce new contextualized learning curriculum called Project Lead The Way (PLTW) into school districts within Oregon and to engage the local community colleges, skill centers, and industry to leverage resources to gain capacity for the program. Goal 1: Increase teachers’ content knowledge and teaching skills. Goal 2: Increase the academic performance of students in math, science, and technology. Goal 3: Establish the infrastructure necessary to implement and sustain the program in Oregon schools.

4. Evaluation Model Evolving process Lessons learned and capacity built Goals and objectives oriented Teacher and student change Program capacity as intervention Mixed methodology Single group – no control or comparison Formative and summative processes implemented Evolving process Lessons learned and capacity built Goals and objectives oriented Teacher and student change Program capacity as intervention Mixed methodology Single group – no control or comparison Formative and summative processes implemented 4

5. Goal 1: Teacher Content Knowledge PLTW assessments developed nationally Same as assessments developed for students No ceiling effect in pre-test experienced Administered as pre/post-test Significant differences PLTW assessments developed nationally Same as assessments developed for students No ceiling effect in pre-test experienced Administered as pre/post-test Significant differences 5

6. Year 2 Teacher Data PLTW COURSE # Teachers % Gain Principles of Engineering 428% Digital Electronics529% Computer Integrated Manufacturing 530% 6 Limitations: Low number of teachers tested Lack of comparison group No delayed post-test

7. Goal 1: Teacher Instructional Practices Use of Reformed Teaching Observation Protocol Overall RTOP scores greater than those reported in other Title IIB MSP projects evaluated by NWREL Limitations Variable number of observations per teacher Range in fidelity of implementation Comparison group issues Use of Reformed Teaching Observation Protocol Overall RTOP scores greater than those reported in other Title IIB MSP projects evaluated by NWREL Limitations Variable number of observations per teacher Range in fidelity of implementation Comparison group issues 7

8. Goal 1: Teacher Instructional Practices Survey of Enacted Curriculum – Part A, Instructional Practices and Beliefs Year 3 is most systematic use of SEC Limitations SEC designed for science and mathematics teachers Low number of teachers in project Comparison groups Survey of Enacted Curriculum – Part A, Instructional Practices and Beliefs Year 3 is most systematic use of SEC Limitations SEC designed for science and mathematics teachers Low number of teachers in project Comparison groups 8

9. Goal 1: Teacher Instructional Practices Documentation of what teachers are doing in a PLTW course Use of SCOOP notebook Captures 5 days of instruction Student assignments and assessments Teacher reflections Limitations Variability of implementation fidelity Unknown use of curricular materials Documentation of what teachers are doing in a PLTW course Use of SCOOP notebook Captures 5 days of instruction Student assignments and assessments Teacher reflections Limitations Variability of implementation fidelity Unknown use of curricular materials 9

10. Goal 2: Increase in Student Achievement Use of PLTW national tests Year 3 focus of the evaluation Issue will be the number of teachers who are implementing full PLTW courses Connection to national PLTW evaluation Supplemental contract with True Outcomes Timeframe and access has been problematic Lack of uniform model of measuring PLTW across the states Use of PLTW national tests Year 3 focus of the evaluation Issue will be the number of teachers who are implementing full PLTW courses Connection to national PLTW evaluation Supplemental contract with True Outcomes Timeframe and access has been problematic Lack of uniform model of measuring PLTW across the states 10

11. Goal 3: Establish Infrastructure OIT PLTW project is not just increasing teacher content knowledge Effort is bringing new curricular program into the state Infrastructure issues Coordination with other engineering initiatives Bringing school districts into process Building master teachers to decrease reliance on national program OIT PLTW project is not just increasing teacher content knowledge Effort is bringing new curricular program into the state Infrastructure issues Coordination with other engineering initiatives Bringing school districts into process Building master teachers to decrease reliance on national program 11

12. Goal 3: Establish Infrastructure 12 To date: 4 “Science” Teachers 8 “Math” Teachers To date: 4 “Science” Teachers 8 “Math” Teachers

13. Goal 3: Establish Infrastructure Two “Master Teachers” are slated to be trained during the 2008 Summer training Institute in Oregon High school teachers that have taught the curriculum at least 2-years National screening process Trained with fully certified Master Teacher Prep training held once per year 13

14. Supplemental Goal: Evidence of Partnership Interview protocol developed based on work of Gordon Kingsley Interviews of key stakeholders conducted spring 2007 Partnership profile Interview protocol developed based on work of Gordon Kingsley Interviews of key stakeholders conducted spring 2007 Partnership profile 14

15. Evaluation Challenges Comparison groups of teachers and students Appropriate instruments PLTW content courses? Instructional practices Adequate sample sizes of teachers and students Sufficient description of nature of intervention Comparison groups of teachers and students Appropriate instruments PLTW content courses? Instructional practices Adequate sample sizes of teachers and students Sufficient description of nature of intervention 15

16. 16 The instructional practices and assessments discussed or shown in this presentation are not intended as an endorsement by the U.S. Department of Education. Questions? Dr. Tim Brower Professor & Chair Manufacturing & Mechanical Engr. Oregon Institute of Technology timothy.brower@oit.edu Dr. Edith Gummer Program Director Classroom-Focused Research & Eval. Northwest Regional Educational Lab gummere@nwrel.org Contacts: