Project Director – Dr. Mark Lung Dept of Natural & Environmental Sciences Western State College of Colorado Project Evaluator – Dr. Dave Shannon Educational Research Methodology Auburn University Partners Western State College of Colorado (IHE) Hindsdale SD Gunnison Watershed SD Ouray SD Moffat County SD

Project Description  Professional development program for middle school math and science teachers in rural Colorado schools.  Includes: Authentic experience with the process of science and its tools (including math) Authentic experience with the process of science and its tools (including math) Training and equipment for implementing project-based (i.e. inquiry) lessons in the classroom. Training and equipment for implementing project-based (i.e. inquiry) lessons in the classroom.

1. Field research experience (2 weeks) - Data collection/analysis - GPS/GIS training - Creating inquiry lessons - Teachers leave with experience, photos, data to build lessons Project Activities 2. Implementation Workshop (2 days) - Skill/content reinforcement - Skill/content reinforcement - Implementing lessons in classrooms - Implementing lessons in classrooms 3. Job-embedded coaching - Observation of inquiry lessons - Observation of inquiry lessons - Immediate discussion/feedback - Immediate discussion/feedback - Repeat observations and feedback - Repeat observations and feedback

Project Outcomes  1. Increase teacher knowledge, experience, and confidence with science concepts and applications (e.g. hypotheses, variables, designing studies, scientific interpretation, etc.)  2. Increase the use of inquiry- based instruction in classrooms  3. Increase in student attitude towards and achievement in science and math

Year 1 Year 2 Year 3 Activity/Evaluation Timeline C1 C2 Field experience Implementation workshop Job-embedded coaching Activities Evaluation Outcome 2 Outcome 1 June 06 June 07 June 08

Evaluation Design  Design - Cross-sectional and longitudinal  Sample – All participating teachers and students. Comparison groups of students sampled when possible.  Data Collection – surveys, content knowledge assessments, class observations, lesson plan analysis

Evaluation  Outcome 1: Increase teacher knowledge, confidence and experience in science process Instruments – Instruments – PPI – s elf-perceptions of knowledge, confidence, and experience. (administered as pre- and post for both cohorts of teachers)PPI – s elf-perceptions of knowledge, confidence, and experience. (administered as pre- and post for both cohorts of teachers) Science Teacher Survey – attitudes toward teaching and learning science, motivating students, teacher efficacy, and the use of constructivist and inquiry-based teaching methods. (administered annually to all teachers)Science Teacher Survey – attitudes toward teaching and learning science, motivating students, teacher efficacy, and the use of constructivist and inquiry-based teaching methods. (administered annually to all teachers) Content knowledge assessment– tailored to the project’s objectives regarding science concepts and applications. (administered as pre in June 2007, post planned for later this year)`Content knowledge assessment– tailored to the project’s objectives regarding science concepts and applications. (administered as pre in June 2007, post planned for later this year)`

Evaluation  Outcome 2: Increase use of inquiry-based instruction in the classroom Instruments – Instruments – *Science Teacher Inquiry Rubric (STIR) - has been used as an observation tool as well as a measure of self-reflection by teachers. (administered in fall and spring)*Science Teacher Inquiry Rubric (STIR) - has been used as an observation tool as well as a measure of self-reflection by teachers. (administered in fall and spring) Teacher and Student SurveysTeacher and Student Surveys *Beerer and Bodzin 2003, 2004

Evaluation  Outcome 3: Increase student attitude and achievement Instruments – Instruments – Attitudes - Survey of Science Classrooms – student attitudes toward science, academic efficacy, goal orientation and motivation, and inquiry-based teaching practices in the classroom. (administered in February 2007)Attitudes - Survey of Science Classrooms – student attitudes toward science, academic efficacy, goal orientation and motivation, and inquiry-based teaching practices in the classroom. (administered in February 2007) Knowledge – The examination of state-mandated achievement tests (CSAP) and the MOSART tests are planned for the upcoming year.Knowledge – The examination of state-mandated achievement tests (CSAP) and the MOSART tests are planned for the upcoming year.

Results Outcome 1: Teacher Knowledge, Experience, Confidence  PPI – Reliabilities estimates exceed.97 Reliabilities estimates exceed.97  Cohort 1 teachers (n=7) reported higher post perceptions with increases of approximately 1 point (on a 5 point scale) occurred on each of the three PPI scales, with slightly larger gains for confidence.  Cohort 1 (experienced) teachers also reported more positive perceptions that the new teachers from Cohort 2.

Outcome 1 (continued)  Science Teacher Survey Teacher Beliefs and Attitudes While all teachers generally agreed that they can overcome student difficulties and attribute student success to their efforts, these beliefs were more positive for teachers from cohort 1 (2006) While all teachers generally agreed that they can overcome student difficulties and attribute student success to their efforts, these beliefs were more positive for teachers from cohort 1 (2006) Teachers from cohort 2 (2007) were more skeptical about student learning potential Teachers from cohort 2 (2007) were more skeptical about student learning potential

Outcome 1 (continued)   Teacher Content Knowledge In general, cohort 1 teachers self-reported having more knowledge on the PPI, especially in terms of: Describing inquiry-based learning Designing inquiry-based lessons and assessments Cohort 1 teachers also performed better than Cohort 2 teachers on the Content Knowledge Assessment in terms of: Describing the principles of inquiry-based learning Comparing research designs and designing studies to investigate the effects of inquiry

Results Outcome 2: use of inquiry  Science Teacher Survey Both cohorts of teachers cohorts generally believed in their ability to use constructivist, student-centered, inquiry-based approaches Both cohorts of teachers cohorts generally believed in their ability to use constructivist, student-centered, inquiry-based approaches Experienced teachers (cohort 1) were more likely to indicate that they made special efforts to recognize individual progress and they encouraged student collaboration. Experienced teachers (cohort 1) were more likely to indicate that they made special efforts to recognize individual progress and they encouraged student collaboration.  While the overall mean for the Inquiry-based teaching measurement scale was just at the midpoint, several items resulted in higher average responses by participating students. Specifically, students reported:

Outcome 2 (continued)  Survey of Science Classrooms Students taught by participating teachers reported that they: were involved in small group discussions to make sense of science, were involved in small group discussions to make sense of science, used math skills to help with science used math skills to help with science gave written explanations about how they solved science problems gave written explanations about how they solved science problems discussed alternative explanations regarding science questions and problems discussed alternative explanations regarding science questions and problems did science projects that lasted several days did science projects that lasted several days

Outcome 2 (continued)  STIR 9 classrooms observed with inquiry-based lesson plans 9 classrooms observed with inquiry-based lesson plans Rubric Rubric 6 items relating to different components of inquiry6 items relating to different components of inquiry scoring: 1 (low) – 5 (high)scoring: 1 (low) – 5 (high) ResultsMean (+SD) ResultsMean (+SD) Cohort 1: 3.1 (+1.4)Cohort 1: 3.1 (+1.4) Cohort 2: 3.6 (+0.8)Cohort 2: 3.6 (+0.8) Considerable variation among teachersConsiderable variation among teachers Low scores primarily due to leading students to content Low scores primarily due to leading students to content

Results Outcome 3: Student Attitude/Achievement  Student Attitudes Eleven measurement scales yielded reliabilities ranging from.63 to.86 (Median =.82) Eleven measurement scales yielded reliabilities ranging from.63 to.86 (Median =.82) Students averaged above the mid-point (3) on 8 of the 11 scales Students averaged above the mid-point (3) on 8 of the 11 scales Students: Students: Have a positive attitude toward science and that science is useful Have a positive attitude toward science and that science is useful Exhibit a Mastery Goal Orientation (task-oriented) toward learning science as opposed Performance Approach (competition) or Avoidance (embarrassment) Exhibit a Mastery Goal Orientation (task-oriented) toward learning science as opposed Performance Approach (competition) or Avoidance (embarrassment) Believe in their ability to learn (Academic Efficacy) Believe in their ability to learn (Academic Efficacy) Perceive that their teachers believe in their ability to learn science Perceive that their teachers believe in their ability to learn science

Outcome 3 (continued)  Student Achievement The use of state-mandated tests (CSAP) is being explored, but the Science tests are only administered in grades 5 and 8 – providing limited usefulness for the project The use of state-mandated tests (CSAP) is being explored, but the Science tests are only administered in grades 5 and 8 – providing limited usefulness for the project The use of commercially-prepared tests (e.g., the MOSART tests) and teacher-made tests are being explored for potential us in the project. The use of commercially-prepared tests (e.g., the MOSART tests) and teacher-made tests are being explored for potential us in the project.

Attribute

Attribute

  Disclaimer   The instructional practices and assessments discussed or shown in these presentations are not intended as an endorsement by the U.S. Department of Education".