Collaborative Evaluation Communities in Urban Schools

CEC Collaborators University of Kansas – School of Education Douglas Huffman, Ph.D. – Science Education Kelli Thomas, Ph.D. – Mathematics Education Danielle Barker – KU doctoral student (Chemistry) Carrie Hohl – KU doctoral student (Engineering) Anita Lundy - KU doctoral student & KCK Teacher (Science Education) Kansas City Kansas Public Schools KCK science and mathematics coordinators Bethel Elementary School – principal, IC, teachers Emerson Elementary School – principal, IC, teachers

Primary Goals 1. Improve the evaluation capacity of urban schools by creating school-based collaborative evaluation communities. 2. Develop graduate level educational leaders with the knowledge and skills to evaluate science and mathematics programs. 3.Provide teachers of science and mathematics professional growth by improving their capacity to use evaluation data and engage in continuous improvement and data based decision-making. 4.Develop new models of evaluation capacity building for K-8 schools and graduate students.

Purposes of CEC  To create collaborative teams to examine teaching and learning of math and science.  To use evaluation and assessment data to inform math and science instruction.  To engage in collaborative “inquiry” as a means of improving student achievement in math and/or science.  Help doctoral students in math and science education engage in evaluation & assessment

Background  Collaborative efforts that engage participants in a shared evaluation experience of data gathering and analysis can lead to changes in evaluation capacity of individuals and organizations, science and mathematics teaching and learning. Sarason (1996) urges districts and universities to create opportunities for collaboration that move across current hierarchies. Cochran-Smith and Lytle (1999) contend, “ Knowledge emerges from the conjoined understandings of teachers and others committed to long-term observation and documentation of learners and their sense making ” (p. 275). Huffman and Kalnin (2003) found that when science and math teachers engaged in collaborative inquiry, they not only changed their instruction, but also began to make school-wide changes. If teachers don ’ t move beyond action research and engage in school-wide changes they are “ just playing in the sandbox ” (Elmore)

School Evaluation Capacity Building Explore District, State and National Data Consider Implications for District Find Focus for Evaluation Collect DataAnalyze Data Develop Plan of Action Monitor Results

Graduate Student Evaluation Capacity Building University of Kansas University of Minnesota Videoconference Coursework Collaborative Evaluation Communities Professional Conferences Coursework Collaborative Evaluation Communities Videoconference Eval Institute Year 1 Explore Evaluation Year 2 Immersion in Evaluation Professional Conferences

The Bethel Elementary Case Finding a Focus for Evaluation 1. Curriculum alignment with state science standards. 2. Pacing of instruction. 3. Instructional practices of teachers as they use the curriculum. 4. Teachers’ science knowledge of content in the curriculum. 5. Alignment with the state science assessment.

Data Collection Bethel Team with our assistance created a survey to gather initial data related to: the science curriculum teacher efficacy instructional practices assessment practices student learning outcomes teacher confidence with their own science knowledge and understanding The Bethel Elementary Case

Data Analysis 1. Teachers were not comfortable with using the FOSS kits. 2. Over 90% of the teachers responded that they struggle to teach science as well as they teach other subjects. 3. Students rarely or never had the opportunity to either design their own science investigation or use science concepts to interpret and solve applied problems. 4. Teachers rarely used assessment to inform instruction in science. The Bethel Elementary Case

Plan of Action Inventory FOSS kits and target specific grade levels to use particular kits Engage students in inquiry using the FOSS kits as a framework for the learning experiences. Write multiple choice mini-assessments for teachers to use as embedded assessment throughout the school year while using FOSS kits. The intent for the mini-assessments is to provide teachers with a means for gathering regular student assessment data aligned with the curriculum, the state science standards, and the state science test. The Bethel Elementary Case

Monitor Results  Sharing of instructional issues encouraged collaborative reflection on changes that could be made in the implementation of the FOSS kits  Analysis of mini-assessments highlighted the need for a deliberate approach to the sequencing of FOSS kits from one grade level to the next  Future work…..developing a sequence for FOSS kits at each grade level that leads to greater depth of knowledge and understanding, student achievement, others?

The Emerson Elementary Case Finding a Focus for Evaluation 1. curriculum alignment with state mathematics standards, 2. pacing of instruction, 3. instructional practices of teachers as they use the curriculum, 4. teachers’ mathematical knowledge of content contained in the curriculum, 5. curriculum alignment with the state mathematics assessment.

The Emerson Elementary Case Data Collection  Emerson Team with our assistance created a survey to gather initial data related to: the mathematics curriculum, teacher efficacy, instructional practices, assessment practices, student learning outcomes, teacher confidence with their own mathematics knowledge and understanding

The Emerson Elementary Case Data Analysis 1. Teacher perceptions about Investigations in Number, Data, and Space (TERC, 1998) varied widely among teachers in the school. 2. Teachers agreed that they had the ability to effectively teach mathematics. However, few teachers felt strongly about this ability. 3. Responses to several questions about student learning, teacher assessment of student learning, and instructional practices lead to an emphasis on the issue of assessment.

The Emerson Elementary Case Plan of Action  Write multiple choice mini-assessments for teachers to use as embedded assessment throughout the school year while using Investigations in Number, Data, and Space  The intent for the mini-assessments is to provide teachers with a means for gathering regular student assessment data aligned with the curriculum, the state mathematics standards, and the state mathematics test.

The Emerson Elementary Case Monitor Results  Team analysis of individual student assessments highlighted areas to target for improvement  Students need experiences working with money and computations involving money  Integrate mathematics with the positive behavior support program  Future work…..student achievement, correlation between district selected norm referenced test and state assessment, including all teachers in data analysis, others?

Future…  The CEC partnerships continue to meet on a regular basis to discuss and debate issues related to mathematics and science teaching and learning.  We anticipate repeating the CEC inquiry process in each current school and with new schools that join the project.  Through this work, we hope to build a culture for sustained evaluation and assessment of educational programs in the schools.

Discussion…  What data would you suggest we collect to study the impact of CEC?  How best to collect?  What are the most important impact questions to ask?  What are your ideas and suggestions?