Presentation on theme: "Student tests are hand scored by classroom teachers trained and calibrated using standard protocols. Students in grades 2 through 10 th /11 th grade are."— Presentation transcript:
Student tests are hand scored by classroom teachers trained and calibrated using standard protocols. Students in grades 2 through 10 th /11 th grade are administered performance exams (5 apprentice tasks per exam). District scoring leaders are trained in using task specific rubrics Student results are collected, analyzed, and reported by an independent data contractor. Random sample of student papers are audited and rescored by SJSU math & CS students. (Two reader correlation >0.95) Performance Exams 40,000 – 70,000 students per year since 1999 9
10 MARS Scoring The MARS tasks use a point-scoring rubric. Each task is assigned a point total corresponding to the complexity of the task and the proportional amount of time that the average student would spend on the task in relation to the entire exam. The task points are then allocated among its parts. – Some points are assigned to how the students approach the problem, – Majority of points to the core of the performance – A few points to evidence that, beyond finding a correct solution, students demonstrate the ability to justify or generalize their solutions. – In practice, usually points are assigned to different sections of a multi-part question.
11 Use of MARS Reports The combination of PA tasks and weighted rubrics provides a detailed picture of student performance – how students approached the different tasks – a description of common misconceptions – evidence of what students understand. Reports – include student work samples at each grade showing the range of students approaches, successes, challenges. – provide implications for instruction: specific suggestions and ideas for teachers as a result of examining students strengths and the areas where more learning experiences are required.
Longer MAC experience Higher Scores Grade/ Course Percentage Proficient Students with Non-MAC Teachers Percentage Proficient Students with MAC Teachers 64264 72959 81525 Algebra I5270 12 Large Scale Assessment
Percent Proficient on MARS and LSA 13 2005200620072008200920102011 Grade 2 65.667.763.664.872.172.5 Grade 357.063.662.857.366.161.366.3 Grade 4 57.956.058.467.457.665.8 Grade 549.051.618.104.22.1687.964.4 Grade 6 52.549.047.646.736.339.6 Grade 742.032.642.743.933.748.623.5 Grade 8 25.518.112.16.025.517.0 Algebra 1 35.236.737.935.643.242.0
Spring 2011 Trends Grade to Grade Grade 2MARS BelowMARS At or ^Total CST Below13.7%6.5%20.2% CST AT or ^7.3%72.5%79.8% Totals21.0%79.0%100.0% Grade 2MARS 1MARS 2MARS 3MARS 4Total Far Below1.0%0.6%0.1%0.0%1.7% Below Basic1.9%4.1%1.1%0.1%7.2% Basic0.8%5.3%4.6%0.6%11.3% Proficient0.4%5.1%16.2%6.5%28.2% Advanced0.2%1.6%15.2%34.6%51.6% Total4.3%16.7%37.2%41.8%100.0% 14
Spring 2011 Trends Grade to Grade Grade 6MARS BelowMARS At or ^Total CST Below34.7%3.8%38.5% CST AT or ^21.7%39.6%61.3% Totals56.4%43.4%99.8% Grade 5MARS BelowMARS At or ^Total CST Below17.3%6.0%29.7% CST AT or ^12.4%64.4%70.4% Totals23.3%76.8%100.1% Grade 4MARS BelowMARS At or ^Total CST Below15.6%5.8%21.4% CST AT or ^12.9%65.8%78.7% Totals28.5%71.6%100.1% Grade 3MARS BelowMARS At or ^Total CST Below16.4%4.5%20.9% CST AT or ^12.7%66.3%79.0% Totals29.1%70.8%99.9% 15
Spring 2011 Trends Grade to Grade Course 2MARS BelowMARS At or ^Total CST Below15.4%0.0%15.4% CST AT or ^36.0%48.7%84.7% Totals51.4%48.7%100.1% Course 1MARS BelowMARS At or ^Total CST Below31.9%4.1%36.0% CST AT or ^21.5%42.0%63.5% Totals53.4%46.1%99.5% Grade 8MARS BelowMARS At or ^Total CST Below55.1%2.8%57.9% CST AT or ^25.0%17.0%42.0% Totals80.1%19.8%99.9% Grade 7MARS BelowMARS At or ^Total CST Below38.1%0.4%38.5% CST AT or ^38.1%23.5%61.6% Totals76.2%23.9%100.1% 16
8 th Grade Geometry Californias Highest Achieving Students GeometryMARS Below MARS At or AboveTotal CST Below15.3%0.0%15.3% CST AT or Above36.0%48.7%84.7% Totals51.3%48.7%100% 17
18 Summary of Results These findings represent what has been consistently found across the years, even with an increasing number of students and teachers. – In 1999, 21 school districts, 462 teachers, and 23,128 students were involved in the study. – The project reached the largest number of students in 2004 with 81,075 students and 1622 teachers in 28 districts. – When funding for the project needed to be supported solely by district funds, the number of participating districts, teachers, and schools decreased. In 2011, 28 districts participated with 38,538 students.
19 Implications MARS tasks represent the assessment consortias descriptions of must-needed performance tasks Better understanding the data from these assessments and their relationship to improved teaching practices and student learning is paramount for informing the pragmatic decisions that will be made around what types of mathematics tasks assess deeper student learning and reflect improved teaching practices. It is NOT just the assessment – its the use of the information via support and coaching of teachers.
20 Limitations and Next Steps This paper is the first in a series studying the usefulness of MARS resources for teachers and their students, and thus provides only a sneak peek at the many layers of data to be studied. This work used simple exploratory analyses and comparisons, and does not utilize the methods for studying longitudinal nested data, which will provide a much clearer view of longer-term impact, following students and teachers over the years, controlling for previous performance and other trends that cannot be seen by simple year-to-year splicing of data.
21 Intensive Vs. General Coaching Score Code Intensive Formative Assessment General Coaching Grade 6 MARS 6945 Grade 7 MARS 4528 Grade 8 MARS 3820 Grade 6 statewide assessment 6550 Grade 7 statewide assessment 5948 Grade 8 statewide assessment 4835
22 Summary of Results A focused intervention around formative assessment using MARS tasks and data shows that students with such teachers have even further gains on both performance and multiple-choice assessments. This finding is even true with the ever-increasing number of students and teachers involved every year, indicating that the process is scalable and sustainable.