The New MCAIII Science Benchmark Reports for 2015 Minnesota Department of Education Science Assessment Specialist Jim WoodDawn Cameron “Leading for educational excellence and equity. Every day for every one.”

What we want from MCA’s “ How are students doing on specific benchmarks?” education.state.mn.us 2

What we get from the MCA’s “ Strand Level Data” education.state.mn.us 3

Effect size is a statistical method of summarizing group differences CLES in these graphs expresses the probability that a student selected at random from a school will receive a higher score on the item relative to the overall performance expected for that school For this comparison, the overall performance expectation for the school is placed at 0.50 on the CLES scale Common Language Effect Size (CLES) education.state.mn.us 4

Benchmark Reports education.state.mn.us 5

Focuses on variation within a school The dashed vertical line fixed on the 0.50 CLES represents the overall performance expectation for the school Solid gray line represents the performance expected from students at the state mean in overall science ability Benchmark information should only be compared to the school expectation line fixed at 0.50 on the CLES axis What’s Different From Past Years education.state.mn.us 6

Initial impression Identify strengths and weaknesses on report Questions Your Schools Data education.state.mn.us 7

Interpreting the Benchmark Reports education.state.mn.us 8 It is important to frame any interpretation in the context of the school’s environment. Experience with the science curriculum, instruction, and data from other classroom assessments is critical to making meaningful inferences from this report.

All data is relative to the expected school average Location of items on the scale do not provide information about the difficulty of the individual test questions There may be more than one item assessing a particular benchmark The number of items on each report corresponds to the number of items on the assessment for each grade Color codes and position of items in the graphs do not correspond to achievement levels education.state.mn.us 9 CAUTION!!! What we can say and not say about the data

Watch the dotted lines – different colors may overlap. In the cases where there is more than 1/2 overlap, the items may be considered statistically equivalent Keep an eye on the CLES metric when comparing schools. The horizontal axis is adjusted to fit each individual school’s data. New format makes does not allow for easy comparison of schools, but deeper understanding within a school Things to watch out for education.state.mn.us 10

Digging into the Reports Test Specifications  Test Design  Content Specifications Look for patterns in data  Do certain standards show a pattern within the year and over time? Start asking questions  Classroom assessments Does the data match what is happening in the classroom education.state.mn.us 11

Consistent with other content areas Allows staff to quickly identify strengths and weaknesses within a school Allows for more flexibility in test design Why The Change? education.state.mn.us 12

Old Format Benchmark Reports Transition year- –Reports available upon request from MDE –Careful with interpretation between reports education.state.mn.us 13

Test Development Process Test Specs Storyboard Writing Item Review Field Test Items Data Review Operational Item Pool Operational Test Bias Review Achievement Levels Storyboard Review Bias Review Item Writing education.state.mn.us 14

Get involved! Assessment Advisory Panels—ongoing Register to be invited to panels at  Educator Excellence  Testing Resources  Register for Advisory panels education.state.mn.us 15

education.state.mn.us 16