Scale Scoring A New Format for Provincial Assessment Reports

Agenda 9: :15Introduction to Scaled Scores 10:15 -10:30Break 10:30 -12:00Scaled Scores continued LSP website Using the Results 12:00 -1:00Lunch 1:00 - 3:00LSP writing

Beginning in the school year, all provincial assessments will be reported on a common scale.

Why Use a Common Scale? Imagine trying to accurately compare a student’s or a school’s achievement when given the following raw scores:  letter grades  numerical marks  rubric levels  percentage grades

What is a Scale? A scale is an arbitrarily established set of numbers used for measurement. Different scales may be used to measure the same thing, but they are calibrated differently. (e.g. rulers, thermometers, speedometers, bathroom scales)

What is a Scale? A scale, simply put, is a mathematical conversion of raw scores to a common scale or a derived score.

A scaled score is a conversion of a student's raw score on a test to a common scale that allows for a numerical comparison between students. Provincial assessments use multiple versions of a test over the years. The scale is used to control slight variations from one version of a test to the next. Scaled scores, when equated, are particularly useful for comparing test scores over time.

Equating: A Statistical Process Equating measures the difficulty of each assessment (content, cognition, difficulty) and adjusts the cut score to account for differences among assessment forms. The difficulty level of tests in different years can be equated and this enables us to compare assessment results from year to year.

Advantages of Scale Scores Scaled scores allow better comparison of assessment results: from year to year from one grade level assessment to another of student achievement in different subject areas

Future Advantages Because each successive year’s assessments are statistically equated with a baseline reference year, the scaled scores can also be used to reflect growth. For example, if a future scaled score is higher than the reference year scale score ( ), it can be interpreted as an improvement from that base year.

Cautions for Comparisons Many factors impact upon a school’s results. No single assessment result tells the whole story of a student’s or school’s achievement. Sample size does matter! Schools with fewer students will show more variability in scores from one year to the next.

Scale Scoring % of students who wrote this assessment Scale Scores Provincial Mean One standard deviation above Cut score One standard deviation below

Things to Consider Distribution of Scores: -Scale ranges from 200 to 800 -Provincial mean is set to 500 -Provincial standard deviation is set to 100 -On most provincial assessments, about 65% of students will place between 400 and 600

Cut Score A cut score defines the point at which a certain required level of achievement has been demonstrated on the assessment. Cut scores are set by a panel of educators who look at the actual test questions to determine levels of acceptable performance. A cut score can be above or below the mean.

Cut Score A cut score differentiates between students who are meeting expectations or not yet meeting expectations.

Things to Consider The cut score for the ELA was set at 384 Higher scores mean stronger performance relative to other students who wrote the same assessment.

Interpreting Results Overall Score Cut Score: How close or how far away from the cut score (384) did the student perform? Did the student meet expectations for this assessment or has the student not yet met expectations for this assessment? REMEMBER: The cut score only applies to the overall score (section 1) the student received.

Using the Results Overall Score Students Meeting Expectations (385 and above) If the overall score is above the cut score (384) the student is meeting expectations. Benchmarks have been established to help determine the level of performance. If a student’s overall score falls between , his/her performance is considered to be satisfactory. If a student’s overall score falls between , his/her performance is considered to be stronger.

Using the Results Overall Score Students Not Yet Meeting Expectations (Overall Score below 384) For every student who is not yet meeting expectations (overall score is below 384) teachers will be expected to: provide on-going, in-class support contact parents/guardians to discuss support options

Using the Results Overall Score Students Not Yet Meeting Expectations (Overall Score below 384) If a student’s overall score falls below 384, his/her score is more than one standard deviation from the mean. Therefore, he/she will require the most support and formal tracking of that support. For every student who is not yet meeting expectations and whose score is below 384, the teacher is expected to: Develop a Literacy Support (LS) document Support student within the class structure The LS document will be on-going for students, with transition meetings at the end of each grade.

Interpreting Results Overall Score Provincial Mean (500) How close or how far away from the mean did the student perform? How did the student perform relative to others that wrote the assessment? How does the student’s performance relative to others compare from subject to subject? How did the student perform this time compared to the time before?

Using the Results The Reading Performance section: Provides information on student performance in the various genres, cognitive levels and some specific outcomes. Allows for comparison to the provincial mean of 500. May help teachers isolate concerns and develop strategies to address these indicators. was not factored into a student’s overall score is intended to provide to teachers information for planning purposes only.

Using the Results The Writing Performance section: Provides information on student performance in the elements of writing: ideas, organization, matters of correctness May help teachers identify concerns and trends.

Using the Results Schools are responsible for tracking Literacy Support (LS) and for monitoring support. Teachers and Principals are expected to: review LS document and monitor support ensure parent/guardian signatures on the LS document ensure that the completed LS document is kept in the student’s cum file

Using Your Data: Classroom Level Teachers are encouraged to use the student reports to identify trends and patterns in literacy development. Class / grade profiles may be used for classroom instructional decision making and for targeting support to individual students.

Using the Results: School Level In April, The Minister of Education will release the provincial, board and school results in the Minister’s Report to Parents.

Using the Results: School Level

Next Steps: Principals have been asked to verify with the DOE that the information sent to them is accurate. NOTE: Assessment information is now sent to principals electronically via the Student Records website. The blue forms will no longer be sent to schools. Print Report for School Administration and Teachers. Put results for students who do not meet expectations in the blue pocket folders and place them in the cum file.

Parent Reports will be sent to schools once verification with the DOE is complete. Please do not send home the school reports. Gather classroom based assessment information to support LSP writing. Commence LSP writing using the electronic template. IMPLEMENTATION DEADLINE: APRIL 30, 2008

In May, there will be a random Audit of LSPs at the grade 6 level. Late May / June: complete the Student Progress and Transition sections of the LSP and meet with Junior High teachers. Print a hard copy of the updated LSP and place it in the blue pocket folder in the cum file. Forward the electronic templates to the Junior High school.

Let’s walk through the set up of a Literacy Support Plan

Program of Learning Assessment for Nova Scotia (PLANS), Nova Scotia Department of Education, Provincial Assessment Reports: A New Format (2007) For more information