“EQAO has an accountability mandate to provide data that inform classroom teaching practices and contributes to improved student achievement in Ontario’s schools.” Marguerite Jackson EQAO CEO June 24, 2004

The Ontario Context for Assessment and Evaluation Session #8

World-Class Standards: The Programme for International Student Assessment (PISA) World-Class Standards: The Programme for International Student Assessment (PISA) -Goal: provide countries with benchmarks for their students’ performance -15-year old students in 32 countries are tested in reading, mathematics and science literacy -two-hour written test and a 30-minute background questionnaire (attitudes and approaches to learning) In 2003, schools across Canada were randomly selected to participate. Canadian students placed second in reading, third in mathematics and fifth in science.

National Assessment: School Achievement Indicators Program (SAIP) - National Assessment: School Achievement Indicators Program (SAIP) -Goal: Council of Ministers of Education assesses achievement for 13 and 16 year olds to determine whether students reach similar levels of performance at about the same age. -The content of the test changes focus annually. For example, in 1993 mathematics was assessed, then reading/writing in 1994 and science in Students are chosen randomly to participate. -There have been two cycles of testing and for both groups, scores have improved over initial measures.

Provincial testing has taken on a prominent role as schools are increasingly held accountable. The results of these evaluations are being used as a form of public accountability to identify whether learning objectives have been met and to make decisions about educational practices. In some provinces these tests are used as a graduation requirement.

Provincial Assessment: Ontario’s EQAO Provincial Assessment: Ontario’s EQAO -Ontario’s standardized testing program was introduced in Education Quality and Accountability Office (EQAO) is an independent arm’s-length government body that was created to design and evaluate annual provincial tests. -Goal: EQAO seeks to provide the public, parents and schools with information about student achievement in relation to the standards defined in the curriculum.

Grade 9 Assessment of Mathematics Currently, there are EQAO assessment programs in Grades 3, 6, 9 and 10… Grade 9 Assessment of Mathematics -versions for Applied and Academic Mathematics students -administered twice annually -items require written responses and multiple-choice format -assesses knowledge and skills of: number sense and algebra, relationships, analytic geometry, measurement and geometry

Grade 10 Ontario Secondary School Literacy Test (OSSLT) Grade 10 Ontario Secondary School Literacy Test (OSSLT) -administered annually in the Fall -two components: Reading & Writing -Reading: 12 reading selections (information, graphic, narrative) with 3 question formats (written short answer, written evaluation,multiple-choice) -Writing: 4 writing tasks (summary, opinion, news report, information)

Grade 3 and 6 Assessments of Reading, Writing and Mathematics Grade 3 and 6 Assessments of Reading, Writing and Mathematics - The Grade 3 and 6 assessments measure student achievement against The Ontario Curriculum. Reading: assesses use of a variety of reading strategies and conventions, understanding concepts, making inferences and connecting ideas

Writing: assesses use of writing strategies and language conventions, understanding assigned tasks, organizing ideas and communicating with the reader Mathematics: assesses demonstration of knowledge and skills across the five strands of mathematics -both multiple choice items and constructed response -constructed response portion is generated from content and skill areas identified in the curriculum

-procedures to ensure consistency and validity, such as teachers are given a script and receive detailed guidelines in the Teacher’s Daily Plans -administration time limits -for the past 5 years, the tests have spanned the course of 5 days and consume 12 hours of time -for 2005, the test versions will be shorter (approximately 6 hours)

- EQAO works with educators from across the province in all aspects of its testing program and engages teachers to score the assessments. -“Using this tool to target areas and monitor the effectiveness of improvement strategies is a critical component of teaching and learning,” said Jackson.

Let’s look at some sample questions. What are some considerations with respect to the test items? -reading ability -different test item content

Some critics believe that the test items present a contrived situation because students are asked questions and to solve problems that they will never see again. There is no link to the real world. In addition students must work alone, without resources and under extreme time limits.

What are some considerations with respect to how the results are qualified? - broad achievement levels -criterion referencing vs. norm referencing

% at Provincial Standard ReadingWritingMath Grade 3 (May/04) 54%58%64% Grade 6 (May/04) 58%54%57% Grade 9 (Jan./04) 68% Academic 26% Applied Grade 10 (Oct./03) 82%85%

There is a gap between boys and girls especially in reading and writing (Grades 3 & 6). Parents receive an individual report showing their students’ achievement in relation to curriculum expectations and a summary of school, board and provincial test results. How should parents interpret these reports or the results that they hear in the media?

School staff and parents need to take into account the complexities of their school. Examining EQAO results along with all of the other information they have about student achievement, such as report cards, classroom assessments and board assessments.

What are some considerations with respect to how the results are interpreted? -base-lines -cross-sectional design -developmental patterns

The EQAO results provide a “snapshot” of how students are achieving at one point in time and do not fully represent the richness and depth of multifaceted schools and their students. In addition, contextual factors, such as attendance patterns, absentee rates, mobility rates and special program needs, can influence student achievement.

It is believed that this is an effective way to hold schools accountable for students’ learning and gain feedback about which areas of learning need greater attention. The Canadian Psychological Association notes that the media draws comparisons between schools based on the outcomes of mandated provincial tests. This places pressure on teachers, administrators and students.

What are some considerations with respect to how the results are presented to teachers? - wide testing intervals -support for students’ needs -authentic performances

Pressure may led some educators to teach to the test and spend time coaching their students. In this manner, teachers narrow their instruction to match the content of the tests and non-assessed areas of the curriculum may be ignored. Some teachers say that the provincial tests infringe on their ability to teach what they believe is best for their students.

Other teachers believe that it is beneficial to prepare students for provincial assessments with general test-taking protocols. What are some of the strategies for preparing students to take an EQAO assessment?