Building Effective Assessments

Agenda  Brief overview of Assess2Know content development  Assessment building pre-planning  Cognitive factors  Building the assessment

Building Effective Assessments Using

Formative Assessment Content Assess2Know  Reading grades 2 – 11  Mathematics grades 2 – 11  Science grades 3 – 11  Social Studies grades 9 – 11 NEW

Formative Assessment Content High Quality Content  Rigorous development process  Tight alignment with Michigan State Standards  Highly trained and experienced assessment development staff

Formative Assessment Content Rigorous Requirements  Clearly and simply written; no “trick questions”  Appropriate vocabulary for the grade level being tested  Functions independently; does not provide information that may be used to answer other items  Requires students to read the passage in order to answer the question  Does not use “all of the above” or “none of the above”  Presents answer options in a logical order

Formative Assessment Content Reading  Passages are commissioned works by well- known authors  Lexiles and Flesch-Kincaid  Genres  Bloom’s Taxonomy, Cognitive Difficulty Levels  Maps, Charts, Graphs, Images, and Primary Sources

Flesch-Kincaid Formula FKRS = (0.39 x ASL) + (11.8 x ASW) FKRS: Flesch-Kincaid Readability Score ASL: average sentence length in words or average number of words in sentence (number of words divided by the number of sentences) ASW :average number of syllables per word (the number of syllables divided by the number of words)

Assess2Know Lexiles Lexile measures are considered to be the most accurate measure of readability because they are the only formula which applies a psychometric formula to the assignation for establishing readability. Since 2005, Metametrics assigns each A2K passage a Certified Lexile Measure.

Assess2Know Reading Items To determine grade level placement of passages, Riverside uses readability guidelines to place passages at each grade level. In addition, passage structure, interest level, topic, and word count are among some of the things used to evaluate where a passage should be placed for grade level benchmarking. District level committees review passages and items and if they had felt a passage was not appropriate, Riverside replaces it and the items at the review. –Advantage to onsite committee reviews because educators in the field are affirming what is appropriate for each benchmark.

Assess2Know Readability Limits GradeFlesh-KincaidLexile minimummaximumminimummaximum L850L L950L L1050L L1075L L1100L L1150L L1200L L1300L L1300L L1300L

Formative Assessment Content  Mathematics, Science, and Social Studies  Real world context  Unique  Scenario Based Science Items  Bloom’s Taxonomy, Difficulty levels  Use of Tools  Maps, Charts, Graphs, Images, and Primary Sources

Research Efforts The Research and Measurement Services Department analyzes data from multiple users of the Assess2Know Benchmark item bank from across the country to monitor the performance of the items in the bank. Goal is to ensure that the item bank provides users with strong quality items with a range of difficulty. Results look at average ranges for statistics such as p-values, discrimination statistics, and point-biserials for each item. This information is used to plan for refresh and future development of our item banks.

High Quality Assessment

Why do we give assessments?

Importance of Designing High Quality Assessments Assessments measure what is being taught in the classroom Align to the district curriculum –Curriculum Maps –Scope and Sequence Results of the assessment are used to report on student achievement –Used to make decisions about student growth, interventions and program placement

Characteristics of High Quality Assessments Design curriculum based assessments that measure student achievement to the state standards Report assessment results at the individual student, class, school and district level Provide achievement information that can be compared from year to year Construct assessments that are valid and reliable

Characteristics of High Quality Assessments Assessment design allows for a variety of assessment types: selected response, open ended items, constructed response and rubrics Delivered in a timely manner Easy to administer

Purpose of Assessment Evaluate students overall achievement and growth in a content domain Diagnose students strengths and weaknesses in and across content domain Plan educational interventions and to design individual instruction

Purpose of Assessment Place students in appropriate educational program Select applicants into programs with limited enrollment Certify individual achievement or qualifications from Standards for Educational and Psychological Testing

What Do We Ask Ourselves Before Building an Assessment? What do we want to measure? What information do we want to gather? How long do we want the assessment to take? How will we determine which standards we want to assess? How will we use the results?

Guiding Principles to Keep in Mind When Building Formative Assessments A minimum of three items for every standard you want to measure. Spread items measuring an individual standard across the assessment. Range of overall assessment difficulty and for each standard measured. Begin assessment with easier items, build to more difficult and conclude with easier items.

Guiding Principles to Keep in Mind When Building Formative Assessments How many passages will be used for reading assessments? What item types will be used? How many items per passage? Total number of items? Can this be completed in the allotted time?

Guiding Principles to Keep in Mind When Building Formative Assessments Spread items with graphics across test. Readability levels for reading passages; genres needed. A minimum of five items per passage. Even spread of answer choices across assessment when possible; spread them out across the form. Check items for appropriateness to use online, paper, or both. Check for clueing

Formative Assessment Content Level 1Basic Skills  recall of information such as fact, definition, term, or simple one-step procedure. Level 2Conceptual Understanding  includes the engagement of some cognitive processing beyond recalling or reproducing a response. A conceptual understanding item requires students to make some decisions as to how to approach the problem or activity and may imply more than a single step. Level 3Extended Reasoning  requires problem solving, planning, and/or using evidence. Items require students to develop a strategy to connect and relate ideas in order to solve the problem while using multiple steps and drawing upon a variety of skills. Cognitive Difficulty Levels

Importance of Content Complexity Vastness of Content Alignment Validity Clarity Teacher Guidance Truth in Advertising

Level One: Basic Skills Recall of information such as fact, definition, term, or simple one-step procedure. Support ideas by reference to details in text Use dictionary to find meaning Identify figurative language in passage Solve a one step word problem Perform a specified procedure

Level 2: Conceptual Understanding Includes the engagement of some cognitive processing beyond recalling or reproducing a response. A conceptual understanding item requires students to make some decisions as to how to approach the problem or activity and may imply more than a single step. Predict logical outcome Identify and summarize main points Represent a situation mathematically in more than one way. Interpret a visual representation

Level 3: Extended Reasoning Requires problem solving, planning, and/or using evidence. Items require students to develop a strategy to connect and relate ideas in order to solve the problem while using multiple steps and drawing upon a variety of skills. Determine effect of author’s purpose on text elements Summarize information from multiple sources Provide a mathematical justification Describe, compare and contrast solution methods

Building Effective Assessments Questions