1. Building Effective Assessments

2. Agenda  Brief overview of Assess2Know content development  Assessment building pre-planning  Cognitive factors  Building the assessment

3. Building Effective Assessments Using

4. Formative Assessment Content Assess2Know  Reading grades 2 – 11  Mathematics grades 2 – 11  Science grades 3 – 11  Social Studies grades 9 – 11 NEW

5. Formative Assessment Content High Quality Content  Rigorous development process  Tight alignment with Michigan State Standards  Highly trained and experienced assessment development staff

6. Formative Assessment Content Rigorous Requirements  Clearly and simply written; no “trick questions”  Appropriate vocabulary for the grade level being tested  Functions independently; does not provide information that may be used to answer other items  Requires students to read the passage in order to answer the question  Does not use “all of the above” or “none of the above”  Presents answer options in a logical order

7. Formative Assessment Content Reading  Passages are commissioned works by well- known authors  Lexiles and Flesch-Kincaid  Genres  Bloom’s Taxonomy, Cognitive Difficulty Levels  Maps, Charts, Graphs, Images, and Primary Sources

8. Flesch-Kincaid Formula FKRS = (0.39 x ASL) + (11.8 x ASW) - 15.59 FKRS: Flesch-Kincaid Readability Score ASL: average sentence length in words or average number of words in sentence (number of words divided by the number of sentences) ASW :average number of syllables per word (the number of syllables divided by the number of words)

9. Assess2Know Lexiles Lexile measures are considered to be the most accurate measure of readability because they are the only formula which applies a psychometric formula to the assignation for establishing readability. Since 2005, Metametrics assigns each A2K passage a Certified Lexile Measure.

10. Assess2Know Reading Items To determine grade level placement of passages, Riverside uses readability guidelines to place passages at each grade level. In addition, passage structure, interest level, topic, and word count are among some of the things used to evaluate where a passage should be placed for grade level benchmarking. District level committees review passages and items and if they had felt a passage was not appropriate, Riverside replaces it and the items at the review. –Advantage to onsite committee reviews because educators in the field are affirming what is appropriate for each benchmark.

11. Assess2Know Readability Limits GradeFlesh-KincaidLexile minimummaximumminimummaximum 32.04.0300L850L 43.05.0500L950L 54.06.0650L1050L 65.07.0750L1075L 76.08.0850L1100L 87.09.0950L1150L 98.010.01000L1200L 109.011.01050L1300L 1110.012.01100L1300L 1211.013.01100L1300L

12. Formative Assessment Content  Mathematics, Science, and Social Studies  Real world context  Unique  Scenario Based Science Items  Bloom’s Taxonomy, Difficulty levels  Use of Tools  Maps, Charts, Graphs, Images, and Primary Sources

13. Research Efforts The Research and Measurement Services Department analyzes data from multiple users of the Assess2Know Benchmark item bank from across the country to monitor the performance of the items in the bank. Goal is to ensure that the item bank provides users with strong quality items with a range of difficulty. Results look at average ranges for statistics such as p-values, discrimination statistics, and point-biserials for each item. This information is used to plan for refresh and future development of our item banks.

14. High Quality Assessment

15. Why do we give assessments?

16. Importance of Designing High Quality Assessments Assessments measure what is being taught in the classroom Align to the district curriculum –Curriculum Maps –Scope and Sequence Results of the assessment are used to report on student achievement –Used to make decisions about student growth, interventions and program placement

17. Characteristics of High Quality Assessments Design curriculum based assessments that measure student achievement to the state standards Report assessment results at the individual student, class, school and district level Provide achievement information that can be compared from year to year Construct assessments that are valid and reliable

18. Characteristics of High Quality Assessments Assessment design allows for a variety of assessment types: selected response, open ended items, constructed response and rubrics Delivered in a timely manner Easy to administer

19. Purpose of Assessment Evaluate students overall achievement and growth in a content domain Diagnose students strengths and weaknesses in and across content domain Plan educational interventions and to design individual instruction

20. Purpose of Assessment Place students in appropriate educational program Select applicants into programs with limited enrollment Certify individual achievement or qualifications from Standards for Educational and Psychological Testing

21. What Do We Ask Ourselves Before Building an Assessment? What do we want to measure? What information do we want to gather? How long do we want the assessment to take? How will we determine which standards we want to assess? How will we use the results?

22. Guiding Principles to Keep in Mind When Building Formative Assessments A minimum of three items for every standard you want to measure. Spread items measuring an individual standard across the assessment. Range of overall assessment difficulty and for each standard measured. Begin assessment with easier items, build to more difficult and conclude with easier items.

23. Guiding Principles to Keep in Mind When Building Formative Assessments How many passages will be used for reading assessments? What item types will be used? How many items per passage? Total number of items? Can this be completed in the allotted time?

24. Guiding Principles to Keep in Mind When Building Formative Assessments Spread items with graphics across test. Readability levels for reading passages; genres needed. A minimum of five items per passage. Even spread of answer choices across assessment when possible; spread them out across the form. Check items for appropriateness to use online, paper, or both. Check for clueing

25. Formative Assessment Content Level 1Basic Skills  recall of information such as fact, definition, term, or simple one-step procedure. Level 2Conceptual Understanding  includes the engagement of some cognitive processing beyond recalling or reproducing a response. A conceptual understanding item requires students to make some decisions as to how to approach the problem or activity and may imply more than a single step. Level 3Extended Reasoning  requires problem solving, planning, and/or using evidence. Items require students to develop a strategy to connect and relate ideas in order to solve the problem while using multiple steps and drawing upon a variety of skills. Cognitive Difficulty Levels

26. Importance of Content Complexity Vastness of Content Alignment Validity Clarity Teacher Guidance Truth in Advertising

27. Level One: Basic Skills Recall of information such as fact, definition, term, or simple one-step procedure. Support ideas by reference to details in text Use dictionary to find meaning Identify figurative language in passage Solve a one step word problem Perform a specified procedure

28. Level 2: Conceptual Understanding Includes the engagement of some cognitive processing beyond recalling or reproducing a response. A conceptual understanding item requires students to make some decisions as to how to approach the problem or activity and may imply more than a single step. Predict logical outcome Identify and summarize main points Represent a situation mathematically in more than one way. Interpret a visual representation

29. Level 3: Extended Reasoning Requires problem solving, planning, and/or using evidence. Items require students to develop a strategy to connect and relate ideas in order to solve the problem while using multiple steps and drawing upon a variety of skills. Determine effect of author’s purpose on text elements Summarize information from multiple sources Provide a mathematical justification Describe, compare and contrast solution methods

30. Building Effective Assessments Questions