1. Test Tips and General Info. Information learned through the Ohio Assessment Item Development and Scoring Workshop

2. Test Developers and Graders OGT –developed by American Institute for Research (AIR) –scored by Data Recognition Corporation OAT – developed by Pearson –scored by Pearson

3. Process in which an OGT question goes through before we see it on the test Preliminary level- writers develop ?s Small group review Content review committee Content II committee Grade level lead person looks at ? Put in ODE pool (ODE then evaluates) Content Committee- Ohio teachers Fairness Committee- Ohio teachers Field Test Pool Field Test chosen items Range finding committee (develops scoring guide for SA/ER ?s –Ohio Teachers If it has ? has good stats it is eligible to be on the test.

4. OGT/OAT released items By Ohio law 40% of the test must be released. Both test companies try for 50%. Entire OGT spring test is release except for field tested items.

5. Fairness Issues of Questions Questions should not present: –stereotypes : age, disability, gender, race/ethnicity, sexual orientation –Sensitive or controversial subjects: death/disease, gambling (can not be used in math probability), current politics, race relations, religion, sexuality, superstition, war –Advice- can not advocate specific lifestyles or behavior except in universally adopted ways –Dangerous activities (no dart board or gambling stems )

6. Evaluating Effective Multiple- Choice Items Item- –must be accurate and realistic –Match standard, benchmark, and indicator –Free from bias –Written at or below grade level –Written in the active voice –Contain only information needed Stem (the question) –Must be open (not yes/no), positive (avoid NOT ?s),

7. Evaluating Effective Multiple- Choice Items Options –Same length or ordered by length, alpha or numeric order –Avoid words such as “all” or “only” –Grammatically parallel –Avoid repeated information Key –Answers the stem –Is the only correct option –Should not contain language from stem –Should not be obvious

8. Effective SA/ER Items Item –Must be accurate, realistic –Match standard, benchmark and indicator –Fair and free from bias –Written at or below grade level –Assess significant information –Written in the active voice –Succinct and not contain extra information

9. Effective SA/ER Items The prompt –Avoids unnecessary technical language –Should require more a more complex response than multiple choice items permit. –Requires more than a yes or no response –Contains enough information to focus the student on the task –Indicates amount and type of information that must be included in response –Allows for score point discrepancies –what is needed to earn all or partial credit –Prompt task should be to be completed in allotted time.

10. Effective SA/ER Items The rubric –Aligned with prompt –Allows for multiple solutions in response –Score –point descriptors state what a student must do to earn each point –Should not contain scoring rules that do not correspond with what is asked in prompt –Should not contain ambiguous response exemplars (response examples)

11. OGT SA/ER scoring by Data Recognition Corporation Blind scoring – all are read 2 times –All scorers are must have 80% reliability or they are taken off the question (Scorers must have 4 year degrees and pass an ability test) –2 scores are averaged if adjacent Ex: earn 1 and 2 - 1.5 is score earn a 2 and 4 then it goes to 3 rd blind reader –OGT SA/ER response pages are completely scanned. Reader will evaluate even if student did not answer ? outside of the answer box

12. OAT SA/ER scoring Blind scored by 1 reader (1 in 9 is read twice) 3 rd /4 th grade responses are scanned completely for grading. 5 th -8 th grade response boxes are only scanned for grading.

13. Time spent on questions Multiple choice- should take less than 1 minute SA/ER- should be completed in 5 minutes