1. New England Common Assessment Program Grades 3-8 & 11 Bias/Sensitivity Review- Part 2 March 26 and 27, 2006 Quechee, VT

2. What Is Item Bias? Bias is the presence of some characteristic of an assessment item that results in the differential performance of two individuals of the same ability but from different student subgroups Bias is not the same thing as stereotyping (but we don’t want stereotypes in NECAP either) We need to ensure that ALL students have an equal opportunity to demonstrate their knowledge and skills

3. The “Big Idea” of Bias-Sensitivity Review NECAP should measure the results of learning opportunities NOT life experiences

4. Reading Passages Selected by Item Developers Bias-Sensitivity Review of Reading Passages – Focus on Sensitivity Issues Reading and Math Items are Developed Bias-Sensitivity Review of Reading & Math Items – Focus on Item Bias Item Review of Reading and Math Items Field-Testing Feedback Pilot-Testing Data Analysis (DIF) How Do We Prevent Item Bias?

5. Review reading passages, and then both math and reading items Complete independent ratings of each passage or item and record comments Be prepared to discuss “Mixed Reviews” and “Problem Items” with your review team; Finalize independent ratings Please Note: The Bias-Sensitivity Review Committee plays an advisory role. Final decisions on which passages and items to include in NECAP will be made by the NH, RI and VT DOEs Role of the Bias-Sensitivity Review Committee

6. Sensitivity to different cultures, religions, ethnic and socio-economic groups, and disabilities Balanced gender roles that are free of gender stereotypes Use of positive language, situations and images Use of context, situations, images and examples that are universally understood across groups and cultures Overall, that the test is free of items and text that may elicit strong emotions in specific groups of students, and as a result, prevent those groups of students from accurately demonstrating their skills and knowledge Overall, that the test is free of items that measure life experiences instead of learning experiences Role of the Bias-Sensitivity Review Committee The Bias-Sensitivity Review Committee DOES need to make recommendations that will help ensure…

7. Reading Level Grade Level Appropriateness GE Alignment Instructional Relevance Language Structure and Complexity Accessibility Overall Item Design Role of the Bias-Sensitivity Review Committee The Bias-Sensitivity Review Committee DOES NOT need to make recommendations concerning… The Item Review Committees Are Addressing These Issues and Factors

8. December 12 th and 13th – Review All Reading Passages March 26th and 27 th – Review All Reading and Mathematics Test Items Process: Today – (1) Training, (2) Practice, (3) Clarification of Guidelines and Procedures, (4) Break Into Two Groups for Independent Review of Item Sets – Group 1 will review reading and math for grades 3, 5 and 7 and math for grade 11, and Group 2 will review reading and math for grades 4, 6 and 8 and reading/writing for grade 11. Tomorrow – (1) Break into Small Groups for Re- Evaluation and Discussion of “Problem Items” (2) Finalize Independent Ratings, (3) Evaluate the Review Process and Provide Feedback The Bias-Sensitivity Review Process

9. Avoid issues and topics considered sensitive by the general public Keep in mind that some topics that are appropriate for classroom discussion may not be appropriate for assessment Consider context and setting; Bias-Sensitivity Review is not a “literal” process Think of “bias” as an assessment issue as well as a cultural issue, that is, a specific topic or feature of a passage or item that may disadvantage one of two otherwise equally matched groups of students General Guidelines

10. Abortion Birth Control Child Abuse/Neglect Creationism Divorce Incest Occult/Witchcraft Questioning Parental Authority Rape Religious Doctrine Sex/Sexuality Sexual Orientation Sexually Transmitted Diseases Suicide Weight Topics to Avoid in Reading Passages & Test Items

11. Death Drugs, Alcohol & Tobacco Family Issues Guns/ Gun Control Handicapping Conditions Holidays Homelessness Misuse of Animals/ Animal Rights Murder Pregnancy Racism, Sexism & Ageism Religion (Except in Historical Context) Violence Topics Which May or May Not Be Acceptable

12. Avoid economic, regional, cultural or gender bias in questions, prompts, identified situations, graphics or reading selections Avoid profanity unless it has a clear literary or historical context Keep in mind that the ability to answer a test question should reflect learning opportunities not cultural opportunities or life experience Remember that we are proud of our states and region and expect to see that pride reflected in assessment passages and items Awareness Issues

13. New England Common Assessment Program Bias & Sensitivity Review Committee Item Review Evaluation Form Read the statement below and select the rating that best describes your evaluation of the item for potential bias and sensitivity concerns. “This item does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students.” RatingDescription 4Strongly Agree 3Agree 2Disagree 1Strongly Disagree Comments Please provide specific comments to explain your evaluation. Your comments should address your particular bias/sensitivity concerns, the subgroup(s) that you feel may be impacted, and the extent of the potential impact within the subgroup(s). Item Review Rating Form

14. Once, I got my head stuck in our porch railing. If that man with a cell phone hadn’t stopped and called the fire department, I’d probably still be there. Cell phones can save lives. Everybody at school has a cell phone. Once, during an English test, somebody actually got a call and in a regular speaking voice, said, “Sorry, I can’t talk now because I’m taking an English test.” Bias-Sensitivity Examples and Non-Examples “This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students” Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree

15. What’s the Point? “Avoid ECONOMIC, regional, CULTURAL or gender bias in questions, prompts, identified situations…” Okay because the writer’s culture and economic status are not factors in the passage Not okay because the passage assumes a cultural and economic status that won’t be shared by all test-takers

16. Bias-Sensitivity Examples and Non-Examples “This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students” Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree  Larry buys an iPod ® that has a regular price of $120. The iPod ® is on sale for 30% off. What is the sale price of the iPod ®?  A. $150  B. $84  C. $36  D. $90  Larry buys a sweater that has a regular price of $40. The sweater is on sale for 30% off. What is the sale price of the sweater? Show your work or explain how you know.

17. What’s the Point? “Avoid economic, regional, CULTURAL or gender bias in questions, prompts, identified situations…” Not okay because many students can’t afford luxury items such as IPods Okay because clothing is a necessity and $40 is a more reasonable amount “(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

18. Bias-Sensitivity Examples and Non-Examples “This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students” Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree Explain how the Model T differed from earlier attempts to build a “horseless carriage.” Use examples from the story. (Related to a reading passage called “Henry’s Horseless Carriage.”) Explain how the antique cars described in the story are different than modern cars. Provide at least 3 examples. (Related to a reading passage called “Henry’s Horseless Carriage.”)

19. What’s the Point? “Avoid economic, regional, CULTURAL or gender bias in questions, prompts, identified situations…” Okay because student is only required to use information contained in the passage. Not okay because the student needs both information from the story AND information from personal experience “(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

20. Bias-Sensitivity Examples and Non-Examples “This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students” Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree

21. What’s the Point? “ We need to ensure that ALL students have an equal opportunity to demonstrate their knowledge and skills” Okay because the context is generally understood by all students. Not okay because some students (particularly VT & NH students) may have experience using the trail guide formula “(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

22. Bias-Sensitivity Examples and Non-Examples “This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students” Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree Rating Descriptions 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree ŒThe word hear belongs in which sentence?  A. People come ____ for vacation.  B. I want to sit _____ by the window.  C. Take cover when you _____ thunder.  D. Please put my books over _____. ŒThe word hear belongs in which sentence?  A. People come ____ for vacation.  B. I want to sit _____ by the window.  C. She can _____ the loud thunder.  D. Please put my books over _____. ? ?

23. What’s the Point? “Avoid economic, regional, CULTURAL or gender bias in questions, prompts, identified situations…” Is this NOT okay because thunder may not be “universally understood” by persons who are deaf? Is this better? Is it okay because the sentence refers to another person’s hearing, not the reader’s “(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”

24. The “Big Idea” of Bias-Sensitivity Review NECAP should measure the results of learning opportunities NOT life experiences