ALS Design, Build, Review: Using PDE’s Online Tools to Implement the SLO Process SAS Portal:

Navigate to the homeroom page: RIA Homeroom site.

Log in and if not a user then register for the site: Pause until entire room is registered or with a partner:

Home Page for information: Open ALS

The ALS Box expands…………..

Assessment Literacy Series 6 -Module 6- Quality Assurance & Form Reviews

SAM-L Development participants will: 1. Apply a quality assurance checklist to the developed materials to ensure all documents meet the requirements. 2. Verify the item is measuring the intended content and standard. 3. Ensure the scoring key/rubrics are correct. 7 Objectives

Participants may wish to reference: Guides Handout #10 – Quality Assurance Checklist QA Screening Tool Performance Measure Rubric-Teacher Other “Stuff” “Smart Book” 8 Helpful Tools

9 Outline of Module 6 Module 6: Form Reviews Content Review Fairness, Sensitivity, Bias Apply Checklist to all forms Alignment Rigor

Form Review Reviewing forms involves the following components:  Content Review  Bias  Fairness  Sensitivity  Accessibility  Alignment Note: Refer to “Smart Book” for detailed explanations of these components. 10

Content Review  Determine if each item/task clearly aligns to the targeted content standard.  Evaluate all items for accuracy.  Judge if each item is age and grade appropriate for students. Reading level is appropriate. Vocabulary is appropriate. Required reasoning skills are appropriate.  Review each item stem and answer options to be sure they are clear and understandable. Student response expectations are clear. 11

Item Bias  Bias is the presence of some characteristic of an item that results in the differential performance of two individuals with the same ability but from different subgroups.  Bias is not the same as stereotyping.  Bias-free items/tasks provide an equal opportunity for all students to demonstrate their knowledge and skills. 12

Fairness and Sensitivity Review to ensure:  Items are sensitive to different cultures, religions, ethnic and socio-economic groups, and disabilities.  There is a balance of gender roles.  Use of positive language, situations, and images.  Items avoid text that may elicit strong emotions in specific groups of students and prevent those students from accurately demonstrating their skills and knowledge. 13

Sensitive Topics to Avoid Abortion Birth control Child abuse/neglect Creationism Divorce Incest Illegal activities Occult/witchcraft Questioning parental authority Rape Religious doctrine Sex/sexuality Sexual orientation Weight Suicide Sexually transmitted diseases *Note: The above list does not contain every sensitive topic that item developers should avoid and does not consider test-taker’s age/maturity. 8

What is Accessibility?  Test accessibility is defined as the extent to which a test eliminates barriers and permits the test-taker to demonstrate his or her knowledge of the tested content.  All items should be reviewed to ensure they are accessible to the entire population of students.  Item reviews must consider: Readability Syntax complexity Item presentation Font size Clarity of images, graphs, tables Amount of work space 15

Alignment Categorical Concurrence ◦ The same categories of the content standards are included in the assessment ◦ Items might be aligned to more than one standard Source of Challenge ◦ Students give a correct or incorrect response for the wrong reason (bias) Balance of Representation ◦ Ensures there is an even distribution of the standards across the test Range of Knowledge ◦ The extent of knowledge required to answer correctly parallels the knowledge the standard requires Depth of Knowledge ◦ The cognitive demands in the standard must align to the cognitive demands in the test item Source: Webb, N. L. (1997). Research Monograph No. 6: Criteria for alignment of expectations and assessments in mathematics and science education. Washington, DC: Council of Chief State School Officers 16

Quality Assurance QA Screening Tool ◦ Use to examine first draft materials ◦ Outlines key criteria Quality Assurance Checklist [Handout #10] ◦ Use with final drafts ◦ Examines each developed component of the performance measure. Performance Measure Rubric-Teachers ◦ Comprehensive self-assessment of teacher- made performance measures. 17

Process Steps 1. Apply the Quality Assurance Checklist [Handout #10] to the performance measure and supporting documents. Apply the checklist to each individual item /task, and then evaluate the measure in its totality. 2. Make notes directly on the paper copies of the documents. 3. Review areas that may need additional focus. 4. Print and conduct a final team review of all documents, particularly the operational form and scoring keys. 5. Turn your packet in to the facilitator, and destroy draft material. 18

19 Summary Module 6: Form Reviews Evaluated the documents created by the performance measure development process, and ensured the measures were of high-quality, rigorous, and aligned to the targeted content standards. All documents went through a quality assurance process prior to finalization.

Joining the SLO Professional Learning Community on SAS. Go to the SAS home page( ) Log in with your user name and password. If you do not have an account with SAS you will have to create one.

Enter your information on the log in page and submit.

Once you have successfully logged in and are at the SAS home page, go to Teacher Tools in the upper right corner.

Click on Teacher Tools, this will provide you with various tools. Locate the button labeled “My Communities.”

This will open your membership to various Professional Learning Communities. If you are not a member of the Student Learning Objectives PLC, type SLO in the search bar.

Once a member of the SLO community you will have access to communication with all other members and a calendar of upcoming events.

Along with posting questions to the entire community you have access to the Digital Repository, in which SLO training materials and supporting documents are located. (This is located at the bottom of the SLO community page.)

Contact Info PDE POC: Mr. O David Deitz RIA POC: Dr. JP Beaudoin

SAS Institute