1. Adapted from Educational Research: Competencies for Analysis and Applications (Gay, L.R., Mills, G. E., Airasian, P., 2006) M.Stewart M.Ed. 2011

2.   “Validity is the degree to which a test measures what it is supposed to measure and, consequently, permits the appropriate interpretation of results” (134).  Validity depends on the context in which the assessment is used. In other words, it is designed for a specific purpose and a specific group of people.  So, a test that is valid one quarter, might not be the next. Validity M.Stewart M.Ed. 2011

3.   “Content validity is the extent to which a test measures the intended content area” (134).  Item validity- individual test items are relevant to intended content. If it wasn’t explicitly taught, it shouldn’t be tested.  *Sampling validity- how well the test samples the total content area being tested.  In other words, the questions on the test should be proportionate to the amount of coverage the content received in the classroom.  *most common area for instructors to have issues Types of Validity M.Stewart M.Ed. 2011

4.   Unclear test directions  Confusing/ambiguous test items  Using vocabulary too difficult for test takers  Overly difficult and complex sent. structures.*  Inconsistent and subjective scoring methods  Untaught items included on achievement tests Factors that threaten validity: M.Stewart M.Ed. 2011

5.   A question should only be as complex as the subject matter demands.  Making questions unnecessarily complicated does not mean that you are testing critical thinking, nor does it mean that you are “challenging” them.  In fact, an unnecessarily complicated question actually reduces the validity of the test because you are no longer testing the subject matter, but rather the students’ ability to decipher the complex sentence structure or instructions.  Simple, straightforward subject matter, should be tested with a simple, straightforward question. Either they know it or they don’t. Factors that threaten validity: Overly difficult and complex sentence structures M.Stewart M.Ed. 2011

6.   For second language students and students with some types of learning disabilities, some question types can be more difficult to understand than others. This can interfere with validity.  Form test questions in a positive rather than negative structure, whenever possible, especially for true/false questions.  For example, for a true/false statement, “Hormones are part of the endocrine system.” is much easier to comprehend than, “Hormones are not a component of the Endocrine system.” Helpful Hints M.Stewart M.Ed. 2011

7.   Expose students to test-like questions before the exam.  Including a few review questions in your PowerPoint lectures can be a great way to do a comprehension check over current material and gives students the opportunity to learn what is expected when answering certain kinds of questions. Helpful Hints M.Stewart M.Ed. 2011

8.   Collect all questions of the same type into their own section. For example, all of the true/false questions are together, all of the multiple-choice together, etc.  Don’t assume that students will know what to do. Every section of questions needs its own set of clear directions, even it seems obvious.  Give each question a point value, so students can apply test-taking strategies. Helpful Hints M.Stewart M.Ed. 2011

9.   Often, one of the duties of the basic skills instructor is to review the exams to look for issues with validity. To facilitate this, there are the following expectations:  1) there will be adequate time to collaborate and comment on the exam (two weeks before study guide is to be given)  2) once comments have been given, a final draft incorporating all changes will be sent to BS instructor  3) NO changes will be made to exam after final draft has been agreed upon/study guide created. Collaborating on Exams M.Stewart M.Ed. 2011

10.   Gray, L.R., Mills, G. E., Airasian, P. (2006). Selecting measuring instruments. Educational Research: Competencies for analysis and applications ( 8 th ed. ) (pp 120-157). Upper Saddle River, NJ: Pearson, Merrill, Prentice-Hall. Works Cited M.Stewart M.Ed. 2011