1. How to Fail a Student Lisa M. Beardsley-Hardy, PhD, MPH, MBA Director of Education General Conference of Seventh-day Adventists

2. Mene, Mene, Tekel, Parsin Mene : God has numbered the days of your reign and brought it to an end. Tekel : You have been weighed on the scales and found wanting. Peres : Your kingdom is divided and given to the Medes and Persians. “ (Dan 5:26-28 NIV)

3. How to Fail a Student  Absolute cut-off scores  Norm-referenced cut-off score (e.g. 1 SD below mean fails 15% of class independent of competency)  Criterion-referenced

4. How to Fail a Student  Provide experience with assessment and feedback.  Sample multiple times.  Plan for leniency to be fair to other students (e.g., require final paper in week 13 of semester) to allow grace.  Use multiple methods (multi-trait, multi-method sampling).  Determine and document reliability.

5. “ Empirical rule ” or 68-95-99

7. Nedelsky Method Determine minimally competent or borderline “ F-D ” student Divide 1 by the number of answers from which the test taker has to guess. Sum expected scores and divide by number of questions on the exam.

8. Nedelsky example My music teacher thinks that Marian Anderson sings ________any other contralto he has ever heard. My music teacher thinks that Marian Anderson sings ________any other contralto he has ever heard. (A) more well than (B) better than (C) the best of (D) more better over Eliminate A & D, guess between B & C = ½=.50

9. Modified Nedelsky Method  Determine minimally competent or borderline “ F-D ” student  Divide 2 by (2 + the number of answers) from which the test taker has to guess.  Sum expected scores and divide by number of questions on the exam.

10. Modified Angoff Method Requires a copy of the test or at least a wide selection of items from the test item pool. Asks each judge to “ state the probability that the ‘ minimally proficient person ’ would answer each item correctly ” or the number of borderline students out of 100 who would be expected to get the item correct. Multiple rounds of ratings and discussion are conducted. Probabilities are then summed and the total sum = cut score. The ratings and discussion must be conducted separately for each cut score being determined. Used in NAEP, CLEP and many state programs, particularly on tests that are all Multiple Choice (see www.collegeboard.com)

11. Konosuke Matsushita Matsushita Company (ticker MC) –Panasonic http://panasonic.net/corporate/philoso phy/code/04.html

13. Evidence of Closing the Loop  Broad discussion of outcomes data  Discussion of implications of data and outline institutional learning agenda  Document institutional learning in committee minutes, annual departmental report, and reports to students, BT, and accreditation bodies

14. Discussion  Observations  Questions  Recommendations  Next steps

15. Faculty Development Purchase or download A Primer on Setting Cut Scores on Tests of Educational Achievement from Purchase or download A Primer on Setting Cut Scores on Tests of Educational Achievement from http://www.ets.org/Media/Research/p df/Cut_Scores_Primer.pdf