1. David Eubanks Director of Planning, Assessment, and Information Services deubanks@coker.edu Coker College Hartsville, South Carolina Assessing The Elephant

2. map

3. The Goal The College's goal is to graduate students with the ability to think analytically and creatively, and to write and speak effectively.

4. The Unmoved Mover [I]t is imperative […] to avoid narrow definitions of student learning or excessively standardized measures of student achievement.

5. The Unmoved Mover Collegiate learning is complex, and the evidence used to investigate it must be similarly authentic and contextual.

6. The Unmoved Mover But to pass the test of public credibility […] the evidence of student learning outcomes used in the accreditation process must be rigorous, reliable, and understandable. All CHEA quotes are from: http://www.chea.org/pdf/StmntStudentLearningOutcomes9-03.pdf

7. public credibility

8. map

9. The question “Can Stanislav write effectively?” should be answered in language that is rigorous, reliable, and understandable.

10. “…the meaning of a word is its usage in the language…” -Wittgenstein language

11. Context and perspective

12. map

13. Power of dialogue Example: If we want to distinguish between 128 possibilities (where n = 7), Monologue requires 127 questions; Dialogue requires 7.

14. map

15. Complexity vs. reliability

16. Example: arithmetic 13 + 5 = 18 ? This can be assessed with high reliability and high validity because it’s algorithmic.

17. Example: Turing Test The Turing Test for Machine Intelligence: If a computer, on the basis of its written replies to questions, could not be distinguished from a human respondent, then ‘fair play’ would oblige one to say that it must be ‘thinking’. -Quoted from Alan Turing: The Enigma, by Andrew Hodges

18. map

19. Assessment as a Tire gauge We can all agree that the tire pressure is about 30 psi.

20. Assessment as telescope You don’t know what you’ll discover when you build the telescope, but we can create language to describe it.

21. map

22. Just The Facs, ma’am The Faculty Assessment of Core Skills (FACS) Mission-based (effective writing, etc.) Rubrics in course syllabi Evaluation of authentic evidence each term Aggregation of results into group perception

23. features Uses authentic data in context Uses full perceptive ability of experts Fosters dialogue among students, faculty Creates a “Wittgensteinlich” language Has high prima fascia validity Has measurable reliability Is “low stakes”, reducing rater bias

24. Web survey

25. 2003-2006 data 11,770 responses 1743 of 2054 traditional students (85%) Correlation with grades =.64

26. More Validity

27. Inter-rater Reliability Probability of matching ratings between two raters for same student and skill:

28. Longitudinal DATA

29. THE matthew effect

30. Distributions

31. Class effectiveness

32. Student/Advisor Report Student has 104 Credits (about seven semesters) Comparison group averages: 1.9, 1.8, 1.8, 1.9, respectively.

33. Program report

34. FEEDBACK AS Assessment

35. Contact Terri Flateby at TFlateby@admin.usf.edu

36. Is this effective writing?