1. Assessing Program Level Student Learning Outcomes: What you need to know University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University SHatfield@winona.edu

2. 1. Assessment is about Learning

3. 2. The expectations have changed

4. DIRECT MEASURES INDIRECT MEASURES 19922012 2002

5. PROCESS MEASURES OUTCOME MEASURES 19922012 2002

6. PROGRAM ASSESSMENT CLASSROOM ASSESSMENT 19922012 2002

7. INSTITUTIONAL RESPONSIBILITY PROGRAM RESPONSIBILITY 19922012 2002

8. INSTITUTIONAL EFFECTIVENESS STUDENT LEARNING 19922012 2002

9. 3. How assessment works

10. How Assessment Works learning event component BASELINE Compare Against Benchmarks, Standards, Targets, Past Performance New / Revised learning event New / Revised learning event New / Revised learning event New / Revised learning event New / Revised learning event New / Revised learning event component OUTCOMEOUTCOME Phase 1 Phase 2Phase 3

11. 4. There is a difference between doing assessment and really doing assessment

12. Reporting Number & Amount Quality & Utility Interpreting Internal QuestionsExternal Questions Collecting it Using it ComplianceCommitment

13. 5. The difference between assessment and evaluation

14. Evaluation Assessment Quizzes Count toward final grade Used to see if students understand Tests Scored and returned Scored, tabulated. returned & discussed; adjustments to syllabus Rubrics Returned to students with grade Returned after being aggregated & analyzed; adjustments to syllabus

15. 6. There is a difference between assessing Program Effectiveness and assessing Student Learning Outcomes

16. Assessment of Program Effectiveness What the program will do or achieve Curriculum Retention Graduation Placement Satisfaction (graduate and employer)

17. Assessment of Student Learning Outcomes What students will do or achieve Direct Methods of Assessment

18. Direct Measures of Learning Capstone experience Standardized tests Performance on national licensure certification or professional exams Locally developed tests Essay questions blind scored by faculty Juried review of senior projects Externally reviewed exhibitions performances Evaluation of internships based upon program learning outcomes

19. Indirect Measures of Learning Alumni, employer, and student surveys (including satisfaction surveys) Exit interviews of graduates and focus groups graduate follow up studies Retention and transfer studies Length of time to degree ACT scores Graduation and transfer rates Job placement rates

20. 7. The key to assessment is Measure what you Value

21. The relationship between values and measures Measures Values

22. 8. It’s a bumpy ride

23. Beginning Mature Making Progress 040506070809101112

24. Beginning Mature Making Progress 040506070809101112

25. Workshops, Speakers and Consultants on student learning Ongoing dialogue on student learning, development and measurement of learning outcomes Student learning has become central to the institution and student learning, performance, and achievement are celebrated BEGINNING PROGRESS MATURING

26. Activities Events SystemsProcesses Traditions BEGINNING PROGRESS MATURING

27. Activities Events SystemsProcesses Traditions BEGINNING PROGRESS MATURING KNOWLEDGE RESOURCES COMMUNICATION COMMITMENT LEADERSHIP

28. 9. The success of any assessment efforts depends on the outcomes

29. Student Learning Outcomes Students will recognize, analyze, and interpret human experience in terms of personal, intellectual, and social contexts.

30. Student Learning Outcomes Students should be able to critically comprehend, interpret, and evaluate written, visual, and aural material.

31. Student Learning Outcomes Students should be able to > >

32. Student Learning Outcomes Learner Centered Specific Action oriented Cognitively appropriate

33. KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower division course outcomes

34. KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Upper division Course / Program outcomes

35. Possible Outcomes Students will appreciate…. Students will be exposed to…. Students will demonstrate knowledge of the history, theories and applications of… Students will attend…. Faculty will provide students with opportunities to….

36. 10. You don’t need a lot of program outcomes

37. Exertion without Intention

38. Exhaustion

39. Intention without Exertion

40. Intention and Exertion

41. 11. Almost all student work can be used for assessment

42. Learning Objects Standardized Exam, abstract, advertisement, annotated bibliography, biography, briefing, brochure, budget, care plan, case analysis, chart, cognitive map, court brief, debate, definition, description, diagram, dialogue, diary, essay, executive summary, exam, flow chart, group discussion, instruction manual, inventory, lab notes, letter to the editor, matching test, mathematical problem, memo, micro theme, multiple choice test, narrative, news story, notes, oral report, outline, performance review, plan, presentation, process analysis, proposal, regulation, research proposal, review of literature, taxonomy, technical report, term paper, thesis, word problem, work of art. (Walvoord Anderson 1998).

43. 12. What an Assessment Plan looks like

44. 44 Mission General Education Program Outacomes Vision Core Competencies External Agencies Outcome

45. Components Supporting Experiences Learning Object/s Assessment Tool Mission Outcome

46. Components Supporting Experiences Learning Object/s Assessment Tool Mission Conduct Independent research Problem significance, previous research, methodology, analysis, interpretation, implications 195, 282, 387, 388, 485 Capstone project in 485 Research Project Rubric

47. Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Present Academic Research Introduction, organization, delivery, visual support, explanation, interaction with audience 191, 282, 387, 389, 485 Capstone presentation in 485 Presentation Rubric, Audience Feedback form

48. Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Propose new solution to ongoing problem Problem, causes, critique of current solutions, alternative solution analysis 282, 366, 381, 365 Analysis of Environmental Issues presentation Proposal rubric

49. Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Analyze Data data processing, statistical analysis, graphic representation, explanation 182, 196, 288, 425 Test in 425 data analysis evaluation scale

50. Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Create a Lesson Plan Objectives, teaching methods, modeling, comprehension check, practice, assessment, materials 151. 162, 182, 196, 252, 268, 324, 325, 350, 355, 425 Portfolio Presentation of lesson plan Lesson plan rubric Observational feedback scale

51. Assessing Program Level Student Learning Outcomes: What you need to know University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University SHatfield@winona.edu