1. Gathering, Analyzing, Using & Reporting Assessment Data University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University SHatfield@winona.edu

2. 1. Gathering Information

3. Learning Objects Standardized Exam, abstract, advertisement, annotated bibliography, biography, briefing, brochure, budget, care plan, case analysis, chart, cognitive map, court brief, debate, definition, description, diagram, dialogue, diary, essay, executive summary, exam, flow chart, group discussion, instruction manual, inventory, lab notes, letter to the editor, matching test, mathematical problem, memo, micro theme, multiple choice test, narrative, news story, notes, oral report, outline, performance review, plan, presentation, process analysis, proposal, regulation, research proposal, review of literature, taxonomy, technical report, term paper, thesis, word problem, work of art. (Walvoord Anderson 1998).

4. Learning Objects Test Project Assignment Portfolio Recital Performance Presentation Exhibit Internship

5. Student Learning Outcome 3 2 1 4 ComponentsTest Questions 1234 5678 101112 1713141516 9 18

6. Student Learning Outcome 3 2 1 4 ComponentsAssignment 1 2 3 4

7. Student Learning Outcome 3 2 1 4 ComponentsAssignment 1 2

8. Student Learning Outcome 3 2 1 4 ComponentsExperience

9. Assessing Learning Test Project Case Study Portfolio Recital Performance Presentation Exhibit Internship Number or percentage correct

10. Student Learning Outcome 3 2 1 4 Test Questions 1234 5678 101112 1713141516 9 18 73% 82% 64% 87% % correct

11. Assessing Learning Test Project Case Study Portfolio Recital Performance Presentation Exhibit Internship Number or percentage correct Scale

12. Semantic Differential Repeat mode effective / ineffective

13. Student Learning Outcome 2 1 Components 1 2 3 4 effective ineffective

14. Scale Semantic Differential Repeat mode effective / ineffective unique descriptors enhances / distracts original / common always / never engaged / disconnected

15. Student Learning Outcome 2 1 Components 1 2 3 4

16. Assessing Learning Test Project Case Study Portfolio Recital Performance Presentation Exhibit Internship Number or percentage correct Scale Rubric

17. Rubrics Analytical Numerous components which you want to assess individually Summative

18. Components Analytical / Summative Rubric Does not meet expectations Outcome:

19. Components Meets expectations Analytical / Summative Rubric Outcome:

20. Components Exceeds expectations Outcome: Analytical / Summative Rubric

21. Components Analytical / Developmental Rubric Outcome: complexity

22. Components Analytical / Developmental Rubric Outcome: detail

23. Components Analytical / Developmental Rubric Outcome: sophistication

24. Components Analytical / Developmental Rubric Fourth year Outcome:

25. Rubrics Analytical Holistic - generally smaller assignments with fewer criteria, shorter turn around time.

26. Exceptional component Level Holistic Rubric Above Average component Average component Below Average component

27. Resource www.winona.edu/air/rubrics.htm

28. PRACTICAL ADVICE: Pilot test your rubrics!

29. Verbal Delivery Nonverbal Delivery Organization Does not meet MeetsExceeds 1% 3% 0% 15% 5% 3% 84% 92% 97% Evidence 1%6% 93% Transitions 2%8% 90%

30. 2. Analyzing Data

31. Interpreting Data  Consistency - over time  Consensus - different populations  Distinctiveness - different situations/ variables / items

32. Consistency Examines the same practice of and individual or group over time Key question: Has this person or group acted, felt, or performed this way in the past / over time?

33. Consistency How well are students performing on the learning outcome? High performance Low performance 0405060708091011

34. Consensus Comparison to or among groups of students Variation between disciplines, gender, other demographic variables Key questions: What is the general feeling, outcome, attitude, behavior? Do other groups of people act, perform or feel this way?

35. Consensus How well are students performing on the learning outcome? FemalesMalesTransfersOTA High performance Low performance

36. Distinctiveness Examines individual or cohort perspectives across different situations, categories Key Question : Does a person or group respond differently based upon the situation, item, issue?

37. Distinctiveness How well are our students achieving our Core Learning Outcomes? High Performance Low Performance

38. 3. Using Data

39. How Assessment Works component BASELINE Compare Against Benchmarks, Standards, Targets, Past Performance New / Revised learning event New / Revised learning event New / Revised learning event Year 2

40. Acting on Results Development Faculty, Staff, Student Infrastructure Policy, Process, Planning Curriculum Learning Opportunities

41. The Seven Principles for Good Practice in Undergraduate Education 1. Student-Faculty Contact 2. Cooperative Learning 3. Active Learning 4. (Prompt) Feedback 5. Time on Task 6. High Expectations 7. Respect for Diverse Talents and Ways of Learning

42. Increase Reduce Determine Maintain the Number Amount Percentage Ratio Level of who that with Someone Majors Faculty Staff Community members Employers > Something Can analyze Performance data

43. Increase Reduce Improve Determine Maintain the Someone Students Faculty Staff Community members Employers Something applications grants Interviews request workshops reports satisfaction engagement by of who that with Action Number Amount Percentage Ratio Level > Something

44. Educating….. Assisting ….. Explaining ….. Assisting ….. Developing ….. Consulting ….. Delivering ….. Facilitating ….. Creating ….. Nurturing ….. Negotiating ….. Sponsoring ….. Training ….. Translating ….. Validating ….. Updating ….. Enforcing ….. Testing ….. Assessing ….. Reporting ….. Issuing ….. Communicating ….. Integrating ….. Responding ….. Arranging ….. Coordinating ….. Providing ….. Organizing ….. Producing ….. Training ….. Maintaining ….. Supporting ….. Directing ….. Designing ….. Collecting ….. Developing ….. Responding ….. Reviewing ….. Studying ….. Distributing ….. Executing ….. Recruiting ….. Generating ….. Implementing ….. Investigating ….. Designing ….. Collaborating ….. Revising ….. Scheduling ….. Sharing ….. Increase Reduce Improve Determine Maintain the Number Amount Percentage Ratio Level Something applications grants Interviews request workshops reports satisfaction engagement by of who that with Someone Students Faculty Staff Community members Employers > Something

45. Increase Reduce Determine Maintain the Number Amount Percentage Ratio Level of who that with Someone Second year students Faculty Staff Community members Employers > Something Demonstrate competence in critical thinking Something Action by

46. Increase Reduce Determine Maintain the Number Amount Percentage Ratio Level of who that with Someone Second year students Faculty Staff Community members Employers > Something Demonstrate competence in critical thinking Something Action by

47. 4. Reporting Data

48. Outcome outcome What did you look at? object What did you find? results What does it mean? analysis What are you going to do about it? action What happened? reaction

49. Spring 2012 Summer 2012 Fall 2012 Spring 2013 Summer 2013 After That

50. Gathering, Analyzing, Using & Reporting Assessment Data University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University SHatfield@winona.edu