1. Assessment 101: The Core Curriculum Susan Hatfield - Winona State UNC Wilmington November 2005 SHatfield@winona.edu

2. Common Reactions to Assessment Initiatives Ignoring it Bribing someone else to do it Complaining about it Losing sleep over it Sitting down and writing it

3. Assessment is -- first and foremost -- about student learning.

4. A Little Quiz

5. Quiz All faculty need to be actively engaged in assessment for a department or program to really be “doing assessment.”

6. Quiz All faculty need to be deeply committed to assessment for a department or program to really be “doing assessment.”

7. Attitudes toward Assessment Level of Commitment HostileAcceptingEnthusiastic 15% 70%

8. Quiz The best way to build a campus culture of assessment is for top administration to be prescriptive in student learning outcomes, assessment measures and methods.

9. Quiz All departments and programs should be moving ahead on assessment at the same pace, meeting specific goals within specified time frames.

10. Quiz Effective assessment programs have eliminated indirect measures and measures of departmental effectiveness (satisfaction, etc.).

11. Quiz Assessment is a time-intensive add-on that will be a huge burden to faculty who are already overburdened.

12. Quiz Effective programmatic assessment plans have every faculty member assess every outcome in every course every semester.

13. Quiz Accrediting organizations expect to see fully realized assessment plans during site visits.

14. Quiz The most effective assessment programs are ones in which the students are not aware they are being assessed.

15. Uncertainty Uncertainty related to questions of How to do it Why it needs to be done What to do with the data How the data will be used How to find the time to implement it What support is available

16. Reducing Uncertainty Clear Understanding Definitions Processes Rules Resources Understanding of pitfalls

17. The Assessment Core Curriculum: 10 things you should know about assessment

18. Core Curriculum 1.Assessment terminology 2.Evolution of assessment initiatives 3.Difference between direct and indirect measures of learning 4.Assessment requires exertion and intention 5. Writing student learning outcomes

19. Language of Assessment A. General skill or knowledge category GOAL B. Specific accomplishments to be achieved OUTCOME C. Activities and Assignments to help students learn LEARNING EVENTS D. The key elements related to the accomplishment of the outcome COMPONENTS

20. Language of Assessment E. The objects of analysis OBJECTS F. Data indicating degree of achievement CHARACTERISTICS G. Combination of data indicating relative degree of achievement of the learning outcome INDICATORS

21. Goals

22. Organizing Principle Category or Topic Area Subjects

23. Goals Composition PE Humanities Fine Arts Natural Sciences and Mathematical Sciences Social and Behavioral Sciences Interdisciplinary Perspectives

24. Learning Outcomes

25. Student Learning Outcomes Communication RelatingSpeakingListeningTeamingWriting

26. Learning Events

27. Assignments (in class and out of class) Feedback on practice Self evaluation Peer evaluation Role Play Pre Tests Simulation

28. Learning Objects

29. Student Learning Outcomes Communication RelatingSpeakingListeningTeamingWriting Sales

30. Components

31. Student Learning Outcomes Goal Outcome Object Learning events

32. Student Learning Outcomes Communication RelatingSpeakingListeningTeamingWriting Demonstration Verbal Nonverbal Organization Evaluative elements

33. Performance Characteristics

34. Student Learning Outcomes Communication RelatingSpeakingListeningTeamingWriting Demonstration Verbal Nonverbal Organization

35. Indicators

36. Student Learning Outcomes GOAL component Outcome Object component Outcome Degree to which outcome is achieved indicator

37. Core Curriculum 1.Assessment terminology 2.Evolution of assessment initiatives 3.Difference between direct and indirect measures of learning 4.Assessment requires exertion and intention 5. Writing student learning outcomes

38. Maturing Assessment INSTITUTIONAL RESPONSIBILITY DEPARTMENT RESPONSIBILITY BEGINNING PROGRESS MATURING

39. Maturing Assessment DIRECT MEASURES INDIRECT MEASURES BEGINNING PROGRESS MATURING

40. Maturing Assessment PROCESS MEASURES OUTCOME MEASURES BEGINNING PROGRESS MATURING

41. Maturing Assessment BEGINNING PROGRESS MATURING PROGRAM ASSESSMENT CLASSROOM ASSESSMENT

42. Maturing Assessment INSTITUTIONAL EFFECTIVENESS STUDENT LEARNING BEGINNING PROGRESS MATURING

43. Evolutionary Trajectories BEGINNING MAKING PROGRESS MATURING 0405990001020398

44. Core Curriculum 1.Assessment terminology 2.Evolution of assessment initiatives 3.Difference between direct and indirect measures of learning 4.Assessment requires exertion and intention 5. Writing student learning outcomes

45. Direct Measures of Learning Capstone experience Standardized tests Performance on national licensure certification or professional exams Locally developed tests Essay questions blind scored by faculty Juried review of senior projects Externally reviewed exhibitions performances Evaluation of internships based upon program learning outcomes

46. Indirect Measures of Learning Alumni, employer, and student surveys (including satisfaction surveys) Exit interviews of graduates and focus groups graduate follow up studies Retention and transfer studies Length of time to degree ACT scores Graduation and transfer rates Job placement rates

47. Non-Measures of Student Learning Curriculum review reports Program review reports from external evaluators Faculty publications and recognition Course enrollments and course profiles Faculty / student ratios, percentage of students who study abroad Enrollment trends 5 year graduation rates Diversity of the student body

48. Core Curriculum 1.Assessment terminology 2.Evolution of assessment initiatives 3.Difference between direct and indirect measures of learning 4.Assessment requires exertion and intention 5. Writing student learning outcomes

49. Exertion without Intention

50. Intention without Exertion

51. Intention and Exertion

52. Core Curriculum 1.Assessment terminology 2.Evolution of assessment initiatives 3.Difference between direct and indirect measures of learning 4.Assessment requires exertion and intention 5. Writing student learning outcomes

53. Learning Outcome Format –Students should be able to > >

54. KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Identify Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower division course outcomes

55. KNOWLEDGE COMPREHENSION APPLICATIONANALYSISSYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Identify Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Upper division Course / Program outcomes

56. Learning Outcome Rules –Only one action verb –Identify single accomplishments –Focus on students, not faculty or curriculum

57. Example #1 Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected of Bias

58. Example #2 Imagine and seek out a variety of possible goals, assumptions, interpretations, or perspectives which can give alternative meanings or solutions to given situations or problems

59. Example #3 Formulate and test hypotheses by performing laboratory, simulation, or field experiments in at least two of the natural science disciplines (one of these experimental components should develop, in greater depth, students’ laboratory experience in the collection of data, its statistical and graphical analysis, and an appreciation of its sources of error and uncertainty)

60. Core Curriculum 6. What an assessment plan looks like

61. Component RelateSpeakListenParticipateWrite

62. Core Curriculum 6.What an assessment plan looks like 7.How assessment works

63. How Assessment Works LEvent 1 LEvent 2 LEvent 3 component BASELINE Process Reflection; Compare results against Benchmarks, Standards, Targets, Past Performance New / Revised LEvent 1 New / Revised LEvent 2 New / Revised LEvent 3 New / Revised LEvent 1 New / Revised LEvent 2 New / Revised LEvent 3 component OUTCOMEOUTCOME Cycle 1 Cycle 2Cycle 3

64. Core Curriculum 6.What an assessment plan looks like 7.How assessment works 8.Why you need to define your outcomes

65. volume poise conclusion Speaking teacher5 eye contact style appearance gestures rate evidence sources examples organization transitions verbal variety attention getter teacher4teacher2teacher1teacher3

66. volume poise conclusion eye contact style appearance gestures rate evidence sources examples organization transitions verbal variety attention getter Can our students deliver an effective Public Speech?

67. Core Curriculum 6.What an assessment plan looks like 7.How assessment works 8.Why you need to define your outcomes 9.How to implement the plan

68. Student Learning Outcomes Course 1 X X X Course 2 X X X X X Course 3 X X X X Course 4 X X X X X Course 5 X X X X Phase Four

69. Core Curriculum 6.What an assessment plan looks like 7.How assessment works 8.Why you need to define your outcomes 9.How to implement the plan 10. What to do with the data

70. Patterns of Evidence DP1DP2DP3

71. Interpreting Data Consistency - over time Consensus - different populations Distinctiveness - different situations/ variables / items

72. Consistency Examines the same practice of and individual or group over time Key question: »Has this person or group acted, felt, or performed this way in the past / over time?

73. Consistency How well are students performing on the departmental learning outcome measures? 040500010203 High performance Low performance

74. Consensus Comparison to or among groups of students »Variation between disciplines, gender, other demographic variables Key questions: –What is the general feeling, outcome, attitude, behavior? »Do other groups of people act, perform or feel this way?

75. Consensus How well are students performing on the departmental learning outcome measure ? FemalesMalesTransfersOTA High performance Low performance

76. Distinctiveness Examines individual or cohort perspectives across different outcomes Key Question: »Does a person or group perform equally as well on different outcomes?

77. Distinctiveness ANALYSISANALYSIS RESEARCHRESEARCH WRITINGWRITING SPEAKINGSPEAKING How well are our students performing on the learning outcomes? High Performance Low Performance ETHICSETHICS THINKINGTHINKING

78. Fundamental Question #1 Do we have enough data from which to really draw conclusions?

79. Fundamental Question #2 Does the data represent an identifiable trend in the level of activity / achievement / accomplishment?

80. Fundamental Question #3 Does the data represent an acceptable level of activity / accomplishment / achievement given our mission and values?

81. Fundamental Question #4 Are the differences in the sub- populations acceptable?

82. Fundamental Question #5 What can we do about it?

83. Really Big Mistakes

84. Big Mistakes in Assessment Assuming that it will go away Allowing assessment planning to become gaseous Assuming you got it right -- or expecting to get it right -- the first time Not considering implementation issues when creating plans

85. Big Mistakes in Assessment Borrowing plans and methods without acculturation Setting the bar too low Assuming that you’re done and everything’s OK, or rushing to “Close the Loop” Doing it for accreditation instead of improvement

86. Big Mistakes in Assessment Confusing program effectiveness with student learning Making assessment the responsibility of one individual Assuming collecting data is Doing Assessment

87. Assessment 101: The Core Curriculum Susan Hatfield - Winona State UNC Wilmington November 2005 SHatfield@winona.edu