Assessing Program Level Student Learning Outcomes: What you need to know University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University

1. Assessment is about Learning

2. The expectations have changed

DIRECT MEASURES INDIRECT MEASURES

PROCESS MEASURES OUTCOME MEASURES

PROGRAM ASSESSMENT CLASSROOM ASSESSMENT

INSTITUTIONAL RESPONSIBILITY PROGRAM RESPONSIBILITY

INSTITUTIONAL EFFECTIVENESS STUDENT LEARNING

3. How assessment works

How Assessment Works learning event component BASELINE Compare Against Benchmarks, Standards, Targets, Past Performance New / Revised learning event New / Revised learning event New / Revised learning event New / Revised learning event New / Revised learning event New / Revised learning event component OUTCOMEOUTCOME Phase 1 Phase 2Phase 3

4. There is a difference between doing assessment and really doing assessment

Reporting Number & Amount Quality & Utility Interpreting Internal QuestionsExternal Questions Collecting it Using it ComplianceCommitment

5. The difference between assessment and evaluation

Evaluation Assessment Quizzes Count toward final grade Used to see if students understand Tests Scored and returned Scored, tabulated. returned & discussed; adjustments to syllabus Rubrics Returned to students with grade Returned after being aggregated & analyzed; adjustments to syllabus

6. There is a difference between assessing Program Effectiveness and assessing Student Learning Outcomes

Assessment of Program Effectiveness What the program will do or achieve Curriculum Retention Graduation Placement Satisfaction (graduate and employer)

Assessment of Student Learning Outcomes What students will do or achieve Direct Methods of Assessment

Direct Measures of Learning Capstone experience Standardized tests Performance on national licensure certification or professional exams Locally developed tests Essay questions blind scored by faculty Juried review of senior projects Externally reviewed exhibitions performances Evaluation of internships based upon program learning outcomes

Indirect Measures of Learning Alumni, employer, and student surveys (including satisfaction surveys) Exit interviews of graduates and focus groups graduate follow up studies Retention and transfer studies Length of time to degree ACT scores Graduation and transfer rates Job placement rates

7. The key to assessment is Measure what you Value

The relationship between values and measures Measures Values

8. It’s a bumpy ride

Beginning Mature Making Progress

Beginning Mature Making Progress

Workshops, Speakers and Consultants on student learning Ongoing dialogue on student learning, development and measurement of learning outcomes Student learning has become central to the institution and student learning, performance, and achievement are celebrated BEGINNING PROGRESS MATURING

Activities Events SystemsProcesses Traditions BEGINNING PROGRESS MATURING

Activities Events SystemsProcesses Traditions BEGINNING PROGRESS MATURING KNOWLEDGE RESOURCES COMMUNICATION COMMITMENT LEADERSHIP

9. The success of any assessment efforts depends on the outcomes

Student Learning Outcomes Students will recognize, analyze, and interpret human experience in terms of personal, intellectual, and social contexts.

Student Learning Outcomes Students should be able to critically comprehend, interpret, and evaluate written, visual, and aural material.

Student Learning Outcomes Students should be able to > >

Student Learning Outcomes Learner Centered Specific Action oriented Cognitively appropriate

KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Lower division course outcomes

KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION Cite Count Define Draw Identify List Name Point Quote Read Recite Record Repeat Select State Tabulate Tell Trace Underline Associate Classify Compare Compute Contrast Differentiate Discuss Distinguish Estimate Explain Express Extrapolate Interpolate Locate Predict Report Restate Review Tell Translate Apply Calculate Classify Demonstrate Determine Dramatize Employ Examine Illustrate Interpret Locate Operate Order Practice Report Restructure Schedule Sketch Solve Translate Use Write Analyze Appraise Calculate Categorize Classify Compare Debate Diagram Differentiate Distinguish Examine Experiment Inspect Inventory Question Separate Summarize Test Arrange Assemble Collect Compose Construct Create Design Formulate Integrate Manage Organize Plan Prepare Prescribe Produce Propose Specify Synthesize Write Appraise Assess Choose Compare Criticize Determine Estimate Evaluate Grade Judge Measure Rank Rate Recommend Revise Score Select Standardize Test Validate Upper division Course / Program outcomes

Possible Outcomes Students will appreciate…. Students will be exposed to…. Students will demonstrate knowledge of the history, theories and applications of… Students will attend…. Faculty will provide students with opportunities to….

10. You don’t need a lot of program outcomes

Exertion without Intention

Exhaustion

Intention without Exertion

Intention and Exertion

11. Almost all student work can be used for assessment

Learning Objects Standardized Exam, abstract, advertisement, annotated bibliography, biography, briefing, brochure, budget, care plan, case analysis, chart, cognitive map, court brief, debate, definition, description, diagram, dialogue, diary, essay, executive summary, exam, flow chart, group discussion, instruction manual, inventory, lab notes, letter to the editor, matching test, mathematical problem, memo, micro theme, multiple choice test, narrative, news story, notes, oral report, outline, performance review, plan, presentation, process analysis, proposal, regulation, research proposal, review of literature, taxonomy, technical report, term paper, thesis, word problem, work of art. (Walvoord Anderson 1998).

12. What an Assessment Plan looks like

44 Mission General Education Program Outacomes Vision Core Competencies External Agencies Outcome

Components Supporting Experiences Learning Object/s Assessment Tool Mission Outcome

Components Supporting Experiences Learning Object/s Assessment Tool Mission Conduct Independent research Problem significance, previous research, methodology, analysis, interpretation, implications 195, 282, 387, 388, 485 Capstone project in 485 Research Project Rubric

Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Present Academic Research Introduction, organization, delivery, visual support, explanation, interaction with audience 191, 282, 387, 389, 485 Capstone presentation in 485 Presentation Rubric, Audience Feedback form

Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Propose new solution to ongoing problem Problem, causes, critique of current solutions, alternative solution analysis 282, 366, 381, 365 Analysis of Environmental Issues presentation Proposal rubric

Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Analyze Data data processing, statistical analysis, graphic representation, explanation 182, 196, 288, 425 Test in 425 data analysis evaluation scale

Outcome Components Supporting Experiences Learning Object/s Assessment Tool Mission Create a Lesson Plan Objectives, teaching methods, modeling, comprehension check, practice, assessment, materials , 182, 196, 252, 268, 324, 325, 350, 355, 425 Portfolio Presentation of lesson plan Lesson plan rubric Observational feedback scale

Assessing Program Level Student Learning Outcomes: What you need to know University of Wisconsin Parkside January 20, 2012 Susan Hatfield Winona State University