Director of Assessment Rose-Hulman Institute of Technology Leading the Revision of Campus-Level Student Learning Outcomes: Obstacles and Accomplishments Tony Ribera, PhD Director of Assessment Rose-Hulman Institute of Technology

Overview Institutional Context Areas in Need of Attention Committee 2017-2018 Goal and Actions Obstacles and Accomplishments Questions and Insights

What is the status of your campus-level student learning outcomes efforts? Recently revised them Recently created them Currently revising them Currently creating them

About Rose-Hulman Institute of Technology The mission of Rose-Hulman is to provide our students with the world's best undergraduate science, engineering, and mathematics education in an environment of individual attention and support. Appx. 2200 undergraduate students and 100 graduate students Student to Faculty ratio: 11:1 Average class size: 20 Over 90 student clubs and organizations

Commission on the Assessment of Student Outcomes (CASO) Founded in 1996 Responsible for campus-level student learning outcomes and campus- level assessment plan Campus-level student learning outcomes first approved in 1998 Last major revision in 2007 Responsible for KEEN outcomes and KEEN assessment plan CASO spent the past two academic years working on criteria and testing rubrics using student artifacts from KEEN courses.

Institutional Student Learning Outcomes & Performance Criteria 6 Domain Student Learning Outcome Performance Criteria Technical Skills Outcomes developed by each department Mapped to specific ABET criteria (engineering programs) Professional Skills Leadership Levels A – C indicate the level of performance achieved by each student Teamwork Communication World Citizenship Cultural & Global Awareness Ethics Service 6

Sample Outcome/Criterion

RosEvaluation Annual (Summer) Faculty raters Inter-rater reliability Rubrics/Rating Scales AAC&U Written Communication VALUE Rubric Teamwork Rubric Pass/Fail Institutional Outcomes KEEN Outcomes

Assessment Institute Indianapolis, IN

Transparency Framework National Institute for Learning Outcomes Assessment. (2011). Transparency Framework. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Retrieved from: http://www.learningoutcomesassessment.org/TransparencyFramework.htm

Assessment at Rose-Hulman Campus Program Course

Assessment at Rose-Hulman Evidence from student work is foundation Assess campus-level outcomes in RosEvaluation Teamwork Communication Cultural and Global Awareness Ethics Areas in need of attention Leadership & Service? Teamwork & Communication criteria? Co-curriculum Campus Program Course

Assessment at Rose-Hulman Other areas in need of attention Emphasis on ABET accreditation Academic program and functional area reporting Monitoring of use and campus-level use by CASO, QOE, and IRPA Assessment resources available to faculty/staff Highlight faculty/staff assessment activities and use of evidence Campus Program Course

Assessment at Rose-Hulman Other areas in need of attention Expanding raters (Staff, supervisors) Expanding activities (internships, study abroad) Providing feedback and opportunities for intervention Campus Program Course

Think-Pair-Share Thinking about campus-level learning outcomes and assessment at your institution, what are some strengths and areas in need of attention?

2017-2018 Goal By the end of the 2017-2018 academic year, CASO will submit a draft of new campus-level student learning outcomes.

Understanding the current landscape/best practices Articulating specific and measurable learning outcomes Identifying characteristics of the learning environment and student artifacts Incorporating Internal and External Feedback Incorporating Internal and External Feedback

Meeting 1 Discuss CASO responsibilities and 2017-2018 charge Fall Quarter Week Meetings/Activities FALL 1 Meeting 1 Discuss CASO responsibilities and 2017-2018 charge 2 3 Meeting 2 Discuss feedback from scholars; Review other campus-level learning outcomes 4 5 Meeting 3 Review our campus-level learning outcomes, VALUE Rubrics, and ABET criteria 6 7 Meeting 4 Discuss feedback from internal stakeholders (e.g., VPAA, CCPEO) 8 9 Meeting 5 Finalize plan for collecting feedback and articulating learning outcomes 10 EXAMS OFF

Team Assignments 4 teams Focus of Winter Quarter: Articulating specific and measureable learning outcomes Emphasis on higher-order thinking/deep learning Distinguishing various levels (A, B, …) Focus of Spring Quarter: Revising outcomes; Identifying learning opportunities in courses and co-curriculum; Identifying course and co- curricular artifacts

Team Assignments TEAM 1 TEAM 2 TEAM 3 TEAM 4 Winter Meetings 1 & 2 Creative & Critical Thinkers Investigators Intercultural & Global Learners Ethical Professionals Meetings 3 & 4 Problem Solvers Lifelong Learners Communicators Team Members & Leaders Spring

Feedback Meeting with VPAA Meeting with Department Heads Findings from Campus Community Report Findings from Employer Report Findings from Departmental Employer Surveys Communication with Internal and External Experts

Obstacles Existing outcomes/process Projects with conflicting goals Engaging students Perceptions of outcomes and assessment “A bureaucratic behemoth” “Invaded the classroom” “Bureaucratic beast”

Accomplishments Member buy-in Using existing data Collecting and considering feedback Flexible/Staying on task

Questions and Insights? Tony Ribera, PhD Director of Assessment Rose-Hulman Institute of Technology 812.877.8632 ribera@rose-hulman.edu