Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone 1,000 Students a Year: Assessing Core Skills      1,000 Students a Year: Assessing Core Skills  Madison College assesses 1,000 students a year in skills that local employers find valuable. Discover how the college 1) captures industry need, 2) secures faculty buy-in, and 3) manages results. Review instructional improvement plans designed around the findings. With the goal to assess 1,000 students a year in skills that local employers find important, the college completed its second round of data collection in core workforce skills assessment. Branded as an action research project, over 750 faculty are invited annually to participate. View summaries of participation, strategies used, and percent met/not met/not assessed. Explore how the college integrates this research into the Faculty Quality Assurance System, which reaches all full and part-time faculty. 2015: 399 full and part time faculty invited School of Arts and Sciences with select programs. Response rate: 30/399=7.5% 2016: 475 full time faculty invited. Response rate: 48/475=10% 2017: All full and part time faculty in all Schools WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Innovations Conference March 13, 2017; 8-9 am San Francisco, CA Robin Nickel, Ph.D. Curriculum Consultant Madison College WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Fact: 38-42% of headcount = transfer students WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Fact: 9 campuses serving 12 counties WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Capturing Industry Need District-centered Quantifiable Meaningful to instructors (buy in) Alissa. Robin can speak to history in trying to build in core workforce skills, assess them, and prove their value. This goes back to the 1991 SCANS report and also the more recent Skills for the 21st Century. In 2013 we asked local employers to tell us what skills were important when hiring Madison College graduates (252 survey completers representing 28 different market segments locally). In response, we prioritized these skills and created a way to assess them using concrete student performance as evidence. In 2017 we repeated the survey. WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Alissa. Robin can speak to history in trying to build in core workforce skills, assess them, and prove their value. This goes back to the 1991 SCANS report and also the more recent Skills for the 21st Century. In 2013 we asked local employers to tell us what skills were important when hiring Madison College graduates (252 survey completers representing 28 different market segments locally). In response, we prioritized these skills and created a way to assess them using concrete student performance as evidence. In 2017 we repeated the survey. WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone http://youtu.be/B3Y1HQdcWXY Robin 2015: 399 full and part time faculty invited School of Arts and Sciences with select programs. Response rate: 30/399=7.5% 2016: 475 full time faculty invited. Response rate: 48/475=10% 2017: All full and part time faculty in all Schools R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Longitudinal Industry Survey Objectives Measure gaps in importance versus overall satisfaction of soft skills by employers Document important core skills that all students should learn during their experience at Madison College © 2017 Madison College. A. Tambone. Alissa WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Objectives Provide Board of Trustees insights on how employers in the district view Madison College graduates Determine trends in industry needs in the next 3 to 5 years © 2017 Madison College. A. Tambone. Alissa WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone 2017 Survey Invitations Sent to 6,011 employers Hiring managers and human resource professionals on 1/23 Reminder notice sent on 1/26 Closure notice on 2/2-open until 2/6 Alissa WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone 833 survey respondents (13.9%) Completion rate of 80% Confidence level of 99% (+/- 4.15) Response rates varied by question Alissa WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Alissa © 2017 Madison College. A. Tambone. WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Alissa A. Tambone, IRE WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone How did you find the Madison College graduates you hired? Job Posting Reply 59% Word of Mouth/Referral 49% Wisconsin TechConnect 24% In-person Networking 22% Job Center 13% Social Media 11% Online Resume Search Engine 9% Wisconsin Department of Workforce Development 8% Employment Agency 4% Other 19% Alissa © 2017 Madison College. A. Tambone. WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Alissa © 2017 Madison College. A. Tambone. WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Alissa 2017 A. Tambone, IRE WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Alissa 2017 A. Tambone, IRE WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Would you consider hiring Madison College graduates in the next 1, 3 or 5 years? Alissa A. Tambone, IRE WTCS Assessment Conference, La Crosse, WI 3/2/17

2017 Employer Important vs. Satisfaction Importance Rank 2017 Employer Important vs. Satisfaction Gaps Analysis (n=430; 417) 2014   2014 Employer Important vs. Satisfaction (n=276; 262) Change in Gaps of Importance vs. Satisfaction from 2014 to 2017 5. Self-Management 30% 34% 4% decrease 3. Critical Thinking 26% 2. Critical Thinking 4. Social Interaction 23% 3. Social Interaction 28% 5% decrease 2. Ethics 20% 4. Ethics 22% 2% decrease 1. Communication-Listening 19% 3% decrease 6. Communication-Speaking 15% 7. Communication-Reading 12% 8. Communication-Writing 8% 9. Communication-Writing 0% 9. Technical Skills 6% 8. Technical Skill Set 6% decrease 10. Global & Cultural Perspectives 3% 10. Global & Cultural Perspective -1% 4% increase

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone The Rubric Ratings 5. Self-Management M NM NA 3. Critical Thinking 4. Social Interaction 2. Ethics 1. Communication-Listening 6. Communication-Speaking 7. Communication-Reading 8. Communication-Writing Alissa…. Findings validate continued use of the Universal Rubric for fall 2017 student assessment. Robin .. Talk about the rubric. A. Tambone, IRE WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Faculty Invitation Process Robin 2015: 399 full and part time faculty invited School of Arts and Sciences with select programs. Response rate: 30/399=7.5% 2016: 475 full time faculty invited. Response rate: 48/475=10% 2017: All full and part time faculty in all Schools WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Incremental Roll-out 2015 Subgroup=full and part time (30/399) Testing of rubric and process 7.5% response 2016 All full-time (48/475) 10% response 2017 All full and part time (750+) Robin 2015: 399 full and part time faculty invited School of Arts and Sciences with select programs. Response rate: 30/399=7.5% 2016: 475 full time faculty invited. Response rate: 48/475=10% 2017: All full and part time faculty in all Schools R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Robin 2015: 399 full and part time faculty invited School of Arts and Sciences with select programs. Response rate: 30/399=7.5% 2016: 475 full time faculty invited. Response rate: 48/475=10% 2017: All full and part time faculty in all Schools R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Robin 2015: 399 full and part time faculty invited School of Arts and Sciences with select programs. Response rate: 30/399=7.5% 2016: 475 full time faculty invited. Response rate: 48/475=10% 2017: All full and part time faculty in all Schools R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Criteria 13 RATERS Ratings Artifact 1 Artifact 2 Artifact 3 Total A.    SELF MANAGEMENT (manage time and stress, practice workplace etiquette, self-assess) M 13 11 6 30 NM NA 2 7 9 B.    CRITICAL THINKING (evaluate, solve problems, make decisions) 12 38 1 C.   SOCIAL INTERACTION (respond to feedback, respect diversity, work effectively in a team) 24 D.   COMMUNICATION (listen and respond effectively) E.    COMMUNICATION (speak so others can understand, convey meaning) 10 3 27 F.    COMMUNICATION (read and comprehend) 26 4 G.   COMMUNICATION (write so others can understand, convey meaning) 35 H.   ETHICS (demonstrate acceptable behavior and judgment) 29 Robin 2015: 399 full and part time faculty invited School of Arts and Sciences with select programs. Response rate: 30/399=7.5% 2016: 475 full time faculty invited. Response rate: 48/475=10% 2017: All full and part time faculty in all Schools R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Faculty prefer Excel, copy, paste and email. A universal rubric was used to assess skills with highest gaps. Student-generated assignments were assessed across multiple programs and disciplines. **SAVE TO CONFERENCE DEVICE R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Project Management Faculty prefer Excel, copy, paste and email. A universal rubric was used to assess skills with highest gaps. Student-generated assignments were assessed across multiple programs and disciplines. **SAVE TO CONFERENCE DEVICE R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone 2015 and 2016 combined: n=79 instructors n=79 R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone 2015 and 2016 combined Years 1 and 2 n=1975 R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone **NEED 2015 and 2016 combined. 2015=33 courses 2016=65 courses Year 2 n=65 R. Nickel WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Year 1: A&S and target program for testing of rubric and process Year 2: All full-time faculty invited Year 3: All faculty invited Successes: increase in participation “Speaking” for example. How do we integrate this into automotive? If we do, how are we assessing it? Intranet location for additional documentation: https://facstaff.madisoncollege.edu/in/4-redefine-reimagine-core-workforce-skills 98 different course numbers 1,975 students 79 instructors WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Communicating Results to Community Executive Summary Integrated into Faculty Quality Assurance System professional development Center for Entrepreneurship Robin WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Future Research Do we have evidence that assessed students are more successful in college and in their respective employment upon graduation? How are core skill needs evolving over time, as indicated by industry insights via the survey? Is the gap between Madison College employer importance and satisfaction narrowing over time? Robin Future 10 years and relationship to accreditation. GENERAL WTCS Assessment Conference, La Crosse, WI 3/2/17

Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Lessons Learned Avoid collecting to much data too often Do not purchase a new system unless it’s tested by faculty Avoid too many performance indicators Avoid complicated point spreads over rubric(s) Don’t assume an LMS is needed Robin WTCS Assessment Conference, La Crosse, WI 3/2/17

Thank You rnickel@madisoncollege.edu Soft Skill Assessment Through Action Research Madison College Presentation by R. Nickel and A. Tambone Thank You rnickel@madisoncollege.edu WTCS Assessment Conference, La Crosse, WI 3/2/17