WIOA Accountability Ben Konruff WTCS Education Director - Performance Analysis & Continuous Improvement

Session Overview Agenda: WIOA Accountability 101 Game Department of Education Negotiations Process Reporting Integrated Education and Training Program Participation Provider Participant Scorecards Framework for Effective Data Use

WIOA Accountability 101 Game Form a team After each question: Discuss your team response; Record the response on the activity handout; Review correct answer together

WIOA Accountability 101 Game - Q#1 What are the current WIOA performance indicators? Measurable Skill Gains, Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, Credential Attainment Rate, and Effectiveness in Serving Employers Educational Functioning Level Gains, Entered Employment in 1st Quarter After Exit, Retained Employment in 3rd Quarter After Exit, Entered Postsecondary Education or Training, and Obtained a GED or Secondary School Diploma Successful Course Completion, Transfer to a Four-Year College, % Completing a Post-test, and Completing 12 Hours of Service There are no WIOA indicators of performance

WIOA Accountability 101 Game - Q#1 What are the current WIOA performance indicators? Measurable Skill Gains, Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, Credential Attainment Rate, and Effectiveness in Serving Employers Educational Functioning Level Gains, Entered Employment in 1st Quarter After Exit, Retained Employment in 3rd Quarter After Exit, Entered Postsecondary Education or Training, and Obtained a GED or Secondary School Diploma Successful Course Completion, Transfer to a Four-Year College, % Completing a Post-test, and Completing 12 Hours of Service There are no WIOA indicators of performance

WIOA Accountability 101 Game - Q#1

WIOA Accountability 101 Game - Q#2 Under WIOA, Wisconsin has moved towards a shared accountability system across all WIOA Core Programs. True False

WIOA Accountability 101 Game - Q#2 Under WIOA, Wisconsin has moved towards a shared accountability system across all WIOA Core Programs. True False

WIOA Accountability 101 Game - Q#2 The success of one is the success of all! Title 1B – Adult Program Title 1B – Dislocated Work Program Title 1B – Youth Program Title 2 – Adult Education Family Literacy Act Title 3 – Wagner-Peyser Employment Services Title 4 – Vocational Rehabilitation Developing collaborative cross-title partnerships is key! _ How can we share services to support success?

WIOA Accountability 101 Game - Q#3 What classifies a student as a participant under WIOA? Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 1 contact hour of service. Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service. The student must also have a reported name, gender, and race/ethnicity. Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service. Enrolled in an AEFLA program i.e. have a grant record.

WIOA Accountability 101 Game - Q#3 What classifies a student as a participant under WIOA? Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 1 contact hour of service. Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service. The student must also have a reported name, gender, and race/ethnicity. Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service. Enrolled in an AEFLA program i.e. have a grant record.

WIOA Accountability 101 Game - Q#4 The WTCS has a pre/post test goal (e.g. 67% of participants who complete a pre test will also complete a post test). True False

WIOA Accountability 101 Game - Q#4 The WTCS has a pre/post test goal (e.g. 67% of participants who complete a pre test will also complete a post test). True False

WIOA Accountability 101 Game - Q#4 The Department of Education requires that we have an Assessment Policy; we establish a pre/post test goal within our WTCS Assessment Policy 2016-17 Goal – 67% 2017-18 Goal – 75% Why is our pre/post test goal important? How does this goal relate to the WIOA performance indicators?

WIOA Accountability 101 Game - Q#5 What are different ways Title 2 (AEFLA) can attain Measurable Skill Gains? Pre/Post Test Educational Functioning Level Gain Only Transition to Employment, Complete a Pre Test, and Complete 12 Hours of Service Pre/Post Test Educational Functioning Level Gain, Exit Program and Enter Postsecondary in the Program Year, Obtain a Secondary School Diploma or Equivalent, and Awarding of Carnegie Units towards a Credential (not applied to WTCS) Each WTCS provider gets to decide

WIOA Accountability 101 Game - Q#5 What are different ways Title 2 (AEFLA) can attain Measurable Skill Gains? Pre/Post Test Educational Functioning Level Gain Only Transition to Employment, Complete a Pre Test, and Complete 12 Hours of Service Pre/Post Test Educational Functioning Level Gain, Exit Program and Enter Postsecondary in the Program Year, Obtain a Secondary School Diploma or Equivalent, and Awarding of Carnegie Units towards a Credential (not applied to WTCS) Each WTCS provider gets to decide

WIOA Accountability 101 Game - Q#6 Reportable individuals (i.e. students who have completed less than 12 contact hours of service) are counted in the WIOA performance indicators. True False

WIOA Accountability 101 Game - Q#6 Reportable individuals (i.e. students who have completed less than 12 contact hours of service) are counted in the WIOA performance indicators. True False Reported individuals should still be reported to the WTCS System Office via Client Reporting or the CBO Portal!

WIOA Accountability 101 Game - Q#7 The WTCS System Office has established data sharing agreements with external organizations to support the data requirements in calculating the WIOA performance indicators. True False

WIOA Accountability 101 Game - Q#7 The WTCS System Office has established data sharing agreements with external organizations to support the data requirements in calculating the WIOA performance indicators. True False

WIOA Accountability 101 Game - Q#7 Existing data sharing agreements Colleges and CBOs National Student Clearinghouse Department of Workforce Development UI Wage System Department of Public Instruction

WIOA Accountability 101 Game - Q#8 Participants without a reported Social Security Number are NOT included in the employment measures that require data matches with external organizations. Employment measures include Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, and part of the Credential Attainment Rate (participants who obtain a secondary credential and are employed within 1 year after exit). True False

WIOA Accountability 101 Game - Q#8 Participants without a reported Social Security Number are NOT included in the employment measures that require data matches with external organizations. Employment measures include Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, and part of the Credential Attainment Rate (participants who obtain a secondary credential and are employed within 1 year after exit). True False

WIOA Accountability 101 Game - Q#8 Employment data matches are done with the Department of Workforce Development’s UI Wage System Matches are made via Social Security Number A Social Security Number is not required to receive services! It is encouraged that you prompt for collection.

WIOA Accountability 101 Game - Q#9 How many days must elapse between program exit and program reentry before a new period of participation begins? 120 Days 30 Days 90 Days 10 Days

WIOA Accountability 101 Game - Q#9 How many days must elapse between program exit and program reentry before a new period of participation begins? 120 Days 30 Days 90 Days 10 Days

WIOA Accountability 101 Game - Q#9 Each period of participation is counted seperately in the WIOA performance indicators

WIOA Accountability 101 Game - Q#10 What are the two circumstances that can lead to financial sanctions under WIOA (i.e. 5% $ reduction)? Failure to submit the State annual performance report by the Wisconsin Titles including the WTCS System Office Failure by the State to meet the required levels of performance for the WIOA performance indicators Both A and B Financial sanctions don’t exist under WIOA

WIOA Accountability 101 Game - Q#10 What are the two circumstances that can lead to financial sanctions under WIOA (i.e. 5% $ reduction)? Failure to submit the State annual performance report by the Wisconsin Titles including the WTCS System Office Failure by the State to meet the required levels of performance for the WIOA performance indicators Both A and B Financial sanctions don’t exist under WIOA

Negotiations with Department of Education

WIOA Negotiations with DoE Only WIOA Measurable Skill Gain (MSG) was negotiated for year 2018-19 and 2019-20 Considerations during negotiations Promote continuous improvement Government and Performance Results Act Targets 3. Other state/area comparisons 4. Statistical adjustment model

2016-2017 MSG Negotiated Levels WIOA Negotiations – Continuous Improvement 2016-2017 MSG Negotiated Levels 2016-17 MSG Actual Levels Difference 90% Threshold 90% Met? 43% 51.9% 8.9% 39% YES

WIOA Negotiations – Government & Performance Results Act Target

WIOA Negotiations – Other State/Area Comparisons

WTCS Negotiated Levels of Performance

90% Requirement 52% * 90% = 46.8% for 2018-19 53% * 90% = 47.7% for 2019-20

2017-18 is the first year of WTCS data collection Reporting Integrated Ed and Training 2017-18 is the first year of WTCS data collection Reported on the S4 or CBO Portal in the Services Provided Through Grant Activity Code 34 – Integrated English Literacy and Civics Education 35 – Adult Basic Education Integrated Education and Training 36 – Adult Secondary Integrated Education and Training 37 – English as a Second Language Education Integrated Education and Training 38 – Integrated English Literacy and Civic Education Integrated Education and Training Program Reference the Client Reporting Manual for definitions!

Provider Scorecards

Participant Characteristics and Demographics What strikes you about the participants served at your organization? How does this compare to the WTCS?

Did your organization reach the WTCS Goal for % Pre and Post Tested? Participant Pre and Post Test Achievement Did your organization reach the WTCS Goal for % Pre and Post Tested? Are there any clear gaps when comparing ABE to ELL, EFLs, or student characteristics? What could be done so your organization reaches the WTCS goal or hypothesize what appears to be working if you already met the goal?

Did your organization reach the WTCS Goal for % Measurable Skill Gain? Participant MSG Achievement Did your organization reach the WTCS Goal for % Measurable Skill Gain? Are there any clear gaps when comparing ABE to ELL, EFLs, or student characteristics? What could be done so your organization reaches the WTCS goal or hypothesize what appears to be working if you already met the goal?

Ben Konruff Education Director – Performance Analysis and Continuous Improvement Wisconsin Technical College System Tel: 608-266-1840 E-mail: ben.konruff@wtcsystem.edu