1. WIOA Accountability Ben Konruff
WTCS Education Director - Performance Analysis & Continuous Improvement

2. Session Overview Agenda: WIOA Accountability 101 Game
Department of Education Negotiations Process
Reporting Integrated Education and Training Program Participation
Provider Participant Scorecards
Framework for Effective Data Use

3. WIOA Accountability 101 Game
Form a team
After each question:
Discuss your team response;
Record the response on the activity handout;
Review correct answer together

4. WIOA Accountability 101 Game - Q#1
What are the current WIOA performance indicators?
Measurable Skill Gains, Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, Credential Attainment Rate, and Effectiveness in Serving Employers
Educational Functioning Level Gains, Entered Employment in 1st Quarter After Exit, Retained Employment in 3rd Quarter After Exit, Entered Postsecondary Education or Training, and Obtained a GED or Secondary School Diploma
Successful Course Completion, Transfer to a Four-Year College, % Completing a Post-test, and Completing 12 Hours of Service
There are no WIOA indicators of performance

5. WIOA Accountability 101 Game - Q#1
What are the current WIOA performance indicators?
Measurable Skill Gains, Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, Credential Attainment Rate, and Effectiveness in Serving Employers
Educational Functioning Level Gains, Entered Employment in 1st Quarter After Exit, Retained Employment in 3rd Quarter After Exit, Entered Postsecondary Education or Training, and Obtained a GED or Secondary School Diploma
Successful Course Completion, Transfer to a Four-Year College, % Completing a Post-test, and Completing 12 Hours of Service
There are no WIOA indicators of performance

6. WIOA Accountability 101 Game - Q#1

7. WIOA Accountability 101 Game - Q#2
Under WIOA, Wisconsin has moved towards a shared accountability system across all WIOA Core Programs.
True
False

8. WIOA Accountability 101 Game - Q#2
Under WIOA, Wisconsin has moved towards a shared accountability system across all WIOA Core Programs.
True
False

9. WIOA Accountability 101 Game - Q#2
The success of one is the success of all!
Title 1B – Adult Program
Title 1B – Dislocated Work Program
Title 1B – Youth Program
Title 2 – Adult Education Family Literacy Act
Title 3 – Wagner-Peyser Employment Services
Title 4 – Vocational Rehabilitation
Developing collaborative cross-title partnerships is key!
_ How can we share services to support success?

10. WIOA Accountability 101 Game - Q#3
What classifies a student as a participant under WIOA?
Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 1 contact hour of service.
Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service. The student must also have a reported name, gender, and race/ethnicity.
Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service.
Enrolled in an AEFLA program i.e. have a grant record.

11. WIOA Accountability 101 Game - Q#3
What classifies a student as a participant under WIOA?
Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 1 contact hour of service.
Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service. The student must also have a reported name, gender, and race/ethnicity.
Must be 18 years of age or older, enrolled in an AEFLA program i.e. have a grant record, and have completed at least 12 contact hours of service.
Enrolled in an AEFLA program i.e. have a grant record.

12. WIOA Accountability 101 Game - Q#4
The WTCS has a pre/post test goal (e.g. 67% of participants who complete a pre test will also complete a post test).
True
False

13. WIOA Accountability 101 Game - Q#4
The WTCS has a pre/post test goal (e.g. 67% of participants who complete a pre test will also complete a post test).
True
False

14. WIOA Accountability 101 Game - Q#4
The Department of Education requires that we have an Assessment Policy; we establish a pre/post test goal within our WTCS Assessment Policy
Goal – 67%
Goal – 75%
Why is our pre/post test goal important? How does this goal relate to the WIOA performance indicators?

15. WIOA Accountability 101 Game - Q#5
What are different ways Title 2 (AEFLA) can attain Measurable Skill Gains?
Pre/Post Test Educational Functioning Level Gain Only
Transition to Employment, Complete a Pre Test, and Complete 12 Hours of Service
Pre/Post Test Educational Functioning Level Gain, Exit Program and Enter Postsecondary in the Program Year, Obtain a Secondary School Diploma or Equivalent, and Awarding of Carnegie Units towards a Credential (not applied to WTCS)
Each WTCS provider gets to decide

16. WIOA Accountability 101 Game - Q#5
What are different ways Title 2 (AEFLA) can attain Measurable Skill Gains?
Pre/Post Test Educational Functioning Level Gain Only
Transition to Employment, Complete a Pre Test, and Complete 12 Hours of Service
Pre/Post Test Educational Functioning Level Gain, Exit Program and Enter Postsecondary in the Program Year, Obtain a Secondary School Diploma or Equivalent, and Awarding of Carnegie Units towards a Credential (not applied to WTCS)
Each WTCS provider gets to decide

17. WIOA Accountability 101 Game - Q#6
Reportable individuals (i.e. students who have completed less than 12 contact hours of service) are counted in the WIOA performance indicators.
True
False

18. WIOA Accountability 101 Game - Q#6
Reportable individuals (i.e. students who have completed less than 12 contact hours of service) are counted in the WIOA performance indicators.
True
False
Reported individuals should still be reported to the WTCS System Office via Client Reporting or the CBO Portal!

19. WIOA Accountability 101 Game - Q#7
The WTCS System Office has established data sharing agreements with external organizations to support the data requirements in calculating the WIOA performance indicators.
True
False

20. WIOA Accountability 101 Game - Q#7
The WTCS System Office has established data sharing agreements with external organizations to support the data requirements in calculating the WIOA performance indicators.
True
False

21. WIOA Accountability 101 Game - Q#7
Existing data sharing agreements
Colleges and CBOs
National Student Clearinghouse
Department of Workforce Development UI Wage System
Department of Public Instruction

22. WIOA Accountability 101 Game - Q#8
Participants without a reported Social Security Number are NOT included in the employment measures that require data matches with external organizations. Employment measures include Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, and part of the Credential Attainment Rate (participants who obtain a secondary credential and are employed within 1 year after exit).
True
False

23. WIOA Accountability 101 Game - Q#8
Participants without a reported Social Security Number are NOT included in the employment measures that require data matches with external organizations. Employment measures include Employment in 2nd Quarter After Exit, Employment in 4th Quarter After Exit, Median Earnings in 2nd Quarter After Exit, and part of the Credential Attainment Rate (participants who obtain a secondary credential and are employed within 1 year after exit).
True
False

24. WIOA Accountability 101 Game - Q#8
Employment data matches are done with the Department of Workforce Development’s UI Wage System
Matches are made via Social Security Number
A Social Security Number is not required to receive services! It is encouraged that you prompt for collection.

25. WIOA Accountability 101 Game - Q#9
How many days must elapse between program exit and program reentry before a new period of participation begins?
120 Days
30 Days
90 Days
10 Days

26. WIOA Accountability 101 Game - Q#9
How many days must elapse between program exit and program reentry before a new period of participation begins?
120 Days
30 Days
90 Days
10 Days

27. WIOA Accountability 101 Game - Q#9
Each period of participation is counted seperately in the WIOA performance indicators

28. WIOA Accountability 101 Game - Q#10
What are the two circumstances that can lead to financial sanctions under WIOA (i.e. 5% $ reduction)?
Failure to submit the State annual performance report by the Wisconsin Titles including the WTCS System Office
Failure by the State to meet the required levels of performance for the WIOA performance indicators
Both A and B
Financial sanctions don’t exist under WIOA

29. WIOA Accountability 101 Game - Q#10
What are the two circumstances that can lead to financial sanctions under WIOA (i.e. 5% $ reduction)?
Failure to submit the State annual performance report by the Wisconsin Titles including the WTCS System Office
Failure by the State to meet the required levels of performance for the WIOA performance indicators
Both A and B
Financial sanctions don’t exist under WIOA

30. Negotiations with Department of Education

31. WIOA Negotiations with DoE
Only WIOA Measurable Skill Gain (MSG) was negotiated for year and
Considerations during negotiations
Promote continuous improvement
Government and Performance Results Act Targets
3. Other state/area comparisons
4. Statistical adjustment model

32. 2016-2017 MSG Negotiated Levels
WIOA Negotiations – Continuous Improvement
MSG Negotiated Levels
MSG Actual Levels
Difference
90% Threshold
90% Met?
43%
51.9%
8.9%
39%
YES

33. WIOA Negotiations – Government & Performance Results Act Target

34. WIOA Negotiations – Other State/Area Comparisons

35. WTCS Negotiated Levels of Performance

36. 90% Requirement
52% * 90% = 46.8% for
53% * 90% = 47.7% for

37. 2017-18 is the first year of WTCS data collection
Reporting Integrated Ed and Training
is the first year of WTCS data collection
Reported on the S4 or CBO Portal in the Services Provided Through Grant Activity Code
34 – Integrated English Literacy and Civics Education
35 – Adult Basic Education Integrated Education and Training
36 – Adult Secondary Integrated Education and Training
37 – English as a Second Language Education Integrated Education and Training
38 – Integrated English Literacy and Civic Education Integrated Education and Training Program
Reference the Client Reporting Manual for definitions!

38. Provider Scorecards

39. Participant Characteristics and Demographics
What strikes you about the participants served at your organization?
How does this compare to the WTCS?

40. Did your organization reach the WTCS Goal for % Pre and Post Tested?
Participant Pre and Post Test Achievement
Did your organization reach the WTCS Goal for % Pre and Post Tested?
Are there any clear gaps when comparing ABE to ELL, EFLs, or student characteristics?
What could be done so your organization reaches the WTCS goal or hypothesize what appears to be working if you already met the goal?

41. Did your organization reach the WTCS Goal for % Measurable Skill Gain?
Participant MSG Achievement
Did your organization reach the WTCS Goal for % Measurable Skill Gain?
Are there any clear gaps when comparing ABE to ELL, EFLs, or student characteristics?
What could be done so your organization reaches the WTCS goal or hypothesize what appears to be working if you already met the goal?

43. Ben Konruff
Education Director – Performance Analysis and Continuous Improvement
Wisconsin Technical College System
Tel: