1. WIOA Performance Workgroup
2/1/2017

2. Effectiveness in Serving Employers Policy Priorities ETP ICR
Agenda
Effectiveness in Serving Employers
Policy Priorities
ETP ICR

3. Policies/Procedures Needed based on TEGL
Performance Measures (TEGL 10-16)
Negotiations and Goals
Common Exit Policy (page 34/35)
Effectiveness on Serving Employers, which 2 for MD (page 28)
All needed before annual is submitted for PY16, negotiation’s needed now

4. Effectiveness In Serving Employers
Departments have determined that this indicator will be measured as a shared outcome across all six core programs within each State to ensure a holistic approach to serving employers.
The Departments are implementing this indicator in the form of a pilot program to test the rigor and feasibility of three approaches, and to subsequently set a standardized indicator.
This indicator is reported on an annual basis; therefore, the reporting period for the effectiveness in serving employers indicator is the program year
States must select two of these three approaches to report on this indicator.
Governors also may establish and report on a third State-specific approach for measuring effectiveness, in addition to the two Departmental approaches selected.
The Departments will evaluate State experiences with the various approaches and plan to identify a standardized indicator that the Departments anticipate will be implemented no later than the beginning of Program Year 2019.

5. Effectiveness In Serving Employers
Employer Penetration Rate- Record the total number of establishments, as defined by the Bureau of Labor Statistics Quarterly Census of Earnings and Wages program, that received a service or, if it is an ongoing activity, are continuing to receive a service or other assistance during the reporting period. DIVIDED By Record the total number of establishments, as defined by the Bureau of Labor Statistics Quarterly Census of Earnings and Wages program, located within the State during the final month or quarter of the reporting period.
Repeat Business Customers- Record the total number of establishments, as defined by the Bureau of Labor Statistics Quarterly Census of Earnings and Wages program, that received a service or, if it is an ongoing activity, are continuing to receive a service or other assistance during the reporting period (E1), AND who utilized a service anytime within the previous three years (E4). DIVIDED By Record the number of unique business customers (establishments - as defined by the Bureau of Labor Statistics Quarterly Census of Earnings and Wages program) who have received a service previously in the last three years.
Retention with Same Employer in the 2nd and 4th Quarters After Exit, looks at FEIN data to determine if yes

6. Step 1- Which measures can we collect and how
What’s Next
Step 1- Which measures can we collect and how
Step 2- Which measures allow us to measure effectiveness to its true meaning for MD

7. Data Collections Options/Ideas- for 2 of the 3 measures
Option 1: let DORS into MWE with confidentiality agreements to serve employers and add services
Option 2: have DORs work with BRR and BSRs to add the employer and services they worked with in MWE
Option 3: data sharing and agreements, especially if different system, figure a way to share the data
Any other options?

8. Data Sharing for Last Measure
TEGLs states DORS supplies data to SWA for data combination for Retention With Same Employer, page 26
DLLR and DORs met Jan 25
Penetration rate 2.8
Repeat 18.2
Retention- Workforce solely (due to data confidentiality) 55%

9. Discuss Measures for MD
Which measures do we like and why??
Do we want a measure that shows MD as a higher or lower outcome?
Note 2 measures will not have all the data for the PY end for submittal.

10. Training
February 24, 2017 at Columbia

11. Performance Questions
ETP ICR

12. Comments
For Data Element 101, Type of Entity, duplicative code values are provided which is confusing. Community colleges can be appropriate recorded as code value 1, as the majority of the credentials awarded are associate’s degrees. Community Colleges can also be appropriately recorded as code 3, as the majority of the credentials awarded are a community college certificate of completion. This situation expectedly will lead to inconsistent reporting by states.
In addition, common occupational training providers are not specifically included among those listed for this data element, and their contribution to workforce development in this country should be identified and reported. Two such providers are private vocational/career schools, two-year private/technical schools.
Also, WIOA makes allowance for secondary schools as potential eligible training providers when their diplomas are offered in conjunction with occupational skills training (et al), but they are not specifically captured via the code values.
Below is a proposed revision to this data element that accounts for the above-cited situations.

13. Some feel it is confusing, agree or disagreed, like or dislike
Comments
Data Element 101: Type of Entity (reference: 20 CFR )
1 = Private Vocational or Career School (awards below associate degree)
2 = Community College
3 = Two-year Private Technical School (awards associate degrees)
4 = Four-year College or University
5 = Registered Apprenticeship Sponsor/Provider
6 = Secondary School
7 = Public Adult School with Occupational Program
8 = Other Private Non-Profit Provider
9 = Other Private For-Profit Provider
10 = Other Type of Provider
Some feel it is confusing, agree or disagreed, like or dislike

14. Comments
For Data Element 103, Program of Study – by potential outcome, multiple code values may apply to any one program. Of course, this situation is not problematic if states may submit multiple code values for this data element. Otherwise, the situation is problematic. For example, a community college certificate program (or an Associate’s degree program, etc.) may also have the potential outcome of an industry-recognized certificate (or of a certification or a license recognized by the state or federal government, etc.)
Further, some programs of study are not clearly captured via the code values, e.g., private vocational/career school training programs and community college non-credit course series (“programs”) and/or credit certificate programs. These are distinct offerings which are the most commonly utilized by WIOA ITA participants in Maryland and likely many other states.
In light of the above, a proposed revision to this data element is provided below, which assumes multiple code selection but is not based upon it.

15. Data Element 103

17. Additional items in ICR
Separate Listing for WIOA ANNUAL STATEWIDE PERFORMANCE REPORT TEMPLATE (ETA-9169)
Annual Report Title 1 and Title 3 Narrative

18. WIOA ANNUAL STATEWIDE PERFORMANCE REPORT TEMPLATE (ETA-9169)
For WIOA core programs, States must report the results of primary indicators of performance set forth at section 116(b)(2)(A)(i) of WIOA: 
1. The percentage of program participants who are in unsubsidized employment during the second quarter after exit from the program.
2. The percentage of program participants who are in unsubsidized employment during the fourth quarter after exit from the program.
3. The median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program.
4. The percentage of program participants who obtained a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent during participation in or within one year after exit from the program. For those program participants who obtained a secondary school diploma or its recognized equivalent, the participant must also have obtained or retained employment or be in an education or training program leading to a recognized postsecondary credential within one year after exit from the program.
5. The percentage of program participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment and who are achieving measurable skill gains toward such a credential or employment.

19. 6. The indicator(s) of effectiveness in serving employers.
Continued
6. The indicator(s) of effectiveness in serving employers.
All of the above primary indicators of performance also will be displayed as a disaggregate value based on the barriers to employment incorporated into the definition of an “individual with a barrier to employment,” as set forth in WIOA section 3(24). The specific population disaggregation is described below in Numbers 22 through 35.

20. Participant Counts and Cost Information
Under section 116(d)(2) of WIOA, the WIOA Annual Statewide Performance Report also must include the following data:
7. Information specifying the levels of performance achieved with respect to the primary indicators of performance described in WIOA section 116(b)(2)(A) for each of the programs described in WIOA section 116(b)(3)(A)(ii) and the State adjusted levels of performance with respect to such indicators for each program.
8. Information specifying the levels of performance achieved with respect to the primary indicators of performance described in WIOA section 116(b)(2)(A) for each of the programs described in WIOA section 116(b)(3)(A)(ii) with respect to individuals with barriers to employment, disaggregated by each subpopulation of such individuals, and by race, ethnicity, sex, and age.
9. The total number of participants served by each of the core programs.
10. The number of participants who received career services during the most recent program year and the three preceding program years.
11. The number of participants who received training services during the most recent program year and the three preceding program years.

21. Cost Information 12. The amount of funds spent on career services.
13. The amount of funds spent on training services.
14. The number of participants who exited from career services, during the most recent program year and the three preceding program years.
15. The number of participants who exited from training services during the most recent program year and the three preceding program years.
16. The average cost per participant of those participants who received career services during the most recent program year and the three preceding program years.
17. The average cost per participant of those participants who received training services during the most recent program year and the three preceding program years.
18. The percentage of participants in a program authorized under WIOA title I who received training services and obtained unsubsidized employment in a field related to the training received.
19. The number of participants who are enrolled in more than one of the core programs.
20. The number of participants with barriers to employment served by each of the core programs, disaggregated by each subpopulation of such participants.
21. The percentage of the State’s annual allotment under section 132(b) that the State spent on administrative costs.

22. Barriers to Employment Data must be collected in a manner so that the results may be disaggregated by the “Barriers to Employment,” as described in section 3(24) of WIOA, for both number of participants served and performance on primary indicators, and then further disaggregated by age, race and ethnicity, and sex. Under section 3(24) of WIOA, an “individual with a barrier to employment” includes:
22. Displaced homemakers.
30. Individuals who are English language learners, individuals who have low levels of literacy, and individuals facing substantial cultural barriers.
23. Low-income individuals.
24. Indians, Alaska Natives, and Native Hawaiians, as such terms are defined in section 166 of WIOA.
31. Eligible migrant and seasonal farmworkers, as defined in section 167(i) of WIOA.
25. Individuals with disabilities, including youth who are individuals with disabilities.
32. Individuals within two years of exhausting lifetime eligibility under TANF (part A of title IV of the Social Security Act (42 U.S.C. 601 et seq.)).
26. Older individuals (55+).
27. Ex-offenders.
33. Single parents (including single pregnant women).
28. Homeless individuals (as defined in section 41403(6) of the Violence against Women Act of 1994 (42 U.S.C e–2(6))), or homeless children and youths (as defined in section 725(2) of the McKinney-Vento Homeless Assistance Act (42 U.S.C a(2))).
34. Long-term unemployed individuals (27 or more consecutive weeks).
35. Such other groups as the Governor involved determines to have barriers to employment.
29. Youth who are in or have aged out of the foster care system.

23. A listing of the waivers for which the State has received approval
Narrative
A discussion of “best practices,” “lessons learned,” and “success stories” that focuses on serving individuals with barriers to employment, employers, and communities, including information based on the state's evaluation and related research projects
Discussion of progress made in achieving the State’s strategic vision and goals
Discussion of State progress in implementing sector strategies and career pathways
A listing of the waivers for which the State has received approval

24. Narrative Continue
A discussion of the activities funded by the State’s discretionary (15 percent) funds
Employer Metrics (including the metrics the State is piloting, as well as any other metrics the State is using).
Brief descriptions of: (a) current or planned evaluation and related research projects, (b) efforts to coordinate the development of such projects with local boards and other state agencies, including those for core programs, (c) completed evaluation and related reports and how they were made accessible to the public electronically, and (d) efforts to provide data, survey responses and timely site visits for Federal evaluations.
A description of any National Dislocated Worker Grants awarded to or within the State, and how those funds are coordinated with State rapid response activities, dislocated worker programs, and disaster/emergency management activities, as applicable.
Description of rapid response activities and layoff aversion (include data on number of companies served, individuals served, strategies for linking to American Job Centers, etc.)
Description of any technical assistance needs or policy challenges that most effect use of the workforce system.
Description of the State’s approach to assessing customer satisfaction, results achieved, and how the State has used the outcomes of customer satisfaction assessment to improve service delivery.

25. Annual Narrative Proposed
The information in narrative format will be helpful for the Departments to evaluate, in both a quantitative and qualitative way, the status and progress of workforce development program performance.
25 page limit
Lastly is Burden on States to Produce- By Title and Report listed

26. Example: Table 4: WIOA Annual Statewide Performance Report – Annualized Burden Hour Estimate

27. MWE System Changes
Still Coming

28. Questions