1. CWA Conference Long Beach, CA January 27, 2015 *As known today
Youth Performance Accountability Data Accuracy and Reporting for WIA and WIOA*
CWA Conference
Long Beach, CA
January 27, 2015
*As known today

2. Objective
Mutual Understanding of Data Collection-Entry-Reporting accountability from Local Areas to State to DOL.
Encourage discussion of data collection and reporting requirements, procedures, and guidance.
Establish open forums for communication for technical assistance.

3. Background
Oversight agencies like GAO and OIG cite data quality issues with ETA’s data (2002 and 2008)
Guidance issued annually containing report submission deadlines and source documentation requirements
TEGL 6-14 for PY13/FY14 Reporting and Data Validation

4. Policies/Procedures and Training
Data management and the resultant quality of reported data are derived from and influenced by the policies, procedures and protocols utilized at the state and/or local levels
Grantees should develop guidance for staff and sub-grantees involved in the collection of data:
Definitions of data elements
Sources of information
Participant record and documentation requirements
Procedures for collecting, entering and reporting data and associated “business rules” that cover timeliness and completeness
Procedures for entering data into an automated database
Procedures for correcting data

5. Training and Monitoring
Data collection and data entry:
Routine training should be provided for data management guidance
All staff involved in the collection or entry of data should be trained in the procedures
The data entry process should include steps for verifying entered data against original sources on a sample basis or for entire population of records

6. REPORTING
If Data is not collected and reported, it didn’t happen!!

7. Performance Accountability *unless State ‘early implementer’
WIA (Workforce Investment Act) effective through PY15
WIOA (Workforce Investment and Opportunity Act) effective PY16*
‘Services’ based Participation and Exit
Data Validation required
Reporting Cohort primarily 1st to 3rd Qtr after Exit
Nine Common Measures
Reporting Participant Information
Sequence of Services – Core, Intensive, Training
‘Services’ based Participation and Exit
Data Validation codified
Reporting Cohort extended 2nd to 4th Qtr after Exit
Twelve Primary Indicators of Performance
Expanded Reporting Participant Information
‘Career Services’ and Training
Performance Accountability *unless State ‘early implementer’

8. WIOA Youth Program Design
Focus on Out of School Youth
75% of formula youth funds minimum
Work Experience Expenditure
20% of local area funds minimum
Eligible Out of School Youth Age
16-24 year olds
New Program Elements

9. WIOA Youth Program Transition
Youth PY 2015 Funding
April 1, 2015
WIOA Implementation
July 1, 2015
While WIOA is effective July 1, 2015 and the Youth funds allotments are provided April 1, 2015, and it will take some time to transition, States and Local Areas must begin to incorporate strategies for recruiting and serving more OSY with the receipt of PY 2015 funds. Progress must be documented, including plans to achieve 75% expenditure rate with future funds.

10. Participation

11. Participant
An individual determined eligible to participate in the program who receives a service funded by the program in either a physical location (e.g., One-Stop Center) or remotely through electronic technologies.
Three Components
Determined eligible to participate in the program
Receives a funded service
In either a physical location or through electronic technologies

12. Components of Participant
Individual determined eligible to participate
Depends on program/funding; doesn’t apply in the case of W-P, which is based on universal access
Receives a service
Not all services trigger participation; it’s important to understand the distinction between those that do and those that don’t
In a physical location or remotely
Many substantial services are remotely accessed; this needs to be captured

13. Multiple Program Participation
Counting Participants in Multiple Programs
Earliest date of service
Can participate in several programs simultaneously
Counted as a participant in each of those programs
The participant won’t exit from the program unless there is a gap of no service for 90 days

14. WIOA Youth Program Participant
When does participation begin?
What services commence participation and inclusion in performance accountability?

15. A Service Is: Any core, intensive or training activity
made available to eligible participants that
allows them to benefit from specific
programs in the workforce system.

16. When a Service is included in Performance?
Core, intensive or training services made available to eligible participants and require significant staff involvement who exit the program.
These aforementioned individuals are included in the performance measures
Those WIA Adult and DW program participants who only receive self service or informational activities are excluded from performance

17. Services that Do Not Begin or Extend Participation
Eligibility determination
Case management administrative activities to obtain information regarding employment status, educational progress, need for additional services, etc.
Income maintenance or Support payments
Visitors to One Stop Centers, etc., for reasons other than its intended purposes
Follow-Up Services

18. Exit

19. Exiter
A participant who hasn’t received a program or partner-funded service for 90 consecutive days and no future services are scheduled
Three components
Hasn’t received a service
For 90 consecutive days
No future services scheduled

20. Components of Exiter The participant hasn’t received a service
Could be program- or partner-funded
For 90 consecutive calendar days
A gap in service can stop the 90-day clock if based on specific/allowable circumstances
No future services scheduled
Specific services and activities as allowable
Does not include any follow-up services or circumstances where the participant voluntarily withdraws or drops out of the program

21. Illustration: Participation and Exit
Eligible and Receives Service
End of 90-Day Period
Last Service
Participation
No Services or Follow-Up Services
Participation Date
Exit Date

22. Non-Compliance with EXIT Requirements
Exit dates not reflective of dates of last service
‘Case management’ used to extend exit date
Hard exits utilized
Date of last contact = Exit date
Date of employment = Exit date
Services provided within 90 days
Lack of common exit date (across core workforce programs)
Exit dates not consistent with dates in MIS 22 22

23. How to Prevent Unintentional Exits: Gaps in Service
Three allowable circumstances - the condition must exist for at least 90 days
Delay before beginning of training
Health/medical condition of participant or family member
Temporary move from the area that prevents participation

24. Exclusions Institutionalized Health/Medical or Family Care Deceased
Reserved Forces called to Active Duty
Relocated to Mandated Program (Youth)
Invalid/Missing Social Security Number

25. WIOA Youth Program Exit
When is it appropriate to exit Youth participants?
Common or Program Exit?

26. Follow Up Services Follow-up begins after the expected last service
Youth are required to receive at least 12 months of follow-up services, which are triggered at exit (the only exclusion is for summer youth employment)
Not intended to take responsibility away from case managers for WIA. Case managers do not have to wait 90 days, for instance, to begin providing follow-up services.

27. Total Youth Common Measures

28. PLACEMENT IN EMPLOYMENT OR EDUCATION
YES
Excluded
Employed, in the military or in post-secondary education at participation?
Has a qualifying outcome* in the 1st quarter after exit? NO
YES
Numerator
Denominator NO
Qualifying Outcomes: employment, military, enrolled in post-secondary education, advanced training, or occupational skills training

29. Placement in Employment or Education Reporting Time periods
For the Quarter ending Sept 2014
2013
2014
Jan-Mar
Apr-June
July-Sept
Oct-Dec
Exit Cohort
Employment/Education Qtr
2 Qtr Wage Delay
Nov 15th

30. LITERACY/NUMERACY GAINS
Definition: Of those out-of-school youth who are basic skills deficient, the percentage who increase one or more educational functioning levels within one year of participation

31. Literacy/Numeracy Definitions
Basic Skills Deficient
The youth computes or solves problems, reads, writes, or speaks English at or below the eighth grade level or is unable to perform these tasks at a level necessary to function on the job, in the individual’s family, or in society.
States and grantees may develop their own definition, but it must include the language above. 31

32. Definitions (Continued)
Out-of-School Youth
High School Dropout (no diploma or equivalent)
High School Graduate (or equivalency) not in postsecondary
High School Graduate (or equivalency) in postsecondary – but basic skills deficient 32

33. About the Assessments
All out-of-school youth must be assessed in basic reading, writing and math
Pre-testing must occur within 60 days of the first youth program service; can use pre-test from up to six months prior to date of first youth service
The same standardized assessment must be used for pre- and post-testing
Youth should be post-tested by the end of one year of participation and compared to pre-test results obtained during initial assessment 33

34. About the Assessments
If a youth continues to be basic skills deficient after the first 12 months of participation, they should continue to receive training in literacy and/or numeracy skills
Youth should be post-tested and included in the measure at the completion of the 2nd year only if they complete two full years in the program 34

35. Literacy/Numeracy Gains - 1st Year of Participation
Of those out-of-school youth who are basic skills deficient, the percentage who increase one or more educational functioning levels within one year of participation
Number of youth participants who increase one or more educational functioning levels
Number of youth who completed a year of participation (based on date of 1st youth service)
Number of youth who exit before completing a year of participation + 35

36. Literacy/Numeracy Gains 2nd and 3rd Years of Participation
Of those out-of-school youth who are basic skills deficient, the percentage who increase one or more educational functioning levels within one year of participation
Number of youth participants who increase one or more educational functioning levels
Number of youth who completed a second or third full year of participation (based on anniversary date of 1st youth service) 36

37. ATTAINMENT OF DEGREE OR CERTIFICATENO
Excluded
Enrolled in education at participation or any time during participation?
Attained diploma, GED or certificate by the end of the 3rd quarter after exit?
YES
YES
Numerator
Denominator NO

38. Credentials Defined
Awarded in recognition of an individual’s attainment of measurable technical or occupational skills necessary to obtain employment or advance within an occupation.
Work Readiness Credentials and those awarded by workforce investment boards are NOT included in this definition. 38

39. Credentials
TEGL established ‘credential’ as the umbrella term encompassing postsecondary degrees, diplomas, licenses, certificates, and certifications.
All states, regardless of the common measure waiver, must collect information on credential attainment in the WIASRD 39

40. Credentials: Approved Awarding Institutions
A state educational agency, or a state agency responsible for administering vocational and technical education within a state
Institution of higher education (including community colleges)
A professional, industry, or employer organization or a product manufacturer or developer
A registered apprenticeship program
A public regulatory agency (e.g., FAA aviation mechanic certification)
A program approved by DVA (Veterans’ Affairs) to offer education and training to veterans and other eligible persons under the Montgomery GI Bill
Office of Job Corps
Institutions of higher education which are formally controlled, or formally sanctioned or chartered by the governing body of an Indian tribe(s) 40

41. WIOA Youth Performance Measures
Education/Employment
2nd quarter after exit
4th quarter after exit
Earnings
Median earnings 2nd quarter after exit
Credential Rate
Up to one year after exit
If HS Dip/GED, must include Employment/Postsecondary
In-Program Skills Gain
Achieving measurable skills gains
Employer Effectiveness
before PY16

42. What’s Eliminated Literacy/Numeracy indicator for youth
Although utilized in development of Skills Gain measure
Customer Satisfaction as statutory measure
State Incentive Funds
But Governor’s reserve may be used for local incentives

43. Additional Provisions
State Targets
Must use statistical adjustment model—use now codified (Sec. 116(b)(3)(A)(viii))
Targets for first two years included in State Plans
Additional Information required in Annual Reports
Example: Amount of funds spent on each type of service
Data Validation now codified (Sec. 116(d)(5))

44. Additional Provisions
Sanctions
State Level
If a state fails performance, Secretaries shall provide TA (used to say will provide TA upon request)
If a state fails for 2nd consecutive year or fails to submit their Annual Report, it can lead to a reduction in statewide funds (stronger language)
Local Level
If failure continues for a 3rd consecutive year, the Governor must take corrective action which shall include development of a reorganization plan (and new local board)

45. Eligibility and Data Validation Documentation

46. Data Validation and Eligibility
States have the burden of identifying required Program Eligibility documents, or have given that responsibility to the Local Area.
While not required, many States use the Data Validation Source Documentation Guide as the required documents listing for Program Eligibility. 46

47. Source Documentation
Whether scanned, paper, or system cross-match, the purpose of source documentation is to have an auditable trail that documents the participant, services delivered and outcomes received.

48. Common Source Documentation Issues
Using documents that are not on ETA’s list of Acceptable Source Documents;
Checkbox in MIS;
Date of Birth;
Youth who Needs Additional Assistance;
Failure to Accurately record ‘Dates’; and
Poor Case Notes. 48

49. Questions?