1. U.S. Department of Labor Employment and Training Administration Transforming WIA Youth Programs through Performance and Reporting

2. 2 Agenda Review ETA’s vision for serving youthReview ETA’s vision for serving youth Clarify statutory & common measuresClarify statutory & common measures Review key terms and conceptsReview key terms and concepts Calculate youth common measuresCalculate youth common measures Youth program design implicationsYouth program design implications

3. 3 New Strategic Vision Out-of-school youth (and those most at risk of dropping out) are an important part of the new workforce “supply pipeline” needed by businesses to fill job vacancies in a knowledge-based economyOut-of-school youth (and those most at risk of dropping out) are an important part of the new workforce “supply pipeline” needed by businesses to fill job vacancies in a knowledge-based economy WIA-funded youth programs will provide leadership by serving as a catalyst to connect youth with quality secondary and post-secondary educational opportunities and high-growth employmentWIA-funded youth programs will provide leadership by serving as a catalyst to connect youth with quality secondary and post-secondary educational opportunities and high-growth employment

4. 4 The New Youth Vision Built on 4 pillars:Built on 4 pillars: –Meeting the demands of business (be demand-driven) –Alternative education –Focus on the neediest youth—special targeted populations –Focus on improved performance THE NEW YOUTH VISION

5. 5 What is Demand-Driven? Using labor market information and other economic data to assess employment opportunities in high growth industriesUsing labor market information and other economic data to assess employment opportunities in high growth industries Awareness of skills and competencies needed to satisfy employer requirementsAwareness of skills and competencies needed to satisfy employer requirements The strategic investment of public workforce system resources to meet employer needsThe strategic investment of public workforce system resources to meet employer needs Forming strategic partnerships to develop effective solution setsForming strategic partnerships to develop effective solution sets Catalyst to promote focused economic growth through talent developmentCatalyst to promote focused economic growth through talent development

6. 6 Why Focus on Alternative Education? 3.8 million dropouts, not employed3.8 million dropouts, not employed Negatively impact state economies through lost tax revenue as well as costs to other systems such as TANF and JJNegatively impact state economies through lost tax revenue as well as costs to other systems such as TANF and JJ The labor force needs these potential workers & 80% of new jobs will require some type of post- secondary trainingThe labor force needs these potential workers & 80% of new jobs will require some type of post- secondary training Youth workforce system is intermediary responsible for reconnecting youth to educational opportunitiesYouth workforce system is intermediary responsible for reconnecting youth to educational opportunities Without reconnecting them to educational opportunities, youth will not be adequately prepared to participate in new, knowledge- based economyWithout reconnecting them to educational opportunities, youth will not be adequately prepared to participate in new, knowledge- based economy

7. 7 Serving the Neediest Youth Youth in foster care or aging out of foster careYouth in foster care or aging out of foster care Children of incarcerated parentsChildren of incarcerated parents Court involved youth or those at risk of involvementCourt involved youth or those at risk of involvement Migrant youthMigrant youth Youth with disabilitiesYouth with disabilities Native American youthNative American youth Homeless and runaway youthHomeless and runaway youth Out-of-school youthOut-of-school youth High school dropoutsHigh school dropouts

8. 8 Improved Performance Implement common performance measuresImplement common performance measures Identify how collaborative efforts can lead to shared outcomes for all youth-serving agenciesIdentify how collaborative efforts can lead to shared outcomes for all youth-serving agencies Use “real time” data to manage programs and work toward program improvementUse “real time” data to manage programs and work toward program improvement

9. 9 Collaboration is Key Workforce programs alone cannot achieve this visionWorkforce programs alone cannot achieve this vision Shared youth vision activities emphasize the importance of collaboration at every levelShared youth vision activities emphasize the importance of collaboration at every level Federal/State/Local agencies that fund youth programs should be coordinating efforts in serving youthFederal/State/Local agencies that fund youth programs should be coordinating efforts in serving youth Critical to engage your partners at the state and local levelCritical to engage your partners at the state and local level

10. 10 Clarifying Accountability Youth WIA 7 & CM 3

11. 11 Implementation of PY 2007 Reporting WIA Statutory Measures Older youth (19-21)Younger youth (14-18) Older youth (19-21)Younger youth (14-18) Entered employment rate Skill attainment rate Retention rate Diploma attainment rate Credential/Certificate rate Retention rate Earnings change Youth Common Measures (14-21) Placement in employment or education Attainment of a degree/certificate Literacy/numeracy gains

12. 12 Accountability Most grantees are held accountable for performance outcomes on the seven WIA youth performance measuresMost grantees are held accountable for performance outcomes on the seven WIA youth performance measures –In the absence of a reauthorized WIA, the statutory measures must be used for accountability purposes All grantees are required to report on the three youth common measures – placement in employment/education, degree attainment and literacy/numeracyAll grantees are required to report on the three youth common measures – placement in employment/education, degree attainment and literacy/numeracy

13. 13 Current vs. Common Measures In General States should continue to ensure success with current measures, while transitioning to a focus on common measuresStates should continue to ensure success with current measures, while transitioning to a focus on common measures Major difference between the two sets of measures is the lack of ISY exclusion for placement and certificate common measuresMajor difference between the two sets of measures is the lack of ISY exclusion for placement and certificate common measures –Keep ISY engaged until graduation/GED attainment Focus on post-secondary education/ occupational skills training, employment and/or literacy/numeracy gainsFocus on post-secondary education/ occupational skills training, employment and/or literacy/numeracy gains

14. 14 Reviewing Key Terms and Concepts

15. 15 Program Participation A participant is an individual who is determined eligible to participate in a program and receives a service funded by the program. The participation date is the date the first service is received after the individual is determined eligible.

16. 16 Program Exit Exit occurs when a participant does not receive a service (funded by program or partner program) for 90 consecutive days and no future services are scheduledExit occurs when a participant does not receive a service (funded by program or partner program) for 90 consecutive days and no future services are scheduled There is no more concept of “hard exit” only what was a “soft exit”There is no more concept of “hard exit” only what was a “soft exit” The exit date is the date of the last service received by the participantThe exit date is the date of the last service received by the participant For measures using UI wage records, the exit quarter is the quarter containing the exit dateFor measures using UI wage records, the exit quarter is the quarter containing the exit date

17. 17 Clarifying Follow-Up Services Youth are required to receive at least 12 months of follow-up services, which are triggered at exitYouth are required to receive at least 12 months of follow-up services, which are triggered at exit With a “soft exit” approach, do you lose the first 3 months of follow-up? NOWith a “soft exit” approach, do you lose the first 3 months of follow-up? NO Begin follow-up after the expected last service –- if only follow-up services are provided for next 90 days, record an “exit”Begin follow-up after the expected last service –- if only follow-up services are provided for next 90 days, record an “exit” –However, if youth need additional services, this 90 days provides a “window” to better ensure success (because no official exit was recorded)

18. 18 Exclusions Circumstances that exclude participants from performance measures (current and common): InstitutionalizedInstitutionalized Health/Medical or Family CareHealth/Medical or Family Care DeceasedDeceased Reservists called to active dutyReservists called to active duty Relocated to a mandated program (such as foster care)Relocated to a mandated program (such as foster care) Invalid or missing SSNInvalid or missing SSN

19. 19 Gaps in Service Participants with a gap in service of greater than 90 days should not be exited if gap is due to one of the following: Delay before beginning of trainingDelay before beginning of training Health/medical conditionHealth/medical condition Temporary move from area that prevents individual from participatingTemporary move from area that prevents individual from participating

20. 20 Data Sources UI wage records and supplemental data sources are allowable for employment-related measures (placement and retention)UI wage records and supplemental data sources are allowable for employment-related measures (placement and retention) UI wage records for OY earnings measureUI wage records for OY earnings measure Administrative records will be data source for non-employment placements as well as degree or certificate, skill attainment & literacy/numeracy measuresAdministrative records will be data source for non-employment placements as well as degree or certificate, skill attainment & literacy/numeracy measures

21. 21 Calculating Youth Common Measures

22. 22 Youth Common Measures Placement in employment or educationPlacement in employment or education Attainment of a degree/certificateAttainment of a degree/certificate Literacy/numeracy gainsLiteracy/numeracy gains NOTE: The measures apply to youth 14-21; no more older/younger distinction!

23. 23 Placement in Employment or Education Definition Of those not in post-secondary education, employment, or the military at participation, the percentage in employment, the military or enrolled in post-secondary education and/or advanced training/occupational skills training in the 1 st quarter after exit.

24. 24 Calculation Of those not employed, in the military, or in post- secondary education at participation… Number of youth employed, in the military or enrolled in post-secondary education and/or advanced training or occupational skills training in the 1 st quarter after exit Number of exiters Placement in Employment or Education

25. 25 Excludes youth in employment, the military or post-secondary education at participationExcludes youth in employment, the military or post-secondary education at participation Employment, military and education status at participation is based on information collected from the individualEmployment, military and education status at participation is based on information collected from the individual Employment and military status in the 1 st quarter after exit is based on wage records (supplemental data allowed)Employment and military status in the 1 st quarter after exit is based on wage records (supplemental data allowed) Education status in the 1 st quarter after exit is based on administrative recordsEducation status in the 1 st quarter after exit is based on administrative records Youth in secondary school at exit are included, consistent with ETA’s vision to ensure youth successfully complete their secondary educationYouth in secondary school at exit are included, consistent with ETA’s vision to ensure youth successfully complete their secondary education Placement in Employment or Education

26. 26 Defining Specific Terms Post-Secondary Education A program at an accredited degree-granting institution leading to an academic degree. Programs offered by degree-granting institutions that do not lead to an academic degree (such as certificate programs) do not count as a placement in post-secondary education but may count as a placement in “advanced training/occupational skills training.”

27. 27 Advanced Training/Occupational Skills Training An organized program of study that provides specific vocational skills leading to proficiency in performing actual tasks and technical functions required by certain occupational fields at entry, intermediate or advanced levels. Such training should: be outcome-oriented and focused on a long-term goal as specified in the ISS,be outcome-oriented and focused on a long-term goal as specified in the ISS, coincide with exit rather than short-term training, andcoincide with exit rather than short-term training, and result in attainment of a certificateresult in attainment of a certificate Defining Specific Terms

28. 28 Employed, in the military or in post-secondary education at participation? Has a qualifying outcome * in the 1 st quarter after exit? NO YES Excluded YES Numerator Denominator NO Qualifying Outcomes: employment, military, enrolled in post-secondary education, advanced training, or occupational skills training. Placement in Employment or Education

29. 29 Attainment of a Degree or Certificate Definition Of those enrolled in education at or during participation, the percentage who attained a diploma, GED or certificate by the end of the 3 rd quarter after exit.

30. 30 Calculation Of those enrolled in education at participation or any time during the program… Number of youth who attained a diploma, GED or certificate by the end of the 3 rd quarter after exit Number of exiters Attainment of a Degree or Certificate

31. 31 Youth in secondary school at exit are included, which is consistent with ETA’s vision to ensure youth successfully complete their secondary educationYouth in secondary school at exit are included, which is consistent with ETA’s vision to ensure youth successfully complete their secondary education Diplomas, GEDs or certificates can be obtained during participation or at any point by the end of the 3 rd quarter after exitDiplomas, GEDs or certificates can be obtained during participation or at any point by the end of the 3 rd quarter after exit Work readiness certificates will not be accepted for this measureWork readiness certificates will not be accepted for this measure Consistent with ETA’s youth vision’s demand- driven approach to ensure training consistent with employer needsConsistent with ETA’s youth vision’s demand- driven approach to ensure training consistent with employer needs Attainment of a Degree or Certificate

32. 32 Education Participation in secondary or post-secondary school, adult education programs, or any other organized program of study leading to a degree or certificate. Diploma Any credential accepted by the State educational agency as equivalent to a HS diploma; also includes post-secondary degrees. Defining Specific Terms

33. 33 Certificate Awarded in recognition of an individual’s attainment of technical or occupational skills by specified institutions such as the State educational agency or institution of higher education. It does not include work readiness certificates or certificates awarded by local Boards. Defining Specific Terms

34. 34 Certificates: Awarding Institutions A state educational agency, or a state agency administering vocational and technical educationA state educational agency, or a state agency administering vocational and technical education Institution of higher educationInstitution of higher education A professional, industry, or employer organizationA professional, industry, or employer organization A registered apprenticeship programA registered apprenticeship program A public regulatory agency (e.g., FAA aviation mechanic certification)A public regulatory agency (e.g., FAA aviation mechanic certification) A program approved by Veterans’ Affairs to offer education and training to veterans and other eligible persons under the Montgomery GI BillA program approved by Veterans’ Affairs to offer education and training to veterans and other eligible persons under the Montgomery GI Bill Office of Job CorpsOffice of Job Corps Institutions of higher education which are formally controlled, or formally sanctioned or chartered by the governing body of an Indian tribeInstitutions of higher education which are formally controlled, or formally sanctioned or chartered by the governing body of an Indian tribe

35. 35 Enrolled in education at participation or any time during participation? Attained diploma, GED or certificate by the end of the 3 rd quarter after exit? YES NO Excluded YES Numerator Denominator NO Attainment of a Degree or Certificate

36. 36 Literacy/Numeracy Gains Definition Of those out of-school youth who are basic skills deficient, the percentage who increase one or more educational functioning levels within one year of participation.

37. 37 Calculation Of those out-of-school youth who are basic skills deficient… Number of youth participants who increase one or more educational functioning levels Number of youth who completed a year of participation (based on date of 1 st youth service) Number of youth who exit before completing a year of participation + Literacy/Numeracy Gains

38. 38 Only common measure that is not exit-basedOnly common measure that is not exit-based Excludes in-school youth and out-of-school youth who are not basic skills deficientExcludes in-school youth and out-of-school youth who are not basic skills deficient Includes individuals with learning disabilitiesIncludes individuals with learning disabilities A positive outcome means the youth must advance one or more Adult Basic Education or English as a Second Language functioning levelA positive outcome means the youth must advance one or more Adult Basic Education or English as a Second Language functioning level Gains can occur in literacy or numeracy (can pre-test at different levels in each category)Gains can occur in literacy or numeracy (can pre-test at different levels in each category) Educational levels are consistent with Department of Education’s National Reporting System (NRS)Educational levels are consistent with Department of Education’s National Reporting System (NRS) Literacy/Numeracy Gains

39. 39 About the Assessments All out-of-school youth must be assessed in basic reading, writing and mathAll out-of-school youth must be assessed in basic reading, writing and math Pre-testing must occur within 60 days of the first youth program service; can use pre-test from up to six months prior to date of first youth servicePre-testing must occur within 60 days of the first youth program service; can use pre-test from up to six months prior to date of first youth service The same standardized assessment must be used for pre- and post-testingThe same standardized assessment must be used for pre- and post-testing Youth should be post-tested by the end of one year of participation and compared to pre-test results obtained during initial assessmentYouth should be post-tested by the end of one year of participation and compared to pre-test results obtained during initial assessment

40. 40 About the Assessments If a youth continues to be basic skills deficient after the first 12 months of participation, they should continue to receive training in literacy and/or numeracy skillsIf a youth continues to be basic skills deficient after the first 12 months of participation, they should continue to receive training in literacy and/or numeracy skills Youth should be post-tested and included in the measure at the completion of the 2 nd year only if they complete two full years in the programYouth should be post-tested and included in the measure at the completion of the 2 nd year only if they complete two full years in the program

41. 41 Implications of the New Youth Vision and Program Design

42. 42 Long-Term Participation Program Design Implication #1 Positive outcomes are unlikely to be attained unless youth participate for extended periods of timePositive outcomes are unlikely to be attained unless youth participate for extended periods of time –In-school younger youth must remain engaged until they graduate or receive a GED and go on to employment or post-secondary education –Out-of-school youth who are basic skills deficient will likely need more time to improve one or more EFLs –Summer employment programs that enroll WIA youth for the summer only are unlikely to achieve positive outcomes

43. 43 Keeping youth connected to an organization orKeeping youth connected to an organization orprogram –Establishing a sense of self-worth through program participation (e.g., they get paid, receive positive feedback; their contributions matter, etc.) Providing incentivesProviding incentives –Opportunities to demonstrate skills to family and peers; field trips and events; stipends, opportunities to serve and lead; supportive services; recognition Youth-centered programsYouth-centered programs –Youth-friendly intake procedures; interpersonal support in programs from personnel, parents, peers; honesty and authenticity in program (provide what’s promised); culturally competent staff and culturally relevant programs Long-Term Participation Engagement and Retention Strategies

44. 44 Since long-term participation is needed to achieve outcomes, more than ever, WIA youth must have access to a network of services, not a single program or providerSince long-term participation is needed to achieve outcomes, more than ever, WIA youth must have access to a network of services, not a single program or provider –Strategic case management strategies will be needed to connect youth to multiple programs –Develop short-term attainable goals for youth and youth programs to support long-term positive outcomes Network of Services Program Design Implication #2

45. 45 Network of Services Strategic Case Management Case management is keyCase management is key –Create and maintain positive relationships with youth; have youth see case manager role as connector to a network of services; use team approach to helping youth

46. 46 For Youth:For Youth: –Setting intermediate goals (outcomes) helps them see progress, which contributes to engagement and retention Think SMART; Set goals WITH the youth; Make it VISUALThink SMART; Set goals WITH the youth; Make it VISUAL For WIA Youth Programs:For WIA Youth Programs: –WIA outcomes aren’t appropriate to use as program outcomes for one-year contracts; develop intermediate measures to monitor program success Including participation rates, skill/goal attainment ratesIncluding participation rates, skill/goal attainment rates Balance outcomes with outputs Network of Services Short-Term Outcomes

47. 47 When done well, assessment sets the stage for all future services across programs; when done poorly, it can have a negative impact on individual youth success and youth program successWhen done well, assessment sets the stage for all future services across programs; when done poorly, it can have a negative impact on individual youth success and youth program success –Youth friendly assessments Shorter versions when possible (subtests), pleasant testing environments, don’t send for a test right from the startShorter versions when possible (subtests), pleasant testing environments, don’t send for a test right from the start –Tell youth why they are being tested and how test will be used – don’t retest too soon –Consider other assessments – style, self-awareness, learning style – remove the high-stakes test pressure Assessment Program Design Implication #3

48. 48