1. Achieving Successful Outcomes Literacy/Numeracy Gains

2. Housekeeping Please mute your phone throughout the presentation using your phone’s built-in mute button. To ask questions, either: –Type in your question (to all) in the chat feature at any point during the presentation –OR –Unmute your phone during the Q and A portion of the workshop to verbally ask your question. 2

3. 3 OUTLINE Why It’s Important The Measure and Definitions Tips for Staying on Track Test Your Understanding! Question and Answers

4. ● Based on PY 2011 WIASRD data, 61.4% of youth exiters were basic skills deficient and 46.3% were out of school. – Both of these numbers are all time highs for WIA Youth. ● If youth do not possess basic literacy skills, they will not be successful in the labor market. ● It is critical that WIA Youth programs help ensure these youth increase their basic literacy skills. Importance of Literacy/Numeracy Gains Measure

5. ● Leverage Partnerships! ● Workforce system does not need to go it alone in providing basic literacy skills training. ● The Workforce System can partner with Adult Education System (WIA Title II) – Overlap in populations served – Titles I and II are required to use the same assessments. ● Potential Areas for partnership include: – Test Administration Training (Academic Assessment) – Service Delivery Partnership – Data Sharing Partnering to Provide Basic Literacy Skills

6. Performance has increased each year, but still room for improvement Literacy/Numeracy Gains National Performance PY 2008 – 2011

7. 7 WHERE’S THE GUIDANCE? TEGL 17-05: Common Measures Policy for ETA Performance Accountability System and Related Performance Issues replaces the existing guidance with a single unified document. TEGL 17-05, Change 2: The intent of this guidance is to further clarify the application of the literacy/numeracy measure initially provided in Attachment C, Educational Functional Level Descriptors, in Training and Employment Guidance Letter (TEGL) 17-05, issued February 17, 2006, and to rescind TEGL 17-05, Change 1, issued August 13, 2007. TEGL 18-11: This TEGL provides guidance to increase literacy/numeracy gains and further clarify reporting policies and requirements. TEGL 28-11: Includes updated information and source documentation requirements concerning ETA data validation policy.

8. 8 Literacy/Numeracy Gains 1 st Year of Participation Of those out-of-school youth who are basic skills deficient, the percentage who increase one or more educational functioning levels within one year of participation Number of youth participants who increase one or more educational functioning levels Number of youth who completed a year of participation (based on date of 1 st youth service) Number of youth who exit before completing a year of participation +

9. 9 Literacy/Numeracy Gains 2 nd and 3 rd Years of Participation Of those out-of-school youth who are basic skills deficient, the percentage who increase one or more educational functioning levels within one year of participation Number of youth participants who increase one or more educational functioning levels Number of youth who completed a year of participation (based on anniversary date of 1 st youth service)

10. 10 Definitions Out-of-School Youth –High School Dropout (no diploma or equivalent) –High School Graduate (or equivalency) not in post-secondary –High School Graduate (or equivalency) in post-secondary – but basic skills deficient

11. 11 Definitions Basic Skills Deficient –The youth computes or solves problems, reads, writes, or speaks English at or below the eighth grade level or is unable to perform these tasks at a level necessary to function on the job, in the individual’s family, or in society. –States and grantees may develop their own definition, but it must include the language above.

12. 12 Highlights of Literacy/Numeracy Gains –Based on date of first youth service (not exit based) –Youth participants may be included for up to 3 years if they remain Basic Skills Deficient –Basic Skill Deficiency is determined at youth program eligibility. Basic skill deficiency is defined as meeting one of the 4: Initial Reading < 9.0, Math Level <9.0, Functionally Illiterate is yes, or Limited English is Yes –Participant is included in the measure even if they exit prior to end of the first year

13. 13 Highlights of Literacy/Numeracy Gains Only common measure that is not exit-based Excludes in-school youth and out-of-school youth who are not basic skills deficient Includes individuals with learning disabilities A positive outcome means the youth must advance one or more Adult Basic Education (ABE) or English as a Second Language (ESL) functioning levels Gains can occur in literacy or numeracy (programs can pre-test at different levels in each category) Educational levels are consistent with Department of Education’s National Reporting System (NRS)

14. 14 Definitions Educational Functioning Level (EFL) EFLABEESLBasic Skill Deficient 1 Beginning ESL Literacy Basic Skill Deficient 2 Low Beginning ESL 3Beginning ABE LiteracyHigh Beginning ESL 4Beginning Basic EducationLow Intermediate ESL 5 Low Intermediate Basic EducationHigh Intermediate ESL 6 High Intermediate Basic EducationAdvanced ESL 7 Low Adult Secondary EducationExit ESL Not Basic Skill Deficient 8 High Adult Secondary Education

15. 15 Literacy and Numeracy Definitions  Literacy = Read at an EFL of 7 or above on approved ABE and ESL assessment instruments.  Numeracy = Perform mathematical operations at an EFL of 7 or above on approved ABE / ESL assessment instrument  Skill Deficient = An EFL of 6 or below in Reading or Math on an approved assessment instrument.  Educational Functioning Level (EFL) are defined by US DOE and they reflect the various reading, speaking, and math skills necessary to function. An EFL roughly equates to two grade levels.

16. 16 About the Assessments  Approved Assessment Instruments : See National Reporting System (NRS) Implementation Guidelines to determine current approved assessments. Latest NRS Implementation Guidelines issued in the Federal Register Sept. 2011. –Tests are determined suitable for three years or seven years. –ABE = CASAS, MAPT, TABE, GAIN, WorkKeys, –ESL = CASAS, BEST, TABE/CLAS-E https://www.federalregister.gov/articles/2011/09/12/2011-23263/tests- determined-to-be-suitable-for-use-in-the-national-reporting-system-for- adult-educationhttps://www.federalregister.gov/articles/2011/09/12/2011-23263/tests- determined-to-be-suitable-for-use-in-the-national-reporting-system-for- adult-education

17. 17 About the Assessments All out-of-school youth must be assessed in basic reading, writing and math Pre-testing must occur within 60 days of the first youth program service; can use pre-test from up to six months prior to date of first youth service The same standardized assessment must be used for pre- and post-testing Youth should be post-tested by the end of one year of participation and compared to pre-test results obtained during initial assessment

18. 18 About the Assessments (cont’d) If a youth continues to be basic skills deficient after the first 12 months of participation, they should continue to receive training in literacy and/or numeracy skills Youth should be post-tested and included in the measure at the completion of the 2 nd year only if they complete two full years in the program

19. 19 Keys to Success Conducting Pre- and Post-Tests –Pre-test must occur within 60 days of the first youth program service, although can use pre-test from up to 6 months prior to participation –Post-test must occur by the end of one year of participation (whether first, second, or third year)

20. ● No Recorded Pre-Test Results = No Success ● If the youth is basic skills deficient without pre- test results, the youth will count as a negative outcome for the measure. ● No Recorded Post-Test = No Success ● Youth can only be a success if the POST- TEST shows a gain of at least one EFL. ● TIP: Record all test results! Keys to Success

21. Everyone counts in the first year. Youth still receiving services at the first year anniversary AND the youth who exited are included. Only youth celebrating an anniversary are in the measure for Year 2 and Year 3. Youth who exit during the second or third year don’t count. TIP: Remember anniversaries! Keys to Success

22. 22

23. 23 Oversight Approach Continual training to Youth Service Providers Regular use of Management Reports MIS reminders Engage Partners Performance-Based Contracts

24. 24 Youth Engagement Strategies Keep Youth connected to an organization or program –Establish a sense of self-worth through program participation (e.g., they get paid, receive positive feedback; their contributions matter, etc.) Provide Incentives –Opportunities to demonstrate skills to family and peers; field trips and events; stipends, opportunities to serve and lead; supportive services; recognition Youth-centered Programs –Youth-friendly intake procedures and testing; safe and supportive environment; honesty and authenticity in program (provide what’s promised); culturally competent staff and culturally relevant programs; develop short term goals

25. 25 Youth Engagement Strategies Smaller learning communities Applied learning Personal attention by caring adults Well-trained staff

26. 26 Smaller Learning Communities Maintain low teacher-student ratio Create small student cohorts based on academic or career interests Assign projects that require students collaboration and cooperation

27. 27 Applied Learning Link curriculum with the workplace Provide opportunities to connect class- based learning with life beyond school through hands-on activities that solve real-world problems

28. 28 Personal Attention Work to create positive youth-adult relationships from the beginning Develop individual service plans with all youth Provide opportunities for one-on-one instruction through tutoring and/or class time Where possible, involve family members as equal partners in youths’ success

29. 29 Well-Trained Staff Case Managers –Create and maintain positive relationships with youth –Have youth see case manager role as connector to a network of services –Use team approach to helping youth

30. Testing Your Knowledge Mary is a 17 year old high school dropout who is basic skills deficient as identified in WIASRD #130. James is a 20 year old high school graduate attending post-secondary school who is basic skills deficient as identified in WIASRD #130. Sue is a 19-year-old high school graduate who is basic skills deficient as identified in WIASRD #130. Peter is a 21-year-old high school dropout who is basic skills deficient as identified in WIASRD #130. Ed is a 16-year-old high school dropout who is not basic skills deficient as identified in WIASRD #130. Chris is a 21-year-old high school dropout who is basic skills deficient as identified in WIASRD #130. Ray is a 16-year-old attending high school who is basic skills deficient as identified in WIASRD #130. 30

31. 31 Testing Your Knowledge Q. Who will be included in the denominator of the Literacy/Numeracy measure?

32. A. All except Ed and Ray. Ed is not basic skills deficient per #130 and Ray is an in- school youth. 32 Testing Your Knowledge

33. 33 Testing Your Knowledge Q. Mary, James, Sue and Chris take the first- year pre-test (Peter does not). They have the following results in Reading and Math. Who is included in the denominator of the Literacy/Numeracy measure? Mary: Reading EFL is 8 and Math EFL is 8 James: Reading EFL is 7 and Math EFL is 4 Sue: Reading EFL is 3 and Math EFL is 5 Chris: Reading EFL is 3 and Math EFL is 7

34. 34 Testing Your Knowledge A. All are included except Mary, whose test results were > EFL 6. (Peter is a negative outcome since he is did not take a pre-test.)

35. 35 Testing Your Knowledge Q. The date of first youth service for James and Sue is 2/15/10. They both take the first year post-tests on 2/12/11, with the following results. Who is included in the numerator? James: Pre-test EFL for Math was 4; Post-test is 4 Sue: Pre-test EFL for Math was 5; Post-test is 6; Pre- test EFL for Reading was 3; Post-test is 3

36. 36 Testing Your Knowledge A. Sue is included in the numerator, as she attained a gain in Math

37. 37 Testing Your Knowledge Q. In the previous scenario, the date of first youth service for James is 2/15/10. If James exited the system on 2/14/11, is he still included in the denominator of the Literacy/Numeracy measure? Is he included in the numerator?

38. 38 Testing Your Knowledge A. James would be included in the denominator only (he had a valid post-test by the end of his first anniversary but did not achieve a gain).

39. 39 Testing Your Knowledge Q. Sue’s date of first youth service is 2/15/10, which makes her first year anniversary 2/15/11. Assume Sue’s second year pre-test (which is the first year’s post-test) continued to show basic skills deficiency in both math and reading. If she exits the system in January 2012 before taking her second year post-test, what will be her impact on the Literacy/Numeracy measure?.

40. 40 Testing Your Knowledge A. No impact. Sue did not complete a full second year, which means she’s excluded from the measure.

41. 41 Testing Your Knowledge Q. Take another look at the previous question. Assume Sue showed a gain in math or reading after taking her second year post-test. Sue exits the system shortly thereafter, on 2/10/12. What is her impact on the Literacy/Numeracy measure?

42. 42 Testing Your Knowledge A. No impact. Sue did not complete a full second year, which means she’s excluded from the measure (irrespective of the gain she achieved).

43. 43 Questions on Literacy/Numeracy Gains?