2. The Collapse of the Teen Job Market in Massachusetts and the U.S.: The Case for A Comprehensive Workforce Development System Response 1) The steep declines in teen labor force participation and employment rates in Massachusetts since 1999; the unprecedented drop in teen employment rates in our state and the nation since 2000. 2) Once a top performer, Massachusetts has fallen far behind the leading states in providing jobs for teens. 3) The failure of teens nationwide to benefit from the national jobs recovery beginning with the fall of 2003. 4) The sharp drop in employment rates among high school students in Massachusetts since 2000; large gaps in employment rates across gender, race-ethnic, and family income subgroups in the state.

3. 5) What can be done to address this crisis? Provide national, state, local leadership in addressing the problem; absence of any national leadership from either political party. Expansion of in-school work opportunities for youth; more support for school-to-career and connecting activities programs. Better school to work transition services for new graduates. An expanded summer jobs program at national/state level; resurrect the summer jobs program with wage subsidies for private sector employers. Employer tax credits for hiring participants in school-to- career programs; costs to employers often far exceed benefits that they will receive.

4. Trends in the Annual Average Civilian Labor Force Participation Rates of Teens in Massachusetts and the U.S., 1999-2006 (in %)

5. Trends in the Employment/Population Ratio of Teens in Massachusetts, Selected Years, 1999-2006

6. Comparisons of the Labor Force Participation Rates of Teens in Massachusetts With Those of the Top Five Ranked States and the Lowest Ranked State in 2006 (in %, Annual Averages)

7. Comparisons of the Employment Rates of High School Students (16+) in Massachusetts in 2000 and 2005, All and by Gender (in %)

8. Comparisons of the Employment Rates of High School Students (16+) in the U.S. and Massachusetts in 2005 by Race-Ethnic Group and Family Income

9. Predicted Probability of Employment in 2005 Among Hypothetical Subgroups of Massachusetts High School Students 16-19 Years Old (in Per Cent)

10. Comparisons of the Actual Number of Employed 16 and Older High School Students in Massachusetts in 2005 with the Number that Would Have Been Employed if Massachusetts Had Matched the E/P Ratios of the Top Five States

11. Primary Objectives of Workforce Development Program Evaluations 1. Identify who was served by the local and state workforce development system Did we serve the target groups as planned Compare characteristics of those served with the estimated universe of need for such services (the potential pool of eligibles: disadvantaged youth or adults, dislocated workers; the general jobless population)

12. 2. Identify the types of services received by participants in our workforce development programs and the intensity of those services (weeks and hours of participation) Identify who gets what services; are certain groups less likely to receive more intensive training services (young dropouts, older workers (55+), adult males from economically disadvantaged backgrounds)?

13. Identify the numbers of individuals receiving multiple treatments (basic education & training) versus single interventions; past evidence that basic skills services only improve outcomes when combined with other interventions such as occupational training or intensive job development; very limited to no positive effect for most at-risk youth from limited single shot interventions. Heckmans law of expected impact of zero for disadvantaged youth.

14. 3. Identify outcomes for participants in both the short-run and long-run; measure the employment status, weekly earnings, quarterly earnings from the first quarter following termination through the first 2-3 post- program years; estimate trajectories in employment and earnings over time Want to identify trends in employment status and earnings overtime; how high are job retention rates, do quarterly earnings improve over time, especially for graduates from longer-term education/training programs. National, regional, state research on JTPA/WIA/welfare reform show that training programs typically have more favorable (steeper) earnings trajectories; impact evaluations of JTPA adult programs show that earnings gains often do not evince themselves until a year or so after completion of training.

15. How do post-program outcomes vary across types of program interventions? Less intensive interventions (job search, job placement) must show favorable short-term outcomes to be effective. Their impacts fade over time. 4. What difference, if any, do workforce development programs make in improving labor market outcomes for participants? Answers to this question require impact evaluations with comparison/control groups.

16. Little to no national evidence for WIA programs; no recent impact evaluations at the national level; very little national evaluations of JTPA programs. Several impact evaluations over past 3-4 years by Commonwealth Corporation have showed significant positive earnings effects for JTPA adult training programs, vocational rehabilitation occupational training by MRC, and very modest earnings gains for participants from the Workforce Training Fund (weak available data bases to perform the analysis); we are not sure who benefits from WTF. The jury is still out.

17. The Shortcomings of the Existing Workforce Development Evaluation System Under WIA and Other State Programs 1. Some important programs, including the Workforce Training Fund, lack a micro-record data base on participants, services, and outcomes; this is a major shortcoming We cannot identify who are the beneficiaries of training services, who got what services, or how these services helped improve their occupational skills, promotions, wages, or earnings. The state should change this situation immediately.

18. 2. There continues to be substantive local differences in procedures for registering individuals into the One Stop System and tracking services and outcomes for participants and employers. This lack of uniformity limits our ability to track changes in local performance over time and to compare performance across one stop centers. There is a need for state uniformity in the collection, processing, and tracking of intake, services, and outcome data. The MOSES system also needs to be linked to the UI wage records to allow longer term tracking of clients in the one stop system and to identify which labor market segments are being served by the one stop system.

19. 3. The tracking of services to participants in youth, adult, and dislocated worker programs under WIA needs to be improved Services from other agencies need to be better documented The intensity of participation needs to be better measured; hours of actual services from each component need to be tracked as was the case under JTPA Identification of occupational areas in which training was provided; how close are the ties between occupational areas of training, job placement, and job vacancy data by occupation in the local labor market

20. 4. Post-program outcomes need to be more uniformly tracked for longer periods of time for WIA, adult basic education, and one stop centers Information on occupations of jobs obtained and industries of employers needs to be collected in a more detailed and accurate manner; establish links between occupations and industries of jobs obtained by our graduates and the occupations and industries of vacancies across the state.

21. Identification of occupational areas in which training was provided; how close are the ties between occupational areas of training, job placement, and job vacancy data by occupation in the local labor market

22. Identify the longer-term employment and earnings experiences of program terminees using the UI wage records; investigate sharing of UI wage records with neighboring New England states, especially for terminees from adult and dislocated worker programs in WIBs on the borders of surrounding states; we will miss up to 20 percent of the job placements for such areas with the UI wage records Improve knowledge of ABE/labor market outcomes; we have to justify the greater expenditures in recent years Estimate the links between employment and earnings outcomes and the types and identify of services received; which interventions appear to be most effective in raising earnings and employment

23. Past evidence has shown that classroom occupational training and OJT appeared to have most positive payoffs under JTPA (state and national); mixing occupational training with basic skills training also is effective; job search training or basic skills education by themselves had limited earnings impacts

24. More Impact Evaluations of Workforce Development Programs Should be Promoted a. The pool of individuals desiring training and education services often greatly exceeds the number of slots available – randomly assign applicants to alternative treatment to derive a control group with more limited services; do the same for adult basic education programs given their waiting lists b. Test out WTF and career ladders initiatives with comparison group firms who apply or express interest for training grants but do not get them.

25. c. Test out benefits from intense job development/specialized brokering units in one stops. d. More testing of the effectiveness of OJT programs; frequent success in past, yet few recent interventions.

26. How to Get There from Here? 1. The Governor is supposed to appoint a Performance Accountability Task Force in the near future to review information needs in workforce development arena Chinese Proverb: Talk doesnt cook the rice.

27. Andrew Sum Proverb: Talking about evaluation doesnt get it done. How can a consortium of local WIBs work together with the state, the regional office of ETA, and foundations to build upon our capacity to measure performance, test out new and innovative ideas, estimate impacts, and learn from one another in the process.