1. KC Greaney, Santa Rosa Junior College Mallory Newell, De Anza College Terrence Willett, RP Group Measuring Our Success: Providing Evidence for CTE Outcomes RP Conference, Kellogg West, CA April 2013

2. 2 Measuring Our Success, RP Conference, April 2013 A Brief Overview - CTE SURVEY OUTCOMES - CTE LAUNCHBOARD PROJECT - CTE USERS FEEDBACK SURVEY - CTE OS DATA VISUALIZATION TOOL

3. 2012 Statewide Effort: CTE Employment Outcomes Survey Pioneered at Cabrillo College over past 12+ years Replicated at other individual colleges including Chaffey, Los Rios, Riverside, Mt. SAC, Santa Monica Statewide effort instigated by Bay Area Vocational Deans Measuring Our Success, RP Conference, April 2013 3

4. CTE Employment Outcomes Survey Collaborative effort among community college consortia, the Chancellor’s Office, and the RP Group Dual purposes Provide insightful information for local program improvement Collect statewide data for documentation of CTE outcomes Practitioner-Driven Effort Statewide meetings, conference calls, collaboration All methodology, implementation and survey questions vetted Report format developed in response to practitioner needs Measuring Our Success, RP Conference, April 2013 4

5. CTE Employment Outcomes Survey Methodology 2 Surveys: Completers, Leavers 3 Modalities: e-mail, then US mail, then phone calls Survey conducted approximately 1.5 years after completing/leaving Sample 15 Colleges All students included who met the criteria By 2009/10, had either earned a vocational award of 6+ units OR met VTEA “concentrator” definition AND did not enroll in 2010/11 Total = 15,105 (7,681 Completers and 7,424 Leavers) Measuring Our Success, RP Conference, April 2013 5

6. CTE Employment Outcomes Survey Data Chancellor’s Office MIS data wherever possible Local data supplement (student contact information) Survey data Complete data set provided to colleges for further analysis Measuring Our Success, RP Conference, April 2013 6

7. CTE Employment Outcomes Survey Response Rates Overall, 30% (Completers: 34%, Leavers: 26%) By College, overall response rate ranged from 19% to 37% Reports Statewide reports for Completers and Leavers College-level reports for Completers and Leavers Measuring Our Success, RP Conference, April 2013 7

8. Statewide Findings: Completers The big questions… Are our CTE students employable? Do they find jobs in the field in which we trained them? Do they see an increase in income? …in other words, was the program worth it? Measuring Our Success, RP Conference, April 2013 8

9. Statewide Findings: Completers 34% Response rate 94% Satisfaction rate with education and training 30% transferred (25% pursuing related field) 77% working in their field of study (or “close”) 73% are interested in taking more classes or training in their field in the near future 45% successfully engaged in a job search, 14% did not (the remainder were not looking or were already employed) Measuring Our Success, RP Conference, April 2013 9

10. Findings: Completers 79% of respondents are employed for pay Measuring Our Success, RP Conference, April 2013 10

11. Findings: Completers More students worked full-time after completing studies and training Measuring Our Success, RP Conference, April 2013 11

12. Findings: Completers The hourly wage of degree/certificate earners increased 39% from their hourly wage before their studies ($17.78) to their hourly wage after completing a degree or certificate ($24.79) Measuring Our Success, RP Conference, April 2013 12

13. Statewide Findings: Leavers The big questions… Are our CTE leavers “losers”? Or have we failed them? Are leavers employable? Do they find jobs in the field in which we trained them? Do they see an increase in income? …in other words, were the courses and training worth it, even if the student did not complete a program of study? Measuring Our Success, RP Conference, April 2013 13

14. Statewide Findings: Leavers Similar to Completers, except: Lower Response Rate (26% versus 34% for Completers) Fewer with the goal of earning a certificate or degree (60% versus 80% for completers) Twice as many with the goal of upgrading job skills and/or renewing a license/permit (19% versus 10% for Completers) Slightly higher transfer rate (34% versus 30% for Completers) Slightly fewer report working in their field of study (or “close” – 70% versus 77% for Completers) Earnings increase is lower (26% increase versus 39% for Completers) Measuring Our Success, RP Conference, April 2013 14

15. Basic Demographics CompletersLeaversAll CCCs Gender: Female60%51%54% Asian18%17%11% Black6%5%7% Latino19%20%30% White45%47%32% DSPS8%4%~ ESL7%5%~ Acad. Disadvantaged47%40%~ Migrant Worker8%6%~ n=7,681n=7,424n=2.8 million Measuring Our Success, RP Conference, April 2013 15

16. “Swirl” CompletersLeavers Attended multiple colleges within the same district 13.5%14.4% Attended multiple districts40.6%39.7% Attended only one college44.6%44.8% Measuring Our Success, RP Conference, April 2013 16

17. CTE Survey: Changes in 2013 Implement all the lessons we learned from the pilot year 35 colleges participating Move away from “Completers” versus “Leavers” Move towards “CTE Skill Builders” Broaden cohort to students who completed 9+ units of SAM-coded vocational courses (to include those who completed a certificate or vocational degree) Tighter timeline (reports out by the end of June) Add questions re: industry certification, impact of coursework, more responses taking transfer into account Measuring Our Success, RP Conference, April 2013 17

18. CTE Survey: Changes in 2014 Santa Rosa Junior College will continue to administer survey, working directly with interested colleges Chancellor’s Office endorses survey and will continue to define cohort via MIS and provide basic data RP Group is supportive, but no longer directly involved CCCAOE continues to support survey At Spring 2013 annual meeting, the CCCAOE “Excellence in Collaboration” award given to RP Group (Kathy Booth), Bay Area Vocational Deans Group (Rock Pfoetenhauer and Kit O’Dougherty) and Santa Rosa Junior College (KC Greaney) CalPASS+ will host CTEOS survey data in future Measuring Our Success, RP Conference, April 2013 18

19. 19 Measuring Our Success, RP Conference, April 2013 CTE User Feedback Survey

20. A link to the survey was emailed to all college contacts who participated in the survey in the 2011-12 academic year Received 7 valid responses Respondents suggested that the results had largely been used to inform decision making in the areas of: Enrollment management Accreditation Perkins or other reporting Program improvements Changes to program degree offerings 20 Measuring Our Success, RP Conference, April 2013

21. CTE User Feedback Survey Additional Highlights Respondents found the following to be most valuable about the survey: Ease of administration by the RP Group The multiple ways students were contacted to increase response rates The overall findings from the survey from my college The model report that was provided Respondents stated the following technical support would have been beneficial: Assistance with the initial data pull of students Assistance with analyzing the results Examples of how other colleges shared the results and with whom 21 Measuring Our Success, RP Conference, April 2013

22. 22 Measuring Our Success, RP Conference, April 2013 Launchboard Project

23. CTE Launchboard 4 Community Colleges participating in the pilot Includes data elements from “original” Launchboard Metrics (OLB) and metrics from the “Common Metrics” (CMLB) associated with the Doing What Matters Framework: http://doingwhatmatters.cccco.edu/.http://doingwhatmatters.cccco.edu/ Currently includes 67 data elements All metrics are presented by program TOPS code Some metrics presented by region as defined in the Doing What Matters Framework Includes data from: COMIS, CalPASS Plus, EDD, Centers for Excellence, and data files submitted by colleges 23 Measuring Our Success, RP Conference, April 2013

24. CTE Launchboard Includes measures in the following areas: Alignment – middle school to high school and high school to CCC Enrollments – course completion in high school and CCC in CTE pathway Capacity – FTEF, students served in CTE programs by CCC, students served in similar programs in non CCC colleges Support – completed work readiness program, internship, took basic skills course, received financial aid, DSPS or EOPS Return on Investment – number of years of income to equal cost per students 24 Measuring Our Success, RP Conference, April 2013

25. CTE Launchboard Milestones Completed major requirements in a CTE area (Primary Goal) (CMLB) Earned more than 8 units in the same TOP code within 3 years of first enrollment (OLB) Term-to-term retention (OLB) Completed college-level English or math within 3 years of first enrollment (OLB) Course completion and success (OLB) 25 Measuring Our Success, RP Conference, April 2013

26. CTE Launchboard Credentials Industry-recognized certificates (OLB) Non-credit certificates (OLB) Chancellor’s Office approved credit certificates in and not in CTE pathway (OLB) Associate’s degree in and not in CTE pathway (OLB) Transfer in and not in CTE pathway (CMLB) 26 Measuring Our Success, RP Conference, April 2013

27. CTE Launchboard Employment: Students are employed in same or similar field (OLB) Wage gain in field (OLB) Wage gain for students earning 9+ units (OLB) Average wage of students who are employed in same or similar field (OLB) Job retention (OLB) 27 Measuring Our Success, RP Conference, April 2013

28. CTE Launchboard Demand: Recent demand in Doing What Matters Economic Region (OLB) Projected demand in Doing What Matters Economic Region (OLB) Present demand in Doing What Matters Economic Region (OLB) Median (50 th percentile) Hourly Earnings (OLB) 28 Measuring Our Success, RP Conference, April 2013

29. CTE Launchboard Live look at the Launchboard: http://www.calpass.org/ 29 Measuring Our Success, RP Conference, April 2013

30. Proposed Help users access CTE Outcomes Survey data with web based reporting tool 30 Measuring Our Success, RP Conference, April 2013 CTE OS Data Visualization Tool

31. Questions? 31 Measuring Our Success, RP Conference, April 2013

32. Find Out More KC Greaney, Santa Rosa Junior College kgreaney@santarosa.edu Mallory Newell, De Anza College newellmallory@fhda.edu Terrence Willett, RP Group twillett@rpgroup.org (831) 461-4586 http://www.rpgroup.org/projects/CTE-Employment- Outcomes-Survey 32