1. College Access and Readiness Outcomes of the University of California Transcript Evaluation Service 2014 NCES STATS-DC Data Conference July 31, 2014 TRANSCRIPT EVALUATION SERVICE (NOTE: Stock photo. Release for web use is on file.)

2. Agenda TRANSCRIPT EVALUATION SERVICE Why transcript data matter TES overview TES outcomes analysis −Characteristics of participating schools −Comparison to baseline −Comparison to non-participants (NOTE: Stock photo. Release for web use is on file.) 2

3. TRANSCRIPT EVALUATION SERVICE Transcript Evaluation Service (TES) An early and complete sequence of courses matters Students who complete 9 th grade college preparatory courses begin a clear trajectory that continues through high school Students who fall off the college preparatory track early in high school tend to move even further from a complete college preparatory program as they move through high school Students who do not get on a college preparatory track never get on (REL WestEd Policy Brief, 2008) College Preparation Trajectories 3 3

4. TRANSCRIPT EVALUATION SERVICE “a-g” Subject Area Benchmarks 4

5. TRANSCRIPT EVALUATION SERVICE Educational partnership tool used to improve access to college Provides programs, schools, and districts with an accurate depiction of course completion patterns to facilitate school improvement and course planning efforts Provides policymakers and researchers with quality information about student achievement measured against college-going benchmarks TES Overview 5

6. TRANSCRIPT EVALUATION SERVICE By providing students, parents, and those who support them with the right data: Students (and their counselors) will choose the right courses at the right time, preparing them for college High schools will be better able to offer the courses that students need to become college eligible TES Vision 6

7. TRANSCRIPT EVALUATION SERVICE 7 Features Individualized transcript evaluations Courses completed and needed to meet college preparatory benchmarks for every student grades 9-12 Courses completed and needed to meet college preparatory benchmarks for every student grades 9-12 Rosters and reports “a-g” course enrollment patterns for students at every grade level to aid school planning “a-g” course enrollment patterns for students at every grade level to aid school planning Training and support Help for schools to make the most of TES as an advising tool and to develop action plans for access Help for schools to make the most of TES as an advising tool and to develop action plans for access 7

8. TRANSCRIPT EVALUATION SERVICE TES Benefits 8 For students and families A plan for life after high school from an accurate, authoritative source A plan for life after high school from an accurate, authoritative source For counselors Increased advising efficiency and capacity Increased advising efficiency and capacity For administrators and policymakers Optimized school action planning around access and curricular rigor Optimized school action planning around access and curricular rigor 8

9. TRANSCRIPT EVALUATION SERVICE TES Process 9

10. TRANSCRIPT EVALUATION SERVICE Who participated? 152 California public high schools Generally (not exclusively): more disadvantaged (FRL), more ELL, more minority, lower academic performance (API) TES effect on CSU and UC eligibility? Eligibility rates improved substantially (up to 30%) More improvement where better implemented (over 50%) Some change in total college prep (a-g) courses taken TES effect on UC applications and admissions? 10-30% increase over 2-4 years Possibly more over longer horizons Outcomes analysis overview 10

11. TRANSCRIPT EVALUATION SERVICE TES program data Participation: Transcript uploads, which schools & when? Evaluation: Courses, GPA (for graduates) Dosage: Intensity of TES use; effective TES users Additional data CBEDS: Enrollment and graduates API: School decile, FRSL, ELL UCOP: applicants and admits from each CA high school Data used in the outcomes analysis 11

12. TRANSCRIPT EVALUATION SERVICE Number of high schools participating in TES, by submission cycle 2004-05 (12/10-9/16) 2005-06 (12/1-11/8) 2006-07 (2/6-11/1) 2007-08 (4/4-11/3) 2008-09 (1/30-5/14) 2009-10 (8/24-6/2) 2010-11 (9/2-7/20) 12

13. TRANSCRIPT EVALUATION SERVICE Number and percentage of TES schools that downloaded TES results 1234567 Percent of TES Schools whose data were downloaded 65%81%82%98%91%100% 13

14. TRANSCRIPT EVALUATION SERVICE Total enrollment in participating high schools, by TES cycle Source: CBEDS 14

15. TRANSCRIPT EVALUATION SERVICE Number of schools participating in TES according to the number of cycles 15

16. TRANSCRIPT EVALUATION SERVICE Selected student characteristics in participating TES high schools, by cycle 16

17. TRANSCRIPT EVALUATION SERVICE Year of TES ParticipationYear 1 to Year 2 1 st 2 nd 3 rd 4 th Change Percent Change Percent of Grads Eligible for CSU 23.825.71.98.0 Percent of Grads Eligible for UC 18.620.21.68.6 Percent of Grads Completing UC Courses 21.923.61.77.8 Mean Number of UC a-g Courses Completed 14.314.40.10.7 Number of Transcripts 28,30229,576 -- Among the 80 TES schools that participated for at least two consecutive cycles… 17

18. TRANSCRIPT EVALUATION SERVICE Among the 37 TES schools that participated for at least three consecutive cycles… Year of TES ParticipationYear 1 to Year 3 1 st 2 nd 3 rd 4 th Change Percent Change Percent of Grads Eligible for CSU 21.623.626.14.520.8 Percent of Grads Eligible for UC 16.818.419.42.615.5 Percent of Grads Completing UC Courses 19.921.723.94.020.1 Mean Number of UC a-g Courses Completed 13.914.014.20.32.2 Number of Transcripts 14,90215,46115,175 -- 18

19. TRANSCRIPT EVALUATION SERVICE Year of TES ParticipationYear 1 to Year 4 1 st 2 nd 3 rd 4 th Change Percent Change Percent of Grads Eligible for CSU 16.814.418.022.25.432.1 Percent of Grads Eligible for UC 13.112.514.015.82.720.6 Percent of Grads Completing UC Courses 15.413.816.820.95.535.7 Mean Number of UC a-g Courses Completed 12.912.612.813.70.86.2 Number of Transcripts 3,2363,6643,5803,362 -- Among the 10 TES schools that participated for at least four consecutive cycles… 19

20. TRANSCRIPT EVALUATION SERVICE Year of TES ParticipationYear 1 to Year 4 1 st 2 nd 3 rd 4 th Change Percent Change Percent of Grads Eligible for CSU 13.014.820.419.46.449.2 Percent of Grads Eligible for UC 8.413.515.813.95.565.5 Percent of Grads Completing UC Courses 11.214.418.717.66.457.1 Mean Number of UC a-g Courses Completed 12.512.713.914.41.915.2 Number of Transcripts 884867702762-- Among “intensive TES participants” (3 schools) participated for at least four consecutive cycles… 20

21. TRANSCRIPT EVALUATION SERVICE Year of TES ParticipationYear 1 to Year 4 1 st 2 nd 3 rd 4 th Change Percent Change Percent of Grads Eligible for CSU 20.831.437.116.378.4 Percent of Grads Eligible for UC 16.424.026.910.564.0 Percent of Grads Completing UC Courses 19.128.835.015.983.2 Mean Number of UC a-g Courses Completed 14.715.215.71.06.8 Number of Transcripts 2,6352,5912,633 -- For “effective implementer” district (6 of 8 schools participating for 3 consecutive years)… 21

22. TRANSCRIPT EVALUATION SERVICE Summary: Percent of Grads Eligible for CSU 22

23. TRANSCRIPT EVALUATION SERVICE Among the 37 TES schools that participated for at least three consecutive cycles… Year of TES ParticipationYear 1 to Year 3 1 st 2 nd 3 rd Change Percent Change a. History/Social Science2.13 2.150.020.9 b. English3.213.193.220.010.3 c. Math2.572.632.700.135.1 d. Laboratory Science1.741.771.780.042.3 e. Language Other Than English1.951.962.020.073.6 f. Visual and Performing Arts1.321.371.360.043.0 g. College-Preparatory Elective1.020.971.00-0.02-2.0 23

24. TRANSCRIPT EVALUATION SERVICE Year of TES Evaluation 2 nd 3 rd 4 th 5 th 6 th 7 th Number of Schools 743710532 Number of Graduates 26,37113,7493,4641,8161,031697 UC Application Percent for TES Schools, in Participation Year 15.416.611.711.9 12.717.4 UC Application Percent for TES Schools, at Baseline (1 st Year) 13.913.18.28.37.06.9 Growth in UC Application Percent for TES Schools (relative to base year) 1.53.5 3.65.710.5 Growth in UC Application Percent for Non- TES Schools (relative to base year) 1.31.62.63.03.75.3 Net Growth in UC Application Percent (relative to non-TES Schools) 0.21.90.90.62.05.2 Net Growth, as Percent of Baseline 1.514.210.46.629.176.9 Applications to UC: TES vs. Comparable Non-TES Schools 24

25. TRANSCRIPT EVALUATION SERVICE Net Growth in Applicants, as Percent of Baseline Year of TES Evaluation 2 nd 3 rd 4 th 5 th 6 th 7 th All Schools 1.514.210.46.629.176.9 Intensive TES Participants 0.822.26.817.140.0155.7 Effective TES Implementer 25.430.2 -- Schools with TES Partners (cycles 5-7) 4.120.3 -- Percent of Graduates Applying to UC: Net Growth, TES vs. Comparable Non-TES Schools 25

26. TRANSCRIPT EVALUATION SERVICE Percent of Graduates Applying to UC: Net Growth, TES vs. Comparable Non-TES Schools 26

27. TRANSCRIPT EVALUATION SERVICE Percent of Graduates Admitted to UC: Net Growth, TES vs. Comparable Non-TES Schools 27

28. TRANSCRIPT EVALUATION SERVICE TES and CSU/UC eligibility: Eligibility rates improved substantially (up to 30%) More improvement where better implemented (more than 50%) Some change on college prep course-taking TES and UC applications and admits: 10-30% increase over 2-4 years Possibly more over longer horizons Summary 28

29. Broader availability: Available to more schools/districts Faster data connections: Pulling in data directly from supported Student Information Systems (SIS) More user friendly: Redesign of TES results site and reports More support: Additional resources for how and when to use TES Ripple effects: More information about SIS data quality and opportunities to improve Transcript Evaluation Service Redesigned and Redefined (NOTE: Stock photo. Release for web use is on file.) 29

30. For more information about… TES Reginald Hillmon reginald.hillmon@ucop.edu www.transcriptevaluationservice.com TES Outcomes Analysis Roger Studley rstudley@rti.org (Photographs by Elena Zhukova, photographer, 415. 852. 8384, www.elenazhukova.com) (NOTE: Stock photo. Release for web use is on file.) 30