1. It’s a Journey Not a Destination: The Development of Virginia’s Postsecondary Education Reports Virginia Department of Education MIS Conference, San Diego, CA February 2012 Images in this presentation are used with permission from Microsoft.

2.  Longitudinal data bridged between the Virginia Department of Education (VDOE) and the State Council of Higher Education for Virginia (SCHEV).  SCHEV is Virginia’s coordinating body for higher education. As part of their responsibilities, SCHEV collects credits earned data from colleges in Virginia while also maintaining the confidentiality of student records in compliance with the Family Educational Rights and Privacy Act (FERPA) and state privacy laws.  Postsecondary achievement reports that meet the conditions of State Fiscal Stabilization Fund (SFSF) indicator (c)(12).  Postsecondary enrollment reports that meet the conditions of State Fiscal Stabilization Fund (SFSF) indicator (c)(11).  Virginia has collected postsecondary enrollment data from the National Student Clearinghouse for several years. To meet SFSF (c)(11),Virginia modified an existing report it provides to Local Education Agencies. Points of Interest We’ll See During Virginia’s Journey

3. Hh This Lane For State Agencies With Different Unique Student IDs

4. Virginia’s journey to create postsecondary achievement reports required that VDOE enter into a data-sharing agreement with the State Council of Higher Education for Virginia (SCHEV).

5. Preparing The De-identified Data Step One: IHEs in Virginia send student enrollment & credits earned data to the State Council of Higher Education for Virginia (SCHEV). Using an algorithm SCHEV acquires as many VDOE student IDs as possible. SCHEV de-identifies their data set so no meaningful student identifier is present. Step Two: High schools in Virginia send cohort graduation data to VDOE. Using an algorithm and human intervention, VDOE acquires as many SCHEV student IDs as possible. VDOE de- identifies their data set so no meaningful student identifier is present. Step Three: A 3rd party puts all the data sets through a 2 nd de-identification process. VDOE is provided with the final data sets. The data sets use random numbers for VDOE and SCHEV student IDs that permit the data sets to be merged without identifying specific individuals. SCHEV Student IDs VDOE Student IDs IHEs in Virginia 3 rd Party High Schools in Virginia

6. VDOE was ready to begin the merge process after it acquired the twice de-identified secondary and postsecondary data sets. YIELD To SCHEV Student IDs When Merging Data

7. Linking Data Students in the 2007-2008 FGI cohort Enrollment and credits earned data for First-time in college (FTIC) students The only cases included in the analytic dataset were cases where the SCHEV Student ID, VDOE Student ID, or SCHEV & VDOE Student ID matched. Data Source

8. Virginia’s journey to develop postsecondary education reports couldn’t continue until VDOE confirmed that the data merge process with SCHEV was successful. Data Validation Next Exit CAUTION Possible Mismatches On Gender, Race/Ethnicity, or Birth Month/Birth Year

9. Birth Month/Birth Year Gender 99% Race/Ethnicity 87% 99% Common Variable VDOE/SCHEV Alignment Rate Outcome There were only a few instances where gender did not align between the two data sources. Cases where misalignment was present were further analyzed to see if other common variables aligned. SCHEV and VDOE have different ways of collecting race/ethnicity. In addition, students may change their race/ethnicity classification upon entering postsecondary institutions. VDOE used these two facts as justifications for keeping cases where race/ethnicity didn’t align provided the other validation variables aligned. In cases where birth month/birth year didn’t align, a random sample was analyzed to determine reasons for the mismatch. The analysis revealed most mismatches were a result of data entry error. Based on this result, VDOE kept cases where birth month/birth year didn’t align provided the other validation variables aligned. Validation Decisions After the de-identification process was completed and the data sets were merged, only 35 cases were removed due to misalignment on multiple common variables.

10. What was Virginia’s match rate?  44,830 First-time first-year students with in-state residency status in the year 2008-2009 (source: SCHEV).  35,381 Virginia public high school graduates from the 2007-2008 graduating cohort matched to SCHEV data (79%).  DOE and SCHEV independently estimated that ~6% of FTIC students would not come from Virginia public high schools or the graduating class of 2007-2008.  Approximately 15% unmatched = 85% match rate (estimated).

11. More on the data-linking process  The use of SCHEV student IDs allowed VDOE to add an additional 4,443 students into the longitudinal data set.  VDOE and SCHEV independently concluded that any estimates about postsecondary enrollment or postsecondary achievement that relied on the longitudinal data set would be underestimates.  Efforts to improve the probabilistic matching process are currently underway as part of Virginia’s grant- funded work to expand the Virginia Longitudinal Data System (VLDS).

12. Virginia’s journey continued with the creation of postsecondary achievement reports that met the conditions of SFSF indicator (c)(12). COLLEGE CREDIT MINIMUM LIMIT 30 FGI GRADS VA Public IHE Zone

13. Conditions of SFSF (c)(12)  Virginia created postsecondary education reports for SFSF indicator (c)(12) consistent with the federally prescribed measure of calculating cohort graduation rates.  Only students who earned Virginia’s standard or advanced studies diploma count as graduates in Virginia’s Federal Graduation Indicator (FGI).  Students reporting as graduating within four, five and six years of first entering the ninth grade were included in the postsecondary achievement reports.

14. More Conditions of SFSF (c)(12)  Only students who enrolled in a Virginia public IHE within sixteen months of graduation were included.  Thirty units of college credit (applicable to a degree) was the benchmark for earning one year of college credit within two years of IHE enrollment.  Credits earned for passing developmental education courses were not included in the calculation.

15. SCHEV and VDOE did not have access to transcript level credits earned data With the conditions of SFSF indicator (c)(12) applied, VDOE was ready to calculate how many students earned one year of college credit within two years of IHE enrollment.

16.  The calculation for credits earned required VDOE to identify the college courses (applicable to a degree) a student enrolled in within a two-year time frame and sum the amount of credits earned for passing those courses.  The impact of unofficial transcript-level data resulted in VDOE and SCHEV making decisions about the following critical topics :  Credits awarded for passing AP courses or earning qualifying scores on AP tests.  Not included in this year’s calculation.  Dual Enrollment Credit  Missing Grades Impacts Of Unofficial Transcript-Level Data

17. This Lane For Including Earned Dual Enrollment Credits VDOE created postsecondary achievement reports for SFSF indicator (c)(12) that show results with and without earned dual enrollment credits included in the calculation of credits earned.

18. Virginia Department Of Education State Fiscal Stabilization Fund Indicator (c)(12) Report 2007-2008 FGI cohort year (students entering high school in 2004-2005) Graduation Rate Type: Four-Year Rate Division: ALL, School: ALL Students that earned a federally recognized diploma and enrolled in a public Institution of Higher Education (IHE) in Virginia. Total number of students in cohort who graduated from high school with a federally recognized diploma and enrolled in a public IHE in Virginia within 16 months of graduation Dual enrollment credits included in total count of credits earned Dual enrollment credits excluded in total count of credits earned Earned one year of college credit within two years of enrollment Amount of credits earned could not be determined due to the presence of missing grades Earned one year of college credit within two years of enrollment Amount of credits earned could not be determined due to the presence of missing grades SUBGROUPNumber PercentNumberPercentNumberPercentNumberPercent All Students315792110267395120852663851 Female174431204769188111906681781 Male1413690556420718946632071 American Indian74516923516923 Asian2160173780362173380362 Black5600294653821291152811 Hispanic11816735731366857313 Native Hawaiian34236813236813 White221401539670237115191692281 Economically Disadvantaged3812187949521183748501 Limited English Proficient Students 12458316727282967272 Students with Disabilities14675804027257039252 Notes: < refers to a group below state definition for personally identifiable results. - = no data available for that group According to the State Council of Higher Education for Virginia (SCHEV), credits from remedial courses do not count towards a college degree and are to be excluded from the total number of credits earned. According to the State Council of Higher Education for Virginia (SCHEV), 30 units is the equivalent of one-year of college credit. Federally recognized diplomas include Standard, Advanced Studies, and International Baccalaureate (IB) diplomas. The number of students enrolled and the number of students earning credit (including and excluding dual enrollment credits) are based on the available data the Virginia Department of Education (VDOE) was able to link with the State Council of Higher Education for Virginia (SCHEV).

19. Missing Grade Crossing In some records, public IHEs did not submit final course grades to SCHEV. Some of these missing grades impact the determination of whether or not a student earned 30 or more credits within two years of enrolling in college. The postsecondary achievement reports for SFSF (c)(12) document the percentage of students for whom insufficient data were available to make a final determination about whether they met the criteria of SFSF (c)(12).

20. Virginia Department Of Education State Fiscal Stabilization Fund Indicator (c)(12) Report 2007-2008 FGI cohort year (students entering high school in 2004-2005) Graduation Rate Type: Four-Year Rate Division: ALL, School: ALL Students that earned a federally recognized diploma and enrolled in a public Institution of Higher Education (IHE) in Virginia. Total number of students in cohort who graduated from high school with a federally recognized diploma and enrolled in a public IHE in Virginia within 16 months of graduation Dual enrollment credits included in total count of credits earned Dual enrollment credits excluded in total count of credits earned Earned one year of college credit within two years of enrollment Amount of credits earned could not be determined due to the presence of missing grades Earned one year of college credit within two years of enrollment Amount of credits earned could not be determined due to the presence of missing grades SUBGROUPNumber PercentNumberPercentNumberPercentNumberPercent All Students315792110267395120852663851 Female174431204769188111906681781 Male1413690556420718946632071 American Indian74516923516923 Asian2160173780362173380362 Black5600294653821291152811 Hispanic11816735731366857313 Native Hawaiian34236813236813 White221401539670237115191692281 Economically Disadvantaged3812187949521183748501 Limited English Proficient Students 12458316727282967272 Students with Disabilities14675804027257039252 Notes: < refers to a group below state definition for personally identifiable results. - = no data available for that group According to the State Council of Higher Education for Virginia (SCHEV), credits from remedial courses do not count towards a college degree and are to be excluded from the total number of credits earned. According to the State Council of Higher Education for Virginia (SCHEV), 30 units is the equivalent of one-year of college credit. Federally recognized diplomas include Standard, Advanced Studies, and International Baccalaureate (IB) diplomas. The number of students enrolled and the number of students earning credit (including and excluding dual enrollment credits) are based on the available data the Virginia Department of Education (VDOE) was able to link with the State Council of Higher Education for Virginia (SCHEV)..

21. VDOE’s journey also included postsecondary enrollment reports that met the conditions of SFSF indicator (c)(11)

22. The data linking process for State Fiscal Stabilization Fund Indicator (c)(11) Institution of Higher Education Step One: Colleges and universities across the nation send student-level enrollment data to the National Student Clearinghouse (NSC ). Step Two: The Virginia Department of Education (VDOE) sends student-level cohort graduation data to NSC. Step Three: NSC uses a proprietary algorithm (plus internal staff review) to compare college enrollment records to VDOE data. NSC supplies VDOE with a data set that shows the results. VDOE creates SFSF (c)(11) reports.

23. Virginia Department Of Education State Fiscal Stabilization Fund Indicator (c)(11) Report 2010-2011 FGI cohort year (students entering high school in 2007-2008) Graduation Rate Type: Four Year Rate Division: ALL, School: ALL Total number of students in the cohort earning a federally recognized high school diploma Students who enrolled in any Institution of Higher Education (IHE) within 16 months of earning a federally recognized high school diploma Students who enrolled in a 4- year public Institution of Higher Education (IHE) within 16 months of earning a federally recognized high school diploma Students who enrolled in a 4- year private Institution of Higher Education (IHE) within 16 months of earning a federally recognized high school diploma Students who enrolled in a 2-year Institution of Higher Education (IHE) within 16 months of earning a federally recognized high school diploma. Subgroup Number PercentNumberPercentNumberPercentNumberPercent All Students8002349739622404330755191814523 Female4094527030661317032425310960723 Male390782270958108732832988853822 American Indian2851635774262286724 Asian4718357476222547354899521 Black186281008954401122193410414422 Hispanic62573141501184193676159025 Native Hawaiian1056057292810 2120 White47402310136515667334621101072523 2 or More259316816585133242958823 Economically Disadvantaged2054594534627571313737532326 Limited English Proficient Students4228208649712171604121429 Students with Disabilities549823284247693116154128 NOTES:< = A group below state definition for personally identifiable results. - = no data available for that group This report uses the best available data according to the National Student Clearinghouse. Students who attended schools that do not participate in NSC are not included in the number or percent of students enrolled in an IHE. Federally recognized high school diplomas include Standard, Advanced Studies, or International Baccalaureate (IB) diplomas. Most subgroups are based on students most recent status. Feb 10, 2012 12:39 PMPage 1 of 1

24. Resources  Postsecondary achievement reports for SFSF (c)(12): https://p1pe.doe.virginia.gov/postsec_public/postsec.do?dowhat=LOAD REPORT_C12 https://p1pe.doe.virginia.gov/postsec_public/postsec.do?dowhat=LOAD REPORT_C12  Frequently Asked Questions about postsecondary achievement reports: http://www.doe.virginia.gov/school_finance/arra/stabilization/reported_d ata/assurance_c/faq_c12.pdf http://www.doe.virginia.gov/school_finance/arra/stabilization/reported_d ata/assurance_c/faq_c12.pdf  Postsecondary enrollment reports for SFSF (c)(11) : https://p1pe.doe.virginia.gov/postsec_public/postsec.do?dowhat=LOAD REPORT_C11 https://p1pe.doe.virginia.gov/postsec_public/postsec.do?dowhat=LOAD REPORT_C11  Frequently Asked Questions about postsecondary enrollment reports: http://www.doe.virginia.gov/school_finance/arra/stabilization/reported_d ata/assurance_c/faq_c11.pdf http://www.doe.virginia.gov/school_finance/arra/stabilization/reported_d ata/assurance_c/faq_c11.pdf  Information about Virginia’s College and Career Readiness Initiative: http://www.doe.virginia.gov/instruction/college_career_readiness/index. shtml http://www.doe.virginia.gov/instruction/college_career_readiness/index. shtml

25. The detours and decisions shown in this presentation were based on real people and actual events that occurred in the Virginia Department of Education. In some cases, the names of the people taking those actions were omitted to protect their identities. For More Information Contact Nathan Carter, Ph.D. Nathan.Carter@doe.Virginia.gov Deborah Jonas, Ph.D. Deborah.Jonas@doe.Virginia.gov Virginia Department of Education