1. U.S. Department of Education Office of Vocational and Adult Education Division of Academic and Technical Education An Overview of Six Recommendations and Issues Relative to State and Local CTE Accountability Systems John A. Haigh, Chief Performance and Accountability Branch Division of Academic and Technical Education Office of Vocational and Adult Education U S Department of Education Lolita B. Hall, Director Office of Career and Technical Education Services Division of Technology, Career and Adult Education Virginia Department of Education National Career Pathway Convention, October19, 2012

2. State Perkins Accountability Congress How would I Evaluate My RPOS Program? 2

3. State Perkins Accountability Congress How would I know My RPOS Program Was/Is Successful? 3

4. Are There separate Metrics? SecondaryPostsecondary 4

5. State Perkins Accountability Congress Agenda Accountability Elements of Department Blueprint The SPAC and Design Team Engaging in the Discussion SPAC Indicators Under Consideration 5

6. State Perkins Accountability Congress SPAC, Design Team and SMEs The SPAC will contribute clarification and uniformity as well as identify issues and options regarding Perkins accountability performance metrics 6 110 Delegate SPAC 50 States + 44 Member Design Team Subject Matter Experts Secondary Postsecondary

7. State Perkins Accountability Congress Department Blueprint Investing in America's Future: A Blueprint for Transforming Career and Technical Education Four core principles: 1.Alignment. Effective alignment between high-quality CTE programs and labor market needs to equip students with 21st-century skills and prepare them for in-demand occupations in high-growth industry sectors; 2.Collaboration. Strong collaborations among secondary and postsecondary institutions, employers, and industry partners to improve the quality of CTE programs; 3.Accountability. Meaningful accountability for improving academic outcomes and building technical and employability skills in CTE programs for all students, based upon common definitions and clear metrics for performance ; and 4.Innovation. Increased emphasis on innovation supported by systemic reform of state policies and practices to support CTE implementation of effective practices at the local level. 7

8. State Perkins Accountability Congress Proposed Accountability Reforms Common Definitions to Strengthen Data Systems and Close Equity Gaps for Participation: – Use uniform definitions for participation and performance indicators to create high-quality data systems that enable meaningful comparisons and identification of equity gaps Within-state Competitions to Distribute Funds to Consortia: Provide states increased autonomy to select and fund high-quality programs responsive to regional labor-market needs Incentives for High Performance: Incentivize and reward local recipients that exceed performance targets 8

9. State Perkins Accountability Congress Blueprint: Possible Negotiated Indicators Common performance indicators, intended to support meaningful crossstate data comparisons and identify equity gaps SHigh school graduation rate BRate of entrance into postsecondary education with and without remediation BRate of attainment of industryrecognized certifications or licensures PRate of attainment of postsecondary certificates and degrees BRate of employment and earnings S=secondary; B=both secondary &/or postsecondary; P=postsecondary 9

10. State Perkins Accountability Congress Blueprint: Non-negotiated Progress Indicators Common non-negotiated progress indicators to measure subpopulation enrollment in CTE, reported by gender, race/ethnicity, and special population status – the number of dual credits earned, – CTE credits meeting high school graduation requirements, and – workbased learning opportunities 10

11. State Perkins Accountability Congress SPAC and Design Team State Perkins Accountability Congress (SPAC) – Provide guidance, suggestions, and feedback to the Design Team – Address global issues that arise during the course of Design Team discussions Design Team – Develop detail of measurement approaches – Consider national and state perspectives on data collection, analysis, and reporting 11

12. State Perkins Accountability Congress Engaging in the Dialogue SPAC Website: http://cte.ed.gov/spachttp://cte.ed.gov/spac House materials and support dialogue – Document storage – Meeting recordings – Forum – Resources All interested stakeholders invited to visit the website, download materials, and engage in the online discussions 12

13. State Perkins Accountability Congress SPAC Website Forum Please visit http://cte.ed.gov/spac/index.php/forum Initiate and participate in discussions about issues related to the indicators, measures, and measurement approaches The forum is open to the field, including SPAC, Design Team, state and local administrators Create an account to participate 13

14. State Perkins Accountability Congress SPAC Forum 14

15. State Perkins Accountability Congress Indicators Under Consideration 1.Rate of diploma or award attainment – Secondary: High school graduation rate – Postsecondary: Rate of postsecondary award attainment (degree, certificate, credential) 2.Postsecondary enrollment rate – Secondary: Rate of enrollment following high school (with/without remediation) – Postsecondary: Rate of further enrollment in postsecondary education 3.Rate of attainment of employment 4.Earnings and wages 15

16. State Perkins Accountability Congress Guiding Ideas We want measures of the core indicators that: 1.Focus on student success 2.Can be acted upon at the local, state, or federal level 3.Are of high quality and efficient in collection and use 4.Are measurable over time 5.Are simple and easily understood 6.Are aligned among related programs and educational sectors 7.Avoid unanticipated consequences 16

17. State Perkins Accountability Congress Accountably Enrolled A term for the threshold that students must reach to be included in reported accountability results Secondary: Students who completed at least 50 percent of a state-approved career preparation program by the end of the reporting year Postsecondary: Students who earned at least 12 cumulative credits (or equivalent) in a state-approved career preparation program or completed a state-approved career preparation program of less than 12 credits (or equivalent) by the end of the reporting year 17

18. Secondary/Postsecondary Comparison Secondary High School Graduation Rate Rate of Secondary Enrollment in Postsecondary Education Rate of Secondary Employment Postsecondary Rate of Postsecondary Award Attainment Rate of Further Enrollment in Postsecondary Rate of Postsecondary Employment 18

19. State Perkins Accountability Congress High School Graduation Rate Measure: Percentage of accountably enrolled secondary students who graduated according to the states computation of its graduation rate as described in Section 1111(b)(2)(C)(vi) of the ESEA Numerator: Students who completed at least 50 percent of a stateapproved career preparation program by the end of the reporting year, were included in the states computation of its ESEA graduation rate in the reporting year, and received a standard high school diploma in the reporting year Denominator: Students who completed at least 50 percent of a stateapproved career preparation program by the end of the reporting year and were included in the States computation of its ESEA graduation rate in the reporting year 19

20. State Perkins Accountability Congress Rate of Secondary Enrollment in Postsecondary Education Measure: Percentage of accountably enrolled high school graduates who enroll in postsecondary education in the following year Numerator: Students who completed at least 50 percent of a stateapproved career preparation program by the end of the reporting year, received a high school diploma or equivalent in the reporting year, and enrolled in any postsecondary institution in the U.S. in the second quarter following the end of the reporting year during which the student graduated Denominator: Students who completed at least 50 percent of a stateapproved career preparation program by the end of the reporting year and received a high school diploma or equivalent in the reporting year 20

21. State Perkins Accountability Congress Secondary Rate of Employment Measure: Percentage of accountably enrolled students who graduated from high school, did not enroll in postsecondary education in the following reporting year, and who were employed or in the military at any time during the following reporting year Numerator: Students who completed at least 50 percent of a stateapproved career preparation program by the end of the reporting year, received a high school diploma or equivalent, were not found enrolled in a postsecondary institution the U.S. in the following reporting year, and were found to be employed or in the military at any time in the following reporting year Denominator: Students who completed at least 50 percent of a stateapproved career preparation program by the end of the reporting year, received a high school diploma or equivalent, and were not found enrolled in a postsecondary institution the U.S. in the following reporting year 21

22. State Perkins Accountability Congress Rate of Postsecondary Award Attainment Measure: Percentage of accountably enrolled students who were not enrolled in a postsecondary institution and who received a degree, certificate, or employer credential in the reporting year or following reporting year Numerator: Students who earned at least 12 cumulative credits or equivalent in a stateapproved career preparation program or completed a stateapproved program of less than 12 credits (or equivalent) by the end of the reporting year; were not found enrolled in any postsecondary institution in the U.S. during the following reporting year; and received a degree, certificate, or employer certification in the reporting year or the following reporting year Denominator: Students who earned at least 12 cumulative credits or equivalent in a stateapproved career preparation program or completed a stateapproved career preparation program of less than 12 credits (or equivalent) by the end of the reporting year and were not found enrolled in any postsecondary institution in the U.S. during the following reporting year 22

23. State Perkins Accountability Congress Rate of Further Enrollment in Postsecondary Measure: Percentage of accountably enrolled students who did not receive a degree, certificate, or employer credential in the reporting year or the following reporting year and who enrolled in postsecondary education in the following reporting year Numerator: Students who earned at least 12 cumulative credits (or equivalent) in a stateapproved career preparation program or completed a stateapproved career preparation program of less than 12 credits (or equivalent) by the end of the reporting year; did not receive a degree, certificate, or employer certification in the reporting year or the following reporting year; and were found enrolled in any postsecondary institution in the U.S. any time during the following reporting year Denominator: Students who earned at least 12 cumulative credits (or equivalent) in a stateapproved career preparation program or completed a stateapproved career preparation program of less than 12 credits (or equivalent) by the end of the reporting year and did not receive a degree, certificate, or employer certification in the reporting year or the following reporting year 23

24. State Perkins Accountability Congress Postsecondary Rate of Employment Measure: Percentage of accountably enrolled students received a postsecondary degree, certificate, or employer credential; were not enrolled in postsecondary the following year; and were employed or in the military at any time during the following reporting year Numerator: Students who earned at least 12 cumulative credits (or equivalent) in a state-approved career preparation program or completed a stateapproved career preparation program of less than 12 credits (or equivalent) by the end of the reporting year and were not found enrolled in any postsecondary institution in the U.S. during the following reporting year; received a degree, certificate, or recognized employer certification in the reporting year or the following reporting year; and were found to be employed or in the military in the following reporting Denominator: Students who earned at least 12 cumulative credits (or equivalent) in a stateapproved career preparation program or completed a stateapproved career preparation program of less than 12 credits (or equivalent) by the end of the reporting year and were not found enrolled in any postsecondary institution in the U.S. during the following reporting year; and received a degree, certificate, or recognized employer certification in the reporting year or the following reporting year 24

25. State Perkins Accountability Congress State and Local Issues Participation vs. concentration Should the indicator definitions be associated with measuring consortium outcomes? Should outcomes be assessed at both secondary and postsecondary levels or only upon completion of a full POS? Should the indicators and measures reflect longitudinal aspects of students educational experiences? Data collection, validity, reliability and reporting Design Team expressed interest in TSA but agnostic on accountable measure Will states be accountable for only Perkins funded programs? Will states and districts be allowed to use surveys? 25

26. State Perkins Accountability Congress Contact Information John Haigh Chief, Performance and Accountability Branch Division of Academic and Technical Education Office of Vocational and Adult Education U.S. Department of Education 550 12th Street SW Washington, DC 20202 P: 202-245-7735 John.Haigh@ed.gov Amanda Richards Associate Director, Preparation for College and Career MPR Associates, Inc. 1618 SW First Avenue, Suite 300 Portland, OR 97201 P: 503-222-5467 x402 arichards@mprinc.com 26