Oklahoma’s Experience: Proposed Measure Construction for 2P1-Completion Phoenix Data Quality Institute June 14-16, 2006 Kathy Spengler Oklahoma State Regents for Higher Education Sarah Mussett Oklahoma Department of Career and Technology Education

Background Oklahoma State System of Higher Education includes 13 colleges, 2 technical branches, and 12 universities. –Technology Centers offer certificate programs to adults at the postsecondary level. There are two systems of public, postsecondary education in Oklahoma. –The two-year colleges offer associate in applied science degrees, certificates of less than 1 year, and certificates between 1 and 2 years. –Three four-year universities also offer AAS degrees. Career Technology System includes 29 centers

Measurement Approach CURRENT Denominator: Entry Cohort ( ) of first-time entering students who declare an AAS program as their major Numerator: Students who are awarded a degree or certificate within six years PROPOSED EXIT GROUP: Denominator: students in an AAS or certificate program who (1) have completed 1/3 of the credits required in their program and (2) did not enroll the following year ( ) Numerator: Students who were awarded a degree or certificate [or who transferred to a 2-year or 4-year institution]

Measurement Approach 1.Identify Concentrators. If the student’s educational goal (theoretically consistent with the major) = AAS or 2-year certificate level, then determine whether the cumulative credit hours >= 20. If the educational goal = 1-year certificate, then determine whether the student had earned at least 10 credit hours. 2.Identify “Exiters.” If the student was a concentrator in , then check for enrollment in any semester of If no enrollment is found in that year, then flag as a “concentrator/exiter” and include in the denominator. SYSTEMWIDE APPROACH

Measurement Approach 3.Identify Degree or Certificate Recipients. Search for degrees awarded in or to match with those students previously identified as “concentrator/exiters.” We used a two-year window to catch many of the “degree-eligible” students who may not have actually received their degrees at program completion (due to outstanding tuition/fines or late applications, etc.)

Findings Current (AAS) # Current (AAS) % Exit Group (AAS only) # Exit Group (AAS only) % Exit Group (AAS & Cert) # Exit Group (AAS & Cert) % Total 7, %7, %8, % Male 2, %2, %3, % Female 4, %4, %5, % Am. Indian % % % Asian % % % Black % % % Hispanic % % % White 4, % %6, % Other % % %

Findings Overall, a change to the proposed exit group method would result in a higher Oklahoma completion rate than was reported in last year’s CAR: –Completion rates, even including certificate programs, would be almost 4 percentage points higher. Comparing only AAS students (ignoring students in certificate programs) under the current and proposed completion measure constructions: –More students, in total, would be included under the proposed method. The number of whites and females, in particular, would increase. Including certificates in the measure would: –improve the completion rates of minority students, up to 5.7% over what would be reported for AAS students only under the proposed plan.

Observations, Questions & Issues Concentrator, eligible coursework – –“Academic and technical coursework that is part of a state established or state approved, locally established CTE program of study may count toward calculating student concentrator status.” –Is it necessary to determine if the cumulative credit hours taken are truly “part of” the CTE program, as opposed to credits which may have been earned during previous study in a different program or may be extra electives that won’t count toward the CTE degree?

Observations, Questions & Issues Concentrator, program sequence – –The proposed definition states that “generally” the average number of credit hours for an AAS degree, two-year certificate, and one-year certificate would be 60 hours, 60 hours, and 30 hours, respectively. –Would there be an expectation that states apply program-specific requirements or institution- specific averages when determining how many hours are needed to be deemed a concentrator?

Observations, Questions & Issues Concentrator, student intent – –Is the use of educational goal to identify students enrolled in AAS or certificate programs appropriate, or is it contrary to the instructions to NOT use student intent in defining a concentrator? If it is inconsistent with our proposed measure, how will other states identify students to be included?

PROPOSED CONCENTRATOR ENTRY COHORT –Insufficient longitudinal data were available in this trial run to identify a concentrator entry cohort. –How difficult is it to determine the first point in time that a student attains concentrator status and avoid duplication among multiple institutions or semesters being reported? Observations, Questions & Issues