Rationale In examining completion outcomes, student-level characteristics are key. Disaggregation of data by student groups are important for both intervention and evaluation of outcomes. In addition to asking “How many graduated?” we should ask “Who graduated?”

Completion With a Purpose Predictive Analytics College Readiness Tool Academic Progress Tool Financial Risk Tool Diagnostic Analytics Dropout Data Tool Workforce Data Completion With a Purpose

Objectives Compare the percentages of students who graduate or expected to graduate within 150% of program length by student group. Track the completion rates of groups of students before and after the innovation project. Identify accomplishments in educating student groups with significant challenges. Identify student groups that are doing worse than their peers. Examine completion rates against targets. Identify questions for further inquiry.

Tool Design Part I Data entry and visual display of graduation rates Part II Reflection questions for team discussions

Definitions Student Population A Student population is defined by type of degree sought (2-year; 4-year) across programs of study and/or by a named program of study (e.g., business, education, nursing). Program length The amount of time necessary for a student to complete all requirements for a degree or certificate according to the institution’s catalog. Graduated within 150% of Program Length Completers are counted through August 31 of the summer following the sixth year of a 4-year program, or the third year of a 2-year program.

Calculation Method The percentage of students who are expected to graduate within 150% of program length in a given year is calculated based on students' enrollment date, characteristic, and credit accumulation. Define the student population Align with the goals of the innovation project Consider all factors that define the population (e.g., type of program, type of degree, enrollment status, etc.) Consider characteristics that may be called out as student groups. (Note: student groups may be defined as demographic characteristics, academic readiness characteristics, or other factors such as participation in Summer Bridge or Experiential Education programs).

Calculation Method (Continued) Identify the number of student in a group (e.g., students older than 24 years) Apply exclusions Calculate the percentage of students who graduated or the students expected to graduate Estimation of number of students expected to graduate within 150% of program length may be based on total credit accumulation to date or a more sensitive approach, which takes into account credit accumulation rate and early warning signs.

Visual Display of Data Each chart depicts graduation rates for one student group (e.g., first-generation students). The left side of each chart shows the graduation year. The middle part of each chart shows the time of assessment (e.g., baseline, Year 1). The right side of each chart shows the percentages of students, by group, who graduated or are expected to graduate within 150% of program length.

Using the Tool in Conjunction With Other Metrics and Tools Completion rates are insufficient to determine success. Additional indicators, which involve cost and staff time, can inform evaluation and action. Academic Progress tool offers metrics for monitoring progress towards career readiness. Workforce Outcomes tools offers metrics for assessing improved working conditions, salary, and well-being. Examples of Additional Indicators Number of students passing a licensure or certification exams related to degree (administrative records) Post-degree plans (student survey) Scores on academic tests designed to measure general education skills or field related tests (administrative records) Number of students who completed job training, apprenticeship, or internship (student survey)

The tool includes: Reflection questions for each year Space to document notes from team meetings to track data analysis over time Recommended Process: Allow for adequate time to drill down into data to gain a deeper understanding of issues. Record the questions and the data that served as impetus for the questions. Include all members of the Innovation Team and other key staff. Take notes of multiple perspectives.

Type your questions in the chat box. Raise your hand to ask questions or provide suggestions.

Bibliography The Hamilton Project. (2013). Using Data to Improve the Performance of Workforce Training. Washington DC: Author. Executive Office of the President of the United States. (2017). Using federal data to measure and improve the performance of U.S. institutions of higher education. Washington DC: Author. Jones, T. (2014). Performance Funding at MSIs: Considerations and Possible Measures for Public Minority-Serving Institutions. Atlanta, GA: Southern Education Foundation. Markle, R., Brenneman, M., Jackson, T., Burrus, J., & Robbins, S. (2013). Synthesizing Frameworks of Higher Education Student Learning Outcomes. Research Report. ETS RR-13-22. ETS Research Report Series, Self, S., Machuca, A., & Lockwood, R. (2014). Differences in the performance on the certified public accountant exam of graduates from minority and non-minority institutions. International Journal Of Education Research, 9(1), 43-56. Seifert, T. A., Gillig, B., Hanson, J. M., Pascarella, E. T., & Blaich, C. F. (2014). The Conditional Nature of High Impact/Good Practices on Student Learning Outcomes. Journal Of Higher Education, 85(4), 531-564.