1. Nobody is unpredictable M aritime Provinces Higher Education Commission March 26, 2013 Forum on Data Collection and Research: Measures of Student Progress and Outcomes

2. www.cespm.cawww.mphec.ca About Us – MPHEC Mandate (2005 Act): Give first consideration to improving and maintaining the best possible service to students Establish public reporting requirements and produce public reports Take measures to ensure continuous improvement in the quality of academic programs and of teaching Promote smooth transitions between learning and work Take measures intended to ensure programs are of optimum length and best quality Take measures intended to ensure teaching quality

3. www.cespm.cawww.mphec.ca MPHEC Commission Stakeholders Commission Staff Commission Members NB NS PEI Universities Govt & non- govt leaders Students & public-at-large

4. www.cespm.cawww.mphec.ca Operating Principles Institutional Autonomy Public Information Required Commission Mandate

5. www.cespm.cawww.mphec.ca Operating Principles Best Processes & Outcomes emerge from: Dialogue and Collaboration Best Processes & Outcomes emerge from: Dialogue and Collaboration

6. www.cespm.cawww.mphec.ca The Commissions Top Two Priorities MPHEC monitors Student progress through their education --- Graduates outcomes Universities assess programs/ services MPHEC validates institutional QA frameworks MPHEC assesses programs (All programs approved on the condition that the institution review them)

7. www.cespm.cawww.mphec.ca Data and Information: The Commissions Approach Research Agenda: Identifying the characteristics of programs and educational streams/pathways, and other factors, that impact upon student progression, credential completion, and graduate outcomes Research Agenda: Identifying the characteristics of programs and educational streams/pathways, and other factors, that impact upon student progression, credential completion, and graduate outcomes Demand Success Educational Pathways Educational Pathways Student Background Student Background Outcomes System Effectiveness Efficiency Value

8. www.cespm.cawww.mphec.ca MPHEC Research Agenda Research Agenda Monitoring of Basic Statistics / Measures Outputs Basic Statistical Tables Tuition Ancillary & Residence Fees Tables Measures of Student Progress & Outcomes Annual Digests Trends in Maritime Higher Education Support for Provincial KPIs Support Of Funding Administration Analysis Institutions Provided Own Statistics for Comparison

9. www.cespm.cawww.mphec.ca Measures of Student Progress and Outcomes Institutions: for example to support efforts to deliver a quality education Governments: assist in the monitoring of the effectiveness of policies Students and their families: understand the nature of educational pathways in the Maritime university system and the outcomes of its graduates The Measures support the Commissions mandate/ mission and are intended to assist stakeholders in their work Sensible markers of student progress and outcomes Intended Purpose and Value: Standardized definitions built from standardized information - extract value from data provided by institutions, graduates responses Building blocks for in-depth analysis

10. www.cespm.cawww.mphec.ca What do we intend to achieve today? Participants understand the Measures - purpose, value and scope - and provide feedback through roundtable discussion Participants understand the GO Survey Program and provide detailed feedback/suggestions: methodology, response rate improvement, quality of contact information, and reporting Participants will discuss the MPHECs traditional reporting practices, and explore options for future reporting RESULT Your feedback today will help ensure: value and relevance of the measures measures are aligned with needs effective reporting Your feedback today will help ensure: value and relevance of the measures measures are aligned with needs effective reporting

11. www.cespm.cawww.mphec.ca Measures of Student Progress and Outcomes Launched 2008 Working Group Working Group Data Sources Data Sources Student Administrative Data (PSIS) Graduate Survey Data Academic Program Information M EASURES OF P ROGRESS AND OUTCOMES

12. www.cespm.cawww.mphec.ca Demand for Maritime university education Student progress Characterizing educational pathways in Maritime universities Distance to institution from address of application Student/graduate outcomes Measures of Student Progress and Outcomes Measures

13. www.cespm.cawww.mphec.ca Overall Overall Full and Part-time Home Province Home Province Full and Part-time Home Province 18-24 Maritime Maritime Full and Part-time National Direct Entry Rate of High School Students Measures of Student Progress and Outcomes Published Feb 2012, Feb 2009 W HAT IS THE LEVEL OF DEMAND FOR UNIVERSITY EDUCATION ? (P ARTICIPATION R ATES ) 9 different definitions provide a comprehensive understanding; (PSIS + Census)

14. www.cespm.cawww.mphec.ca Understanding Participation Rates as a Measure of Demand Total FT enrolment population 18-24 Maritimers Enrolled FT in Region population 18-24 Maritimers Enrolled FT within Home Province population 18-24 Maritimers (18-19) Enrolled FT in Region population 18-19

15. www.cespm.cawww.mphec.ca Measures of Student Progress and Outcomes H OW IS STUDENT PROGRESS MEASURED IN MARITIME UNIVERSITIES ? Published January 2010 Within institution of first entry What proportion graduate within 6 years? (Graduation Rate) …Based on cohort tracking using longitudinal PSIS data What proportion of students re-enroll for second year of study? (Persistence Rate)

16. www.cespm.cawww.mphec.ca Published January 2010 Measuring Student Progress 79% of students re-enroll at the same university for a second year of study

17. www.cespm.cawww.mphec.ca Measuring Student Progress Published January 2010 58% of students graduate at the university of first entry within 6 years

18. www.cespm.cawww.mphec.ca Measures of Student Progress and Outcomes H OW IS STUDENT PROGRESS MEASURED IN MARITIME UNIVERSITIES ? …Based on cohort tracking using linked longitudinal PSIS data At the system level What proportion of students re-enroll for second year of study in any Maritime university? (Persistence Rate) What proportion graduate within 6 years at any Maritime university? (Graduation Rate)

19. www.cespm.cawww.mphec.ca The next 7 slides showcase measures under development; the statistics provided are preliminary numbers and are intended to provide examples only

20. www.cespm.cawww.mphec.ca Measuring Student Progress Preliminary system-level information Preliminary system-level information 83% of students re-enroll at any Maritime university for a second year of study

21. www.cespm.cawww.mphec.ca Measuring Student Progress Preliminary system-level information Preliminary system-level information 64% of students graduate from any Maritime university within 6 years

22. www.cespm.cawww.mphec.ca Measures of Student Progress and Outcomes What we know: 39% of students complete a 4-year bachelors degree at their university of first entry within 4 years Measures under development : How are different pathways related to persistence and graduation at the university of first entry and system level? …Based on cohort tracking using longitudinal linked PSIS data C HARACTERIZING EDUCATIONAL PATHWAYS IN M ARITIME UNIVERSITIES

23. www.cespm.cawww.mphec.ca Time-to-degree 4-year degrees What is the average time to complete a bachelors degree? (expected completion time 3.67 years) 4.2 years On average, how many terms do students complete? (expected time enrolled 8.0 terms) 8.6 terms Stop-outs1-2 terms off What proportion of students take a break from studying? 13% Characterizing Educational Pathways Measures under development – Preliminary examples

24. www.cespm.cawww.mphec.ca Characterizing Educational Pathways Measures under development – Preliminary examples Switching programs/disciplineWithin an institution What proportion of students switch programs after their first year? 18% Switching InstitutionsSystem-wide What proportion of students switch university after their first year? 4%

25. www.cespm.cawww.mphec.ca Number of courses completed relative to number required How many students earned more credits than required to graduate?41% Course load (60% course load is considered full-time) How many full-time bachelor students are taking a full course load (100% or more)? 55% Characterizing Educational Pathways Measures under development – Preliminary examples Course success/failure How many students fail at least one course?35%

26. www.cespm.cawww.mphec.ca Distance to University Measures under development – Preliminary examples Distance from institution from address of applicationAfter 1 year What is the persistence rate of Maritimers from within 40 km of the university attended? 80% What is the persistence rate of Maritimers from more than 80 km of university attended? 85%

27. www.cespm.cawww.mphec.ca Financing Education Percent who borrowed (any source) – First Degree 70% Average amount borrowed – First Degree$30,767 Percent debt-free two years after graduation (borrowers+non-borrowers) 44% Employment Employment Rate86% Percent with jobs requiring university education/management 59% Average Earnings (mean)$37,669 Graduate Outcome Measures Examples (statistics from Class of 2007 in 2009 graduate survey):

28. www.cespm.cawww.mphec.ca Mobility Percent Maritimers remaining in region two years after graduation 80% Percent non-Maritimers remaining in region two years after graduation 19% Further Education Percent who pursued further education within two years of completing first degree 59% Graduate Outcome Measures

29. www.cespm.cawww.mphec.ca Graduate Perceptions Education worth/well worth investment of time83% Education worth/well worth financial investment69% Percent who would choose the same field of study74% Percent reporting satisfied/very satisfied with quality of teaching in most classes 95% Graduate Outcome Measures

30. www.cespm.cawww.mphec.ca Thank you

31. www.cespm.cawww.mphec.ca Roundtable Discussion I 1. Is the focus on student progress and outcomes the right one to support the Commissions mandate? Explain. 2. Do the measures adequately describe student progress? student/graduate outcomes? Are there other measures that should be included? 3. What are the key information requirements in your work? What sources of information do you draw on (e.g., do you run/participate in surveys/data collection)? 4. Do/can these measures support you in your work? How?/Why not? What is missing? 5. What is the relative relevance of these measures within the landscape of other sources of measures/information you may use or consult, e.g., ACUDS, Macleans rankings? How can their relevance be increased? Measures of Student Progress and Outcomes: Scope, Value and Relevance

32. www.cespm.cawww.mphec.ca Roundtable Discussion II Roadmap 1.Is the roadmap clear; is there anything missing, or that can be improved (i.e. elements for greater clarification)? 2.Does the Roadmap help you understand how the Graduate Outcomes Survey Program will support the Programs research objectives, and how the Programs various components are inter- connected? Measures of Student Progress and Outcomes: Focus on the Graduate Outcomes Survey Program

33. www.cespm.cawww.mphec.ca Roundtable Discussion II Key Challenges 1.What could institutions do to encourage graduates to take part in the GO Surveys? Quality of Contact Information 2.How up-to-date, accurate, comprehensive are institutions graduate contact databases (i.e. e-mail, phone, address)? 3.What are the challenges in managing the contact database and keeping it up-to-date, what are some ways institutions can improve contact information? Reporting/Dissemination of Findings 4.What reporting format would be most useful and relevant for how you use the Graduate Outcomes Survey findings? For example, does the format proposed in the conceptual roadmap work or not work for you? 5.What are the reporting/dissemination options youd like to see explored?

34. www.cespm.cawww.mphec.ca Reporting In addition to public outputs institutions are provided: MPHEC FTEs, WFTEs, enrolments and credentials at the student level – provided during PSIS submissions process Graduate Surveys custom data – data sharing agreements Custom statistics to accompany reports

35. www.cespm.cawww.mphec.ca Roundtable Discussion III A. Is the traditional reporting practice useful? Does your institution make use of the custom information provided? B. Is there interest among the institutions in sharing with each other measures and other information at the institutional level (but not publicly reported) under a data-sharing agreement? If yes, would it be preferable to share anonymized information, or to identify institutions? What might be some pros and cons to such an initiative? C. How do you rate the usefulness of the measures/ information reported at the different levels – e.g., discipline, institution and system? What other level of reporting would be informative? Reporting