We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byJoshua Hill
Modified over 2 years ago
© The Open University, Institute of Educational Technology 1 Alison Ashby, Naomi Jeffery, Anne Slee Student Statistics and Survey Team, IET The Open University What Works? Student Retention and Success Two Day Retention Convention 3 rd and 4 th March 2010 Using Institutional information to understand student retention and support enhancement
© The Open University, Institute of Educational Technology 2 Background information The Open University (OU) is the UKs largest university and teaches 35% of all part- time undergraduates in the UK. There are over 150,000 student course registrations each year (over 75,000 FTE). The OU has a diverse student population, for example: just over 40% of students have less than two A levels or equivalent on entry (the OU has an open entry policy) Just over 10% of students are aged 55 or over and 14% are aged under 25. There are 450 course/modules at undergraduate level Source Facts and Figures 2007/08 The Open University
© The Open University, Institute of Educational Technology 3 The case for a course pass model Comparisons of course pass rates take no account of the scale and diversity of students and the curriculum. It is easy to explain away differences in course pass rates by suggesting they are due to different course types/students. Attempts to explore differences in a univariate way are too complex and too resource intensive given the scale. A course pass rate model: –controls for known factors about students and courses –provides a fairer comparison of course pass rates –enables the University to focus attention on courses that have higher/lower pass rates than expected. –Increases understanding / opportunities for enhancement
© The Open University, Institute of Educational Technology 4 Key characteristics of the model The model predicts the probability of an individual student passing a course. Separate models produced for new and continuing students. For each course, individual probabilities are added together to provide a predicted pass rate for a course. Each variable included in the model is treated as a categorical variable e.g. age is converted to age bands. Logistic regression has been used to produce the model. The current model is based on five years of data. A statistic called the Z score, which takes into account course population, is produced to compare actual pass rates with predicted pass rates. This is used to determine which courses appear to be under/over performing.
© The Open University, Institute of Educational Technology 5 Examples : student characteristics gender age occupation previous study ethnicity qualifications disability financial support
© The Open University, Institute of Educational Technology 6 Examples: Course characteristics Course in first year of presentation credit points Number of assignments level Exam / end of course assessment Academic unit
© The Open University, Institute of Educational Technology 7 Model benefits and limitations Benefits: More meaningful comparison of course pass rates Systematic process for identifying courses where there may be opportunities for enhancement or sharing good practice. Limitations: Models are approximations Some important factors may be missing from the model The model, on its own, could excuse performance based on known factors. e.g. the model predicts a lower course pass rate where a higher percentage of students have characteristics which have a negative effect
© The Open University, Institute of Educational Technology 8 An example of a predictor variable (scale = maximum likelihood estimates, blue bar = standard errors)
© The Open University, Institute of Educational Technology 9 How is the course pass rate model used at the Open University? At a local level by course teams as part of the annual course review process. At University level, by the Student Experience Advisory Group (SEAG) which has responsibility to advise on : The effectiveness of the use of information on the student experience for quality enhancement purposes. Areas of particular strength and weakness in the quality of the student experience and performance. The results of the model are not used in isolation. Given the limitations, it is important to triangulate results.
© The Open University, Institute of Educational Technology 10 Annual Course Review In addition to the results of the course pass rates model, the following data sources are available to course teams: –Actual registrations and course pass rates over a five year period for all courses to allow a review of trends and comparisons with other courses. –A profile for each course which includes the distribution and pass rates analysed by key indicators e.g. age, gender, previous courses taken and credit points gained. –A table showing results for the ten key performance indicators identified from the end of course survey for each course surveyed in the period under review. –Detailed survey results for each course –Open comments for each course
© The Open University, Institute of Educational Technology 11 Addressing the evidence Briefing sessions take place to support course teams in using and interpreting the information provided. As part of the review process course teams are asked to consider the evidence sources and comment on: The strength of the course and actions planned/being taken to share successful practice of the course in its programme of study/academic unit areas for improving the course which have been identified and actions planned/being taken to address them Each academic unit has responsibility for ensuring the oversight of these reviews and the actions planned/taken.
© The Open University, Institute of Educational Technology 12 The Student Experience Advisory Scrutiny Group (SEAG) SEAG has agreed a set of three key performance indicators to select annual course review reports for scrutiny: The course pass rate model : Z score outside acceptable value thresholds Absolute pass rate of one standard deviation above or below the mean for level 1, level 2 and level 3 courses. Overall satisfaction/dissatisfaction with the quality of the course is equal to or above pre-determined thresholds (for the last presentation of the course surveyed) A detailed report is prepared for SEAG and a summary for the Learning and Teaching and Student Support Committee
© The Open University, Institute of Educational Technology 13 Some examples: via central academic units/ SEAG Good practice FAQs on student course forums Use of wikis, teleconferences and forum notes to support associate lecturers Extensive use of student and tutor online discussion which led to improved ICT skills among students Areas for improvement Student workload higher compared to similar courses Blended learning less well received compared to other new courses
© The Open University, Institute of Educational Technology 14 An example of action taken at course level A third level course team responded to student feedback on workload by: Making some material optional Changing the assessment strategy Improving the guidance on assessment Results Pass rate increased by 10% points and the Z score, although still negative, improved substantially in Additional changes in 2009:further advice on study skills and exam added to course website. Slight improvement in Z score but no change in pass rate.
© The Open University, Institute of Educational Technology 15 To summarise : Closing the feedback loop? Data tools Data tables Open comments Key Performance Indicators Academic reviews VCE / University Committees Strategic projects External audits Actions to improve Sharing good practice Collecting/measuring Student Retention Student Feedback
© The Open University, Institute of Educational Technology 16 Next steps Continue to increase the focus on identifying and monitoring action where improvement required. Continue to improve the identification and sharing of good practice through cross unit staff development. Continue to develop the evidence base e.g. use of data from the Virtual learning Environment (VLE). Explore a theme based approach to enhancement. Develop a model for student progression.
© The Open University, Institute of Educational Technology 17 The good news! Completion rates have increased in 2008/09 compared to 2007/08. The University has received consistently high ratings in the National Student Survey, 94% of respondents were satisfied in There are a number of strategic projects aimed at enhancing the student experience and increasing retention and progression.
Institute of Educational Technology The Open University Walton Hall Milton Keynes MK7 6AA
2012 SQA Central Training (New Centers) Venue – BISU, Beijing 9, Sept am– 5:00 pm Mary Gao.
ELA/Literacy K-2 Session 1: Developing Criterion-Based Feedback 1.
1 InCAS and Annual Report/KS1&2 Assessment Information Seminar for Principals September / October 2010.
Unit 2: Managing a business People in business Improving organisational structures Chapter 19.
Admissions Policy and HE Strategy Professionalising Admission to Higher Education in Further Education Conference Tuesday 2 March 2010 Annie Doyle, Senior.
GROWING SUCCESS Assessment, Evaluation, and Reporting in Ontario Schools First Edition Covering Grades 1-12.
Developing and Implementing Institutional Policy on PDP J University of Plymouth A bottom-up approach to PDPs, offering a choice to Departments : an evaluation.
Internal Quality Assurance seminar Common challenges among QAA ENQA Review Committee perspective Guy Aelterman Den Haag,15-16 June 2009.
School Standards Unit Making a Difference on Behaviour and Attendance An Action Plan for Improving behaviour and attendance to help raise standards.
Student Engagement: Indicators 5, 6 and 7 Dr Sarah Williamson Head of Learning and Teaching Support and Sarah Ingram Student Voice Officer
1 Session 3: Benefits Definition and Measurement v4.0 May 2009 eHealth Benefits Management Toolkit.
Key Stage 3 National Strategy Departmental evaluation and forward planning for Key Stage 3 science.
Assessment and feedback Principles, practice and technologies.
Capacity development & learning in evaluation Uganda Evaluation Week 19 th to 223 rd May 2014.
Thinking about student evaluation of teaching and using the SETU framework to improve teaching and learning Stuart Palmer and Dale Holt Institute of Teaching.
Page 1 First Call for Proposals Improving services in the informal areas, enhancing employability and promoting income generation Training Session (First.
Family Resources Survey Proposals for the treatment of unlinked FRS data* Valerie Christian & Philip Clarke.
Your Institutional Report 2014 – Step by Step General advice: For best results, connect to this Webinar using a high- speed connection. Visit https://admin.acrobat.com/common/help/en/support/
Wyoming Accountability Advisory Committee Scott Marion & Chris Domaleski Center for Assessment June 14, 2012.
Teacher Evaluation in Newark: Evaluator Training August 21 or 22,
HELPING THE NATION SPEND WISELY Jonathan Mackay Audit Principal Staying the course: The retention of students in higher education.
Heidi Good Practice Seminar 26 March 2009 London.
Selecting Outstanding Teachers for Level 4 Schools Spring 2010 Massachusetts Department of Elementary and Secondary Education.
EUROPEAN DRUG PREVENTION QUALITY STANDARDS EQUS conference, Brussels, June 2011.
1 Psychological Practical (Year 2) PS2001 Introduction Dr. John Beech.
1 Schools financial value standard Welcome. 2 Aims The aims of the day are to: provide guidance on the requirements of the Schools Financial Value standard.
1 Progress File Senior Manager Information Seminar Anne Cromie and Paul Wright March 2012.
Presentation title: 32pt Arial Regular, black Recommended maximum length: 1 line A Mandatory Management/Leadership Development Framework for Doctors Professor.
London Strategic Performance Network 26 November 2010 DOC G Transparency in outcomes: a framework for adult social care.
OSCR Survey Results (Survey conducted Oct / Nov 2003)
© 2016 SlidePlayer.com Inc. All rights reserved.