1. Enhancing the First Year Student Experience at MMU: Clearing a path & interpreting digital footprints
Derby | Sep 2014
Professor Mark Stubbs
Head of Learning & Research Technologies

2. Context | An interesting place to work!
Large, popular, multi-campus university
36,000+ students | 1,000+ courses | consolidating from 7 sites
Pursuing an ambitious transformation initiative
New Buildings, New Curriculum, New Admin Processes & Systems, New Learning & Mobile Tech, New Quality Processes…
80%+ of 8, /12 UG intake young from state schools / colleges

3. Context | Strategic improvement imperative
Recruit to target
League table rankings
Marketing &
Recruitment Processes C
Inform decision-makers
Reputation B
Improve satisfaction, retention & success
Learning, Teaching, Assessment & Personal Development Processes, Facilities
& Resources
Student Intake
(Aspirations, Attitude & Abilities)
Student Retention Success & Satisfaction
Resource allocation
2010 NSS Where to intervene?
All Year Numbers

4. CSI | Student Experience
Careful diagnosis of (diverse) learning experiences
Recurrent messages
NSS analysis: course organisation determines Q22
Mobile surveys: deadlines, timetables… (m-admin)
Focus groups: consistent, easy to find info
Great (and not-so great) expectations
“engaging, well-organised courses”
“inspirational tutors who know me”
‘Hygiene factors’
Stimulating pedagogies

5. CSI | Student Retention & SuccessB
Improve satisfaction, retention & success
Assumptions 1st years have requisite motivation, academic ability, adaptive capacity and resilience to
a) Sort  / £ /  //  /  so they can focus
b) Adapt study methods to new academic requirements
c) Perform to required standard
d) Establish supportive social / working relationships
Literature Journeys Wed 9, N31
Living
Learning
Social / Community
Interventions R&A / pre-entry, personalised info, early formative feedback
Review literature…
+ Multi-dimensional
+ Belonging
+ Habitus
+ Polishing
+ …
Reflect on…
+ Assumptions
+ Interventions
Examine data… + Focus Groups + Surveys + Journey journals
+ Observations + National data

6. Coordinating our responseB
Improve satisfaction, retention & success
Simplify
Unwieldy Curriculum
Great Online Experience (Seamless, personalised)
Great Learning Spaces
Great Teaching
EQAL
£350M

7. EQAL New Curriculum New Admin Systems & Processes
A coordinated strike for step-change improvement
In the current climate
Diminishing unit of resource
Everything depends on everything else
We are large and risk averse
but
New Curriculum
designing new units, …
New Admin Systems & Processes
personal timetabling, …
New Virtual Learning Environment
Moodle & myMMU web/mobile, Talis Aspire…
Programme Board chaired by DVC Student Experience
New QA & QE Processes
facilitating curriculum transformation

8. MISSION:
2010 Q1: Plan Programme & Projects Q2: Approve UG curriculum rules Q3: Develop smart curriculum capture forms Q4: Enter 800+ new L3 + L4 module Q1: Approve new modules Q2: Set up SRS, TT & VLE Q3: L4 curriculum & systems go live Q4: Enter new L5 modules… Q3: L5 goes live Q3: Entire new UG curriculum live

9. Smart forms
868 L3+L4 modules
Minimise
‘big ask’
Jul 2010

10. past exams, digitizations
Core+ VLE blueprint
deadlines, marks
enrolments m
resource lists
past exams, digitizations
sso links
timetable
Feb 2011

11. More familiar

12. Wrapping the institution around the learner
This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence
Wrapping the institution around the learner
Student ID
Timetable
Sync to personal device

13. Less early access than expected
Pre-entry access to timetables
Less early access than expected

14. Wrapping the institution around the learner
This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence
Wrapping the institution around the learner
Student ID (+ Unit code)
Deadlines / extensions / feedback return dates / provisional marks
Personalised submission sheet

15. Wrapping the institution around the learner
This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence
Wrapping the institution around the learner
Unit code
Resource list
Relevant resource

16. Wrapping the institution around the learner
This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence
Wrapping the institution around the learner
Unit code
11,000+ past exams uploaded
Past exam papers
Past exam paper

17. MyMMU App
m-admin

18. Summary | we cleared a path to the information our learners needed…

19. Are we in a better place now?
Quartile improvements in course org & learning resources in NSS
Engaging our students
Providing business intelligence
Automating our processes
30,000 regular users, 52m+ hits on Moodle, 26,000+ App registrations!
EQAL wins inaugural Guardian Student Experience Award!
10,000+ students post 40,000+ comments in our biannual student surveys
Submission tracking, Personal timetables, …

20. … we can now begin to interpret some digital footprints

21. Using the data Refining understanding of a problem space
2. Identify relevant data sources
3.Summarise individual data sources
4 Join on common identifiers
5 Prep the data for analysis
6.Analyse & visualise
1.Appreciate the issue
Refining understanding of a problem space

22. Understanding student satisfaction
2. Identify relevant data sources
3.Summarise individual data sources
4 Join on common identifiers
5 Prep the data for analysis
6.Analyse & visualise
1.Appreciate the issue
Satisfaction predictors
↗ Confidence
↘ Organisation
→Teaching
Student demographics
Entry profiles
Internal survey responses
Refining understanding of student satisfaction
Random Forests analysis
Sum entry points & qual type
Handle missing values, collapse categories…
Join on student ID

23. Understanding student success
2. Identify relevant data sources
3.Summarise individual data sources
4 Join on common identifiers
5 Prep the data for analysis
6.Analyse & visualise
1.Appreciate the issue
Predictors
YNG NSSEC4-7 Entry Tariff NSS
HESA PI
League Tables
Refining understanding of student continuation
Random Forests Regression Tree
Add UKPRN
Handle missing values, collapse categories…
Join on UK PRN

24. Conditional Inference Trees
Party Library

25. Limitations Confirmation bias Meaningful comparison
Quantitative V Qualitative
Data quality
Correlation ≠ causation
“Not everything that can be counted counts.
Not everything that counts can be counted.”
William Bruce Cameron
Reality of our current data warehouse build: careful thought about error handling + painstaking checking of joins!

26. Continuous monitoring & improvement
MDL
SRS TT
Data Warehouse
Analysis Cubes
CMI dashboard for all programme leaders for Sep 2014

27. CMI for Programme Leaders

28. CMI for Programme Leaders

29. Change management lessons
Understand important dynamics
Be clear what needs to be done
Set high-level goals
Understand where change is required
Be bold and plan holistically
Ensure you’ve got the right team to drive
Make ‘big asks’ as small as possible
Make it stick with reinforcing change
Find out more!
Curriculum Design
Assessment & Feedback