2. Student Surveys - Best Practice
Eric Bohms
“If you can't measure it, you can't manage it”
Peter Drucker – Drucker Institute

3. Student Surveys - Best Practice
Session Agenda:
Best Practice Survey
Paper vs. Online: the ongoing debate
Best practices for HE survey administration
Ideas for driving cost and efficiency saving
Why surveys? The ‘Audit’ culture vs. the ‘Feedback’ culture

4. Student Surveys - Best Practice
Population: UK Quality Managers and University Administration staff at 58 institutions 2010

5. Paper vs. Online Advantage Disadvantage Instant feedback
High response rate
Easy for the student
Low deployment cost
Easy to administer
Location independent
Survey Administration
Printing costs
Hand writing
Low completion rates (statistically significant)
Strong opinions likely
Amend green thing
Amend arguments 5

6. Online Surveys Paper Data Capture Hybrid Add-on Survey Outsourcing
The Debate Continues...
Online Surveys
Paper Data Capture
Hybrid
Add-on Survey
Outsourcing
Amend green thing
Amend arguments 6

7. Things to Consider When Managing Surveys in Higher Education: Survey Best Practice
Amend green thing
Amend arguments 7

8. Survey Best Practice Use in-class time Treat survey like an assignment
Give proper time to complete
Amend green thing
Amend arguments 8

9. Most likely to complete
Survey Best Practice
Paper: In class surveys: ‘herd Instinct’
Most likely to complete
Satisfied
Dissatisfied
Amend green thing
Amend arguments
+30% higher response with in-class paper surveys over Online surveys 9

10. Most likely to complete
Survey Best Practice
Online: In class surveys: ‘herd Instinct’
Most likely to complete
Satisfied
Dissatisfied
Amend green thing
Amend arguments
Same principle: higher response with in-class Online surveys over to participants 10

11. Survey Best Practice
Amend green thing
Amend arguments 11

12. Why is obtaining high response rate important?
Survey Best Practice
Why is obtaining high response rate important?
Institutional Research / Overall Student Satisfaction
Survey Universe
The University
Amend green thing
Amend arguments
Possible to draw some conclusions from a low response sample 12

13. Why is obtaining high response rate important?
Survey Best Practice
Why is obtaining high response rate important?
Course & Module Evaluation
Survey Universe
A 25% response rate from a class of 20 is only 5 students who’s opinions may differ greatly
The Course
Amend green thing
Amend arguments
The challenge is gaining representative data from each and every course or module 13

14. Course & Module evaluation is not easy
Survey Best Practice
Course & Module evaluation is not easy
Its best in evaluating a course and the teaching to gain a majority opinion 50%+ response
It is important to be able to compare to show improvement (against dept, university average, over time)
Online survey systems fall short - average 15-30% response rate
Amend green thing
Amend arguments 14

15. Survey Best Practice Other considerations
Gain stakeholder buy-in as the key to participate – The ‘myth of survey fatigue’
Create surveys that make sense and report on only what you need to know
Reinforce the concept of anonymity to get honest feedback
Amend green thing
Amend arguments 15

16. Efficiency and cost savings
Ways to drive efficiency and cost savings for surveys in Higher Education
Amend green thing
Amend arguments 16

17. #1 Centralise survey administration
Efficiency and cost savings
#1 Centralise survey administration
Avoid ‘academic run’ quality enhancement
Run research projects on a common platform
Know what surveys are being run at all times
Amend green thing
Amend arguments 17

18. Efficiency and cost savings
Amend green thing
Amend arguments 18

19. Efficiency and cost savings
Amend green thing
Amend arguments 19

20. Individual Academic with 5 courses = 1 day per semester
Efficiency and cost savings
Academic
Survey Design
Data Capture
Analysis
Deploy Survey
Academic run survey admin:
Amend green thing
Amend arguments
Individual Academic with 5 courses = 1 day per semester 20

21. Department with 50 academics
Efficiency and cost savings
Dept 1
Academic run survey admin:
Amend green thing
Amend arguments
Department with 50 academics 21

22. 20 departments / 1000 instructors = 1000 teaching days per semester
Efficiency and cost savings
Arts
Business School
Engineering 1 1 1
Law School
Languages
Sciences 1 1 1
Amend green thing
Amend arguments
20 departments / 1000 instructors = 1000 teaching days per semester 22

23. Efficiency and cost savings
Main point:
Funding cuts to teaching means less time and more work for academics
1000+ teaching days per semester is better spent teaching rather than on survey admin
Amend green thing
Amend arguments 23

24. Efficiency and cost savings
3 weeks .5
Centralised survey admin:
Amend green thing
Amend arguments
Best practice example: London School of Economics
23 Departments /1000+ staff / 9000 students
Complete survey administration 15 days per semester
(Deployment: 4 days / Data Capture: 10 days / Reporting: 1 day) 24

25. #2 Create standardised questionnaires
Efficiency and cost savings
#2 Create standardised questionnaires
Develop compulsory question sets for lectures and modules
Focus on teaching & learning and the student experience
Biology
English
Teaching Quality & the Student Experience
Amend green thing
Amend arguments 25

26. #3 Standardise core reporting functions
Efficiency and cost savings
#3 Standardise core reporting functions
Develop and implement a simplified reporting strategy
Supply stakeholders only with what they need
Standardised reporting allows for comparison to show trending over time
Amend green thing
Amend arguments 26

27. #4 Utilise advanced technology
Efficiency and cost savings
#4 Utilise advanced technology
End of the day:
Surveys can be a powerful tool for making decisions and transforming an organisation
If you are invested in a survey system that produces low response rates it is not going to meet the diverse requirements of Higher Education institutions
Survey administration should be a second job not a day job
Amend green thing
Amend arguments 27

28. Quality Assurance (Audit)
Why Surveys?
Quality Assurance (Audit Culture)
Quality Enhancement (Feedback Culture)
Quality Enhancement
Quality Assurance (Audit)
Inhibits learning from outcomes
Promotes incremental improvement of academic practice
Discourages risk taking (low risk appetite)
Encourages and requires frank reporting
Facilitates transformational change to surface & deep conditions for learning
Supports and manages risk taking
QAA
EESC OFFICER
Amend green thing
Amend arguments
External Examination Sub Committee Officer
“(There is a need for) a more holistic view of how a University’s administration and procedures affect a department’s teaching.”
Sue Wright, Enhancing the Quality of Teaching in Universities 28

29. Quality Assurance vs. Enhancement
EvaSys is designed to provide a ‘holistic’ platform to drive quality enhancement efforts across the organisation facilitating continuous improvement
Develop
Process
Implement Process
Evaluate Process
Plan for Outcomes
Amend green thing
Amend arguments 29

30. Student Surveys - Best Practice Eric Bohms Presented by: Philip Wyse
Conclusion
Student Surveys - Best Practice
Eric Bohms
Presented by: Philip Wyse
Questions? Thank you for your time!!
EvaSys Best Practice Blog: