Getting the Results You Want: A Response Rate Strategy
Student Evaluation, Review and Reporting Unit (SERRU) Contents 1. SERRU’s role 2. Survey Implementation 3. What we achieved in response rates –2 snapshots 4. How we improve our methods. 5. How we increase our response rates. 6. How we impress upon our stakeholders.
SERRU supports the University to…… Develop a strong evaluative culture to improve the student and staff experience at the University of Tasmania Establish strategic relationships with a range of stakeholders to ensure quality processes and practices are maintained and improved across the University Provide staff support on a range of data needs, priorities and projects Collaborate with people internally, nationally and internationally to build partnerships in the quality of teaching and learning, standards and review
What we do at SERRU with our data skill sets…. Survey implementation and analysis Course and unit data evaluation Data analysis Research reporting Benchmarking Standards Policy development Project management
Survey Implementation Develop a strong evaluative culture to improve the student and staff experience at the University of Tasmania External Surveys Australian Graduate Survey (AGS) University Experience Survey (UES) International Student Barometer (ISB) Ad hoc national surveys Internal Survey eVALUate - the UTAS online student feedback system on units and teaching
Australian Graduate Survey - Timeline of Response Rates 2009 – 2013
Building awareness and engaging students and staff through… Workshops with Schools and Faculties Student Advisers’ Newsletter Student Union Representative Social Media Engagement MyLo – Online Student Learning Environment Electronic Notice Boards
Building awareness and engaging students and staff through… Posters around high traffic areas around campus Flyers distributed at Graduation Ceremonies Alumni Newsletter Bulletin on the Graduation Webpage Presence on SERRU External Surveys and eVALUate webpages with links to the respective survey sites
Maximizing response rate potential through….. Initial email and letter invitation from relevant University management Telephone Reminder of 300 calls per day at commencement of survey Follow up telephone reminder towards the end of the survey round 2 to 3 SMS reminders per round 2 email reminders from Survey Manager Regular response rate updates to Student Advisers Offering an incentive Using the correct terminology e.g. International Student Barometer Survey (for international students) Maintaining a Survey Register
Communicating results to stakeholders…… Council Senior Management Team (SMT) Schools and Faculties Division of Students and Education (Divisional Management Team) Students
For more information on the University of Tasmania’s response rate strategy Please contact: Gheeta Chandra Krishnan External Surveys Coordinator Student Evaluation, Review and Reporting Unit (SERRU) Gheeta.Krishnan@utas.edu.au Gheeta.Krishnan@utas.edu.au http://www.utas.edu.au/student- evaluation-review-and-reporting- unit/
Brett Ryan Survey Manager - Griffith University
The Australian Graduate Survey captures the thoughts, opinions and outcomes of recent graduates from all Australian universities. Your valuable feedback is completely CONFIDENTIAL and it will: -inform future students on study, career, & employment outcomes -assist the federal government with higher education policy development -inform program improvements here at Griffith -qualify you to receive an engraved Griffith commemorative pen -and enter you into a draw to win other great prizes too! Look out for an email with your personal survey link in November 2014.
Email Prep Spending time before you launch is critical Email hierarchy – D&A, PS update, home, business, uni account Cleansing the list: –Check priority 1s Is it a GU account? Are they a continuing student or a staff member? Is it an international student’s education agent? –Check for duplicates –Search for domain names
Other emails…. Heads of School Pro Vice Chancellor (International) Deans (Learning & Teaching) Survey Manager –Partial completers –Staff non-responders –Prize draw deadline looming We use multiple senders, with multiple messages.
SMS & Paper SMS Postal surveys are expensive so it is important to have good data –We use a similar process as to how we attack the email file International students receive two pieces of Direct Mail; one to their Australian address and one overseas We use a different PURL for people who complete the survey online as a direct result of receiving the letter (to track its effectiveness) And we also use this….
The Final Push Telephoning fieldwork –Centralised Telephone Service via GCA The final, final push (read into panic) –the next 200 responses receive a guaranteed… And if you are really desperate…. –Guaranteed offer extended! or –Try again on the phones yourself….but you can only do the GDS.
AGS Collection at UTS Graduations Alberto Mendez Survey Coordinator Planning and Quality Unit University of Technology, Sydney
The typical AGS situation In AGS 2008 only achieved 48% response rate … thus unable to publish data outside UTS! In AGS 2009 and 2010 achieved 51%, 52% but … required huge effort to achieve target (printing and posting 2 or more sets of paper forms) not able to drill down very far into data Must be an easier way …
The rather fortuitous UTS solution AGS commencement dates align well with Graduations: AGS = April 1 and October 1 UTS Graduations = start of May and start of October Main difficulties: reluctance from Graduations Office to fit us in possibly disrupt existing process, delay ceremonies Trial run agreed on, successful*
Collection mode structure Thus UTS able to increase AGS collection modes: hardcopy (postal + Graduations) and online telephone interviews also briefly adopted, since discontinued So, current order of modes: OctoberApril 1. hardcopy (Graduations)1. online 2. hardcopy (postal)2. hardcopy (Graduations) 3. online3. hardcopy (postal) 4. telephone
In the lead up to Graduations Preparation work: request graduate list from Student Systems prepare email communications (push incentives) compile a “mark-off” spreadsheet and assign each graduate their oAGSID print individualised hardcopy forms and letters and stuff into addressed postal envelopes align graduates and ceremonies: sort hardcopy forms into individual ceremony’s box prepare list of graduates for individual ceremony and mark- off online completions (April only)
At Graduations UTS setup: component of pre-ceremony official process (1 hour): registration gowning AGS Alumni AGS physical space: 5 tables, each with 6 chairs AGS staff: 2 people, spruiker (promoter, welcomer) and form provider AGS completion time: typically 5-10 minutes (few if any blanks, incompletes) Graduates + AGS + chocolate = happiness!
Conclusions Now seamlessly part of pre-ceremony process, graduates go with the flow, many see it as a ‘must do’ They don’t do it for chocolate but appreciate the gesture Small number of latecomers with no time left to complete AGS, asked to take AGS and return it in post Small but annoying features of setup: relatives/friends wanting to spend time with them (sometimes hard to keep them out of AGS area, seats) pen stealing
Final thoughts Great mode for maximising responses but requires considerable time investment from dedicated staff – use of casual, un-invested staff can reduce effectiveness Graduates overwhelmingly happy, good time to target them for participation, but possible mode effect on CEQ scores?