1. CSC Alumni Survey Harry Bui Jared Guan Ryan Guest

2. Outline  Background  Goals and Objectives  Procedures  Things That We Kept in Mind  Data Collection Outcome  Changes to Survey  Web Survey  Conclusion

3. Background  Receive feedback on CSC program  Given to CSC alumni and seniors  About 50 questions, Four pages  20 to 30% response rate

4. Goals and Objectives  Not to determine better incentives  Evaluate current CSC Alumni Survey  Prevent users from quitting while taking the survey  Incorporate team assignments with project  Redesign, redevelop, and reorganize  Convert paper version to web survey

5. Procedure  Data Collection  Data Analysis  Application of Results

6. Procedure: Data Collection  Setup survey stations  Users took survey  Question users for feedback

7. Procedure: Data Analysis  Read user feedback  Analyzed mouse metrics  Analysis of observations

8. Procedure: Application of Results  Shortened survey  Removed redundancy  Clarified questions  Restructuring

9. Things That We Kept in Mind  Information is not lost  Reduce redundant information  Survey shouldn’t be longer  Organize logically  Use the feedback/observation from data gathering lab  Allow participants a way to submit the survey

10. Data Collection Outcomes Positives  Few leading questions  Very detailed questions  Gathered a lot of data Negatives  “Too long” / “I quit”  “Redundant information”  “Too wordy”

11. Changes  Moved away from WebSurveyor  Reordered scales to comply with common web practices  Grouped logical information and combined questions  Replaced text areas with radio buttons where appropriate

12. Survey Comparison  Original Survey Original Survey Original Survey  New Survey New Survey New Survey

13. Benefits of Web Survey  Web vs. Mail Response rate 20 percent higher than mail Response rate 20 percent higher than mail Overall cost is lower Overall cost is lower  Web vs. Phone 33% (web) vs. 18% (phone) said that they would definitely participate in future studies 33% (web) vs. 18% (phone) said that they would definitely participate in future studies “No Response” answered more frequently in phone surveys “No Response” answered more frequently in phone surveys Conway, Malcom. Web Survey Guidelines http://www.learningcircuits.org/2004/apr2004/conway.htm

14. Web Survey Guidelines  Questions are visible on the same screen and not split between screens.  Use page numbers or a progress bar graphic for long surveys.  Limit the amount of information on a screen to minimize scrolling.  Don’t force the respondent to answer a question before the next one is shown.  Allow respondents to interrupt and reenter the survey.  Limit the use of graphic images. Conway, Malcom. Web Survey Guidelines Conway, Malcom. Web Survey Guidelineshttp://www.learningcircuits.org/2004/apr2004/conway.htm

15. Web Survey  Web Survey Web Survey Web Survey

16. Conclusion  Discovered a lot problems with old survey  Verified with data collection  Improved usability after data analysis

17. Questions?