1. Copyright Shanna Smith & Tom Bohman (2003). This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors.

2. Learning about your IT community via online surveys Shanna E. Smith, Ph.D. Thomas M. Bohman, Ph.D. ITS Research Consulting The University of Texas at Austin

3. Four key topics Defining and maximizing survey benefits Online surveys: Advantages and disadvantages A multiple-method approach Realizing survey benefits

4. Potential Survey Benefits Documents services –Usage of current services –Perceived importance of services Evaluates services –General satisfaction –Problem areas Informs managerial decisions –Reactions to policy –Needs assessment and marketing Communicates care and concern

5. Designing to Maximize Benefits Buy-in is required –Sponsor –Other stakeholders –Developer Questions must be relevant and useful Questions should adhere to social science research guidelines and practices Sampling and data collection must be unbiased to allow generalization of results

6. Possible Survey Methods Telephone MailOnline

7. Online Surveys Advantages: –Inexpensive, efficient data collection –Questionnaire flexibility Disadvantage : –Possibility of sample bias

8. Respondent currently uses austin.utexas.edu or forum.utexas.edu uts.cc.utexas.edu a non-UT fee based account and ‘my other e-mail account’

9. The user answers satisfaction questions only regarding the services that are currently used.

10. Online Surveys Advantages: –Inexpensive, efficient data collection –Questionnaire flexibility Disadvantage: –Possibility of sample bias

11. Generalizing to your Population Population: all the people to whom you want to apply your survey results. For example: All students who use computer labs at your University All student residents at your University All faculty teaching graduate courses at your University

12. A Population

13. A representative sample

14. A biased sample

15. Who might be more likely to respond to your online survey? People who more widely use computers and the Internet People who already use your services

16. A multiple-method strategy Choose a random sample Send an initial contact through postal mail E-mail all respondents with an invitation to complete the survey online For those without valid e-mail addresses, send a paper version of the survey Send reminders! For those who do not respond within two weeks, attempt a telephone interview

17. ITS Survey 2002 Responses

18. Realizing Survey Benefits Report results in timely, appropriate fashion –General results –Individualized reports for each stakeholder Acknowledge public feedback –Publicly available results –Press releases Keep interest piqued

19. ITS Survey 2002 Results Areas Usage and Satisfaction: Familiarity, usage, satisfaction with core services, e.g., Internet access, e-mail, Help Desk Computer Labs: Usage of student computer labs and satisfaction with lab software, hardware, proctor helpfulness, etc. Technical Support: Types of computer problems experienced by faculty and staff, and where they go for help Computer Use: Purchase information, operating system, age, and other details regarding home and office computers Training: Necessity for various types of technical training and preferences regarding training methods UT Direct: Usage of UT Direct portal and its various components Needs Assessment: Importance of e-mail, the campus network, and other services to respondents' work at UT Computer Security: Familiarity with ITS acceptable use policies, usage of anti-virus software, password sharing behavior Specific Issues: Questions of interest to particular Directors, e.g., familiarity with Microsoft Campus Agreement.

20. Example: the Help Desk and Instant Messaging Faculty and staff were more likely to use the Help Desk, but all groups who used it were uniformly satisfied with the service. Over half of the UT population uses Instant Messaging, but most of them are students.

21. Instant Messaging Undergraduates are most likely to use IM Undergraduates were most satisfied with IM (94% said they were ‘satisfied’ or ‘very satisfied’).

22. For more results... See our ITS Survey 2002 website at http://www.utexas.edu/its/surveys/2002/