1. The Survey as an Assessment Method: Why, when and how surveys provide evidence to inform decision-making Carrie Towns, Office of Institutional Research and Planning and the Information Services Assessment Council April 13, 2006

2. assessment an ongoing process in which services, resources and performance are measured against the expectations of users, and improvements are made to satisfy user needs effectively and efficiently.

3. What do we need to know?

4. Who can tell us?

5. How can we get the information?

6. What will it enable us to do?

7. How much will it cost?

8. Definition of Survey An assessment tool/system for collecting information that is used to describe, compare, and explain knowledge, attitudes, and behavior of a defined group or groups of respondents.

9. Conduct a Survey When: Need input from a large, well-defined group. Have a set of focused questions developed to meet a specified objective. Results will inform specific decisions. Time and other resources permit.

10. Characteristics of a Good Survey Specific objectives Straightforward questions Sound research design Appropriate resources

11. Types of Surveys Information gathering attitudes/opinion/behavior Market research tools Public relations tools Educational tools

12. Survey Process Set survey context – specify objectives Establish target audience – sampling frame Determine mode Prepare cost estimate Establish tentative calendar Design survey

13. Survey Process cont. Develop questions Plan for analysis Field test Administer survey Summarize and interpret data Report results

14. Survey Context – Specify Objectives Who is asking for the information? What do they want to learn? How will the information be used? Starting point – existing instrument?

15. Establish Target Audience/Sampling Frame Who can provide the information? Type of analysis needed – how detailed Number of subgroups of interest Plans for follow-ups

16. Modes of Surveys Web Paper and pencil Telephone Focus Groups

17. Prepare Cost Estimate OIRP professional time $25/hour Commercial tools and associated costs Other costs

18. Establish Tentative Calendar Development Field testing Administration Programming and analysis Reporting

19. Design Survey Length and layout Organization Question formats

20. Characteristics of Good Questions Make sense to the respondent Are concrete Use conventional language Avoid emotionally-charged language Avoid negative phrasing Ask for only one piece of information Have a specific purpose

21. Plan for Analysis Presentation of data drives item development Different audiences require different level of analysis

22. Field Test/Revise and Fine Tune Instrument Test drive instrument Administer Survey Data collection phase

23. Summarize Data Examine the data Run preliminary analysis Dig in – interpret and draw conclusions Report Results Written/oral

24. Information Services Assessment Council members Susanne Clement, Libraries Jill Glaser, IT Ryan Papesh, NTS Thelma Simons, IT John Stratton, Libraries Bill Myers, IS

25. Call on ISAC members to: Consult, advise and assist in the development of assessment initiatives. Identify other campus resources for assessment-related services. Provide oversight and assure coordination with other IS assessment activities.