Presentation on theme: "ULS Online Discussion: Surveys in Libraries. ULS Facilitators: Julie Watson Research Librarian Marywood University Emily Daly Head, User Experience Department."— Presentation transcript:
ULS Facilitators: Julie Watson Research Librarian Marywood University Emily Daly Head, User Experience Department Duke University Rebecca Blakiston Instructional Services Librarian University of Arizona
Actionable Survey - Overview Process Improvement What is an Actionable Survey? Survey Design
Overview of Process Improvement Develop/Define a question/problem Review information Gather your data Measure your process Analyze/test your data Implement your changes Feedback or summary
Actionable Survey Emphasis – Survey questions “What will we do with this information?” Ex. Likert Scale responses Feedback – Plan and share feedback & summaries “This is what you told us you wanted. This is what we’re delivering/changing.”
Actionable Survey Service Example - Action Gap Survey 1 What do we do well? List of 10 services (multiple choice: choose 3 services) What do we need to improvement? (multiple choice: choose 3 services) Which service(s) are important to respondent? (rank all 10 services) 1 Cravendo, J., & Sandvig, B. (2003). Survey for Action, Not Satisfaction. Quality Progress, 36 (3), 63-68.
Survey Design Tips Don’t try to solve World Peace in a Survey Keep it simple and focused Edit out “promotional” questions No Silver Bullet, or magic formula for questions Do ask demographic questions Ex. Undergraduate or Graduate question: will you change a service or product based on the response? Will your survey results summaries be different or specific to separate groups? If you can, do meet with [Statistics or Sociology] faculty to discuss and review your survey/data/methodology For big changes in services/products/space, check and consult with other [customer] data sets to see
SURVEYS: CONCEPTUALIZATION AND CONSTRUCTION Dr. Jason Martin Stetson University firstname.lastname@example.org
Bad Surveys What are some experiences you had with “bad surveys?” What made them bad?
Survey Blueprint What do you want to know? Information needs Stay on topic Use a theoretical framework or model Information Literacy Standards Customer Satisfaction Parts of a Service or Topics in a Course
IL Standards TopicQuestions Information Needs2 Access Information6 Evaluate Information4 Use Information2 Ethically Use Information1 Information Literacy15
Types of Survey Questions Likert Scale Select One Open-Ended Drop Down Menu Check all that Apply Continuum of Data
Likert Scale This webinar is interesting. Disagree (1) Somewhat Disagree (2) Neither Agree Nor Disagree (3) Somewhat Agree (4) Agree (5) NA Poll Question! Does “Neither Agree Nor Disagree” have a value?
Other Questions Open Ended Avoid overuse Value of the answers is hit or miss Good catchall at the end Check All that Apply Avoid overuse Primacy effect
Other Questions Drop Down Menu Make sure all the possible options are present Continuum of Data No overlapping One year to five years; five years to ten years; ten years to fifteen years Less than one year to five years; more than five years to ten years; more than ten years to fifteen years
Writing Survey Questions Poll Question!! Use clear and concise language. Do not confuse respondents or leave words or phrases open to interpretation.
Writing Survey Questions Poll Question!!! Avoid negative questions. Answer in the negative when you mean the positive.
Writing Survey Questions Did you use the library on 17 November 2012? Too specific Do not make survey respondents go find information. Ask questions they can easily answer.
Survey Layout Clear Navigational Flow Avoid Clutter Avoid Jumping Around from Question Type to Question Type Place like question types together Not necessarily subject type Place Demographic Information at the End Somewhat controversial
AN ALTERNATIVE TO LIBQUAL Rick Stoddart Assessment Librarian Oregon State University email@example.com
Poll Question: What tool does your library mainly use to gather patron feedback about library services? A. A LibQUAL product B. LibSat survey by Counting Opinions C. A survey built in-house D. Other (some other tool or technique) E. Our library doesn’t formally gather patron feedback
Pilot Learning Outcome ACRL STS standard The information literate student determines the nature and extent of the information needed. Pilot outcome Undergrads will be able to employ information access skills learned while at MIT in order to identify appropriate sources of information in their own field and in unfamiliar subject areas.
Process Office of Institutional Research COFHE surveys MIT Surveys: Pre-freshman survey Enrolled student survey Senior Survey Alumni Survey Triennial MIT Libraries’ Survey
Question to include in MIT Surveys Learning Outcome: Undergrads will be able to employ information access skills learned while at MIT in order to identify appropriate sources of information in their own field and in unfamiliar subject areas Based on what you know now, how well do you think your undergraduate experience prepared you to: Select scholarly or professional information from a variety of sources.
Results 2010 Senior Skill Pearson Coeff*2012 Senior Skill Pearson Coeff* Selecting scholarly or professional information from a variety of sources1 1.00 Make firm decisions and take action even if some of the facts about the best choice are not clear0.587 Conducting scholarly research0.539 Deliver on all elements of a difficult job or project you agreed to do within the accepted time frame0.531 Think analytically and logically0.304Thinking analytically and logically0.398 Acquire broad knowledge in the arts and sciences0.287 Acquiring broad knowledge across a number of fields0.393 An ability to use the techniques, skills, and modern tools necessary for my profession0.392 Synthesize and integrate ideas and information0.340 Synthesizing and integrating ideas and information0.379 Gain in-depth knowledge of a field0.388In-depth knowledge of a field or discipline0.378 * The most frequently used method to determine the strength of the relationship between two variables is called the Pearson Correlation Coefficient. The closer the number is to 1, the more strongly the two variables relate to one another.
Takeaways Actionable surveys require that you understand what you intend to do with the data, and that you report back to the respondents indicating how you acted on it. Survey blueprints help to identify your information needs and keep you on topic. Keep your questions and survey simple. An advantage of vendor surveys is that they compile data into meaningful reporting tools, helping to provide actionable data. Incorporating a library question into a campuswide survey enables you to get longitudinal user feedback and to focus on life-long learning skills.
Thank you for joining us! ULS blog: http://www.acrl.ala.org/ULS/http://www.acrl.ala.org/ULS/ ULS Facebook: https://www.facebook.com/acrl.ulshttps://www.facebook.com/acrl.uls ULS Midwinter events: http://alamw14.ala.org/sessions/all?keys=uls http://alamw14.ala.org/sessions/all?keys=uls