Why Collect Data? November 2, 2007 Colleen Cook, Dean of Libraries, Texas A&M University NISO Usage Data Forum Dallas, TX

Why do I do assessment at Texas A&M?

Spending millions wisely Learn from mistakes Quality management Accreditation Good citizens of associations The right thing to do In the end….a Culture of Assessment

What do I assess? …just about everything I can

What are the characteristics of qualitative methods? The observer/researcher inseparable from the study Consists of a set of interpretive practices that try to make sense of a cultural context Data sources: field notes, interviews, conversations, photographs, recordings, and memos to the self Study a natural setting, attempting to make sense of, or to interpret, phenomena in terms of the meanings people bring to them

How do I assess at Texas A&M?

Multiple Methods of Listening to Customers Transactional surveys* Mystery shopping New, declining, and lost-customer surveys Focus group interviews Customer advisory panels Service reviews Customer complaint, comment, and inquiry capture Total market surveys* Employee field reporting Employee surveys Service operating data capture Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000). Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

Total Circulation Note. M. Kyrillidou and M. Young. (2005). ARL Statistics Washington, D.C.: ARL, p.6.

Climate for Customer Service

Leadership Climate

Climate for Interpersonal Treatment

The End woof

LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred

How did qualitative methods evolve? Beginnings in Sociology: 1920s and 30s in the “Chicago School;” in Anthropology: in the studies by Boas, Mead, Benedict, Bateson, Evans-Pritchard, Radcliffe-Browne, and Malinowski Through seven moments (Denzin & Lincoln, 2001) Today: influences of poststructuralism and postmodernism from textual studies

How do qualitative and quantitative methods differ? Multiple realities, not a single one “out there” to be discovered Value laden, subjective rather than objective Seeks closeness with the investigated through interviewing and observation rather than abstract relationships Inductive rather than deductive Purposeful sampling chosen for diversity rather than random sampling

Thick descriptions rather than crisp and terse background information Comfort with contradictions, ambiguity Representations include ethnographic prose, historical narratives, first-person accounts, still photographs, life histories, biographical and graphs, third- person narratives autobiographical materials rather than mathematical models, statistical tables

What data are collected by the qualitative researcher (researcher as bricoleur, montage maker)? Case studies, personal experience, introspection, life story, interview, artifacts, cultural texts and productions, observational, historical, interactional, and visual texts, statistics that describe routine and problematic moments and meanings in individuals’ lives. Inherently multimethod in focus: triangulation

What fields of study are included in qualitative methods? Ethnomethodology, phenomenology, hermeneutics, feminism, deconstructionism, ethnography, interviews, psychoanalysis, cultural studies, survey research, participant observation

Qualitative research: the method of choice

LibQUAL+ as a research example

Purposeful sampling Unstructured interviews - “conversations with a purpose” Peer review Immediate and continuous analysis informing further exploration Journal Member checks Audit review

Establishing Trustworthiness: A Comparison of Conventional and Naturalistic Inquiry CriterionConventional Term Naturalistic Term Naturalistic Techniques Truth valueInternal validityCredibilityProlonged engagement Persistent observation Triangulation Referential adequacy Peer debriefing Member checks Reflexive journal ApplicabilityExternal validityTransferabilityThick description Purposive sampling Reflexive journal ConsistencyReliabilityDependabilityDependability audit Reflexive journal NeutralityObjectivityConfirmabilityConfirmability audit Reflexive journal Adapted from Lincoln & Guba, 1985.

Randolph High School Stability Within Transition Index CardPhotos Artifacts B: Base D:District S: School C: Calendar A: Annuals N:Newspaper I. A Rich History (p.44) A. Location of the base/the TAJ B. Base description/access C. Base housing description D. History of the base E. Mission of the base/Education of personnel F. History of the school district/funding II. The High School (p.47) A. Perimeter Road 1. Description 2. Stadium 3. Trees B. Campus description C. High school office description/secretaries D. Teachers/ lounge description E. Patio area description F. Main classroom building description/display cases III. The Principal—”Do what’s best for the kid!” (p. 51) A. Physical description 1. “Conservative” 2. “Clean-cut” 3. “Honest” 4. “Country Gentleman” 5. “Western-cut clothes” 6. “Never having a hair out of place” 7. “Trim” IA,IB IB IB,IC IE(1-3) IF(1-2) IIA 8,9 8 IIB(1-2) IIC(1-6) IIE IIF IIIA2(1-2) IIIA2(1) IIIA2(2),IIIA3 IIIA4 IIIA2(1) IIIA6(1-2) 4,5,6 6 B1,D2 7 10,11,14,80,81 65,66 67, , ,49 2,90 90,92 B1 A(1,2),D2,P32 N(34,35,41) A(1-5,12) A16 The Audit Trail Excerpted from Skipper, 1989.

Affect of Service “I want to be treated with respect. I want you to be courteous, to look like you know what you are doing and enjoy what you are doing. … Don’t get into personal conversations when I am at the desk.” Faculty member

Library as Place “One of the cherished rituals is going up the steps and through the gorgeous doors of the library and heading up to the fifth floor to my study. … I have my books and I have six million volumes downstairs that are readily available to me in an open stack library.” Faculty member

Library as Place “I guess you’d call them satisfiers. As long as they are not negatives, they won’t be much of a factor. If they are negatives, they are a big factor.” Faculty member

Information Control “By habit, I usually try to be self-sufficient. And I’ve found that I am actually fairly proficient. I usually find what I’m looking for eventually. So I personally tend to ask a librarian only as a last resort.” Graduate student

Information Control “…first of all, I would turn to the best search engines that are out there. That’s not a person so much as an entity. In this sense, librarians are search engines [ just ] with a different interface.” Faculty member

Dimensions of Library Service Quality Empathy Information Control Responsiveness Symbol Utilitarian space Assurance Scope of Content Ease of Navigation Self - Reliance Library as Place Library Service Quality Model 3 Refuge Affect of Service Reliability Convenience Timeliness Equipment

So…..what did we do at Texas A&M?

What are we doing now?