1. Observation and Usability Studies Ericka Brunson Dorothy Hirsch Mollee March Joanna McCloud Denise Tiller

2. Focus on human behavior & actions Part of mixed-methods approach Naturalistic: how people really behave What they have in common

3. Test validity of other findings Study specific processes or behaviors Access behaviors of those unable to communicate Observation studies can

4. Three types of studies Participant Observation Reactive Observation Unobtrusive Observation

5. Study reference transactions Analyze patron/librarian behavior Study patron use of resources and tools Library uses

6. a thorough study over time conducted in a natural environment when the observer becomes part of the group being studied rarely used by libraries because of expense in time and money the basis of most cultural anthropological/ethnology group studies Participant observation is

7. Only studies behaviors, not cognition Prone to research bias/misconceptions Can be expensive, both in time and money Ethical considerations Limitations

8. How information is obtained What is done with the information Do no harm: emotionally mentally socially economically Ethical issues

9. Are you watching me right now…or am I watching you? Reactive observation

10. A “real world” observational research method that studies individuals and groups of people in a natural, controlled setting Participants are aware that they are being studied. What is reactive observation?

11. Behavior reflects aspects of social desirability Observations can be manipulated to fit the research desig n Reactive observation positives

12. Individuals are reacting to observation External validity Reactive observation negatives

13. Continuous monitoring Time allocation Common methods of reactive observation

14. Involves studying the behavior of participants and recording as much as possible. Works best in organizational settings such as: the workplace educational institutes when monitoring nutrition (exercise, food intake, etc.) Continuous monitoring

15. The “Hawthorne Effect” states that “workers react to the attention they are getting from the researchers and in turn their productivity increases.” The “Hawthorne Effect”

16. Done at random Researcher will choose the time and place to record activities before the researcher’s presence is known Time allocation

17. Descriptive Inferential Evaluative Variables to consider in evaluating observations

18. Subjects are unaware that they are being observed and researchers do not intrude on the study. Unobtrusive measures

19. Behavior trace studies Disguised field observations Types of measures

20. Natural behavior of subjects Reduces bias Unobtrusive positives

21. Researcher has little control over type of data collected Not always a usable method Subjects unaware of data being gathered Other ethical issues Unobtrusive negatives

22. Tests system performance with typical users Many library applications Quick, easy, inexpensive Usability studies - a bright idea

23. Exploratory – early stage using screen shots and paper visuals Assessment – mid-point focusing on the actual product Verification – final stage to ensure product meets standards Comparison – used in any stage to compare design options Four types of usability studies

24. Usefulness Ease of learning Ease of use User satisfaction Usability studies test

25. Card sort Prototypes Usability tasks Focus groups Individual interviews Methods used to collect data

26. Type of study Exploratory PurposeTest JoCo Library Children’s Web site Methods usedPrototypes, usability tasks, focus group ParticipantsEleven children ages 6 to 11 Examples of library usability studies Johnson County Public Libraries JoCoKids Usability Study April 25, 2009

28. Type of studyAssessment PurposeTest new web site Methods usedPrototypes, usability tasks, focus group ParticipantsFive mothers with children under age 6 Johnson County Public Libraries 6 by 6 Usability Study September 1-2, 2009

29. Step 1Let participants explore the prototype web site and record comments Step 2Usability tasks 6 by 6 usability study

31. Type of study Verification PurposeVerify usability of new paper application Methods usedUsability task Participants10 patrons Johnson County Public Libraries New Paper Application Card Usability Study June 16, 2009

32. Anschutz Library Renovation Project 2009-2010 Dec Emailed survey to faculty, staff and students Mar-SeptRedesigned the main library floor and created the Learning Studio Oct “Snapshot Day”

34. Beck, S.E. & Manuel,K. (2008). Observation and usability. In Practical research methods for librarians and information professionals. New York, N.Y.: Neal-Schuman Publishers, Inc. Brown, L. & Trochim, W.M. (2006). Research methods knowledge base. Retrieved from Cornell University website: http://researchmethods.net. Brown, L. (n.d.). Observational ield Research [Online tutorial]. Retrieved from Cornell University Library website: http://www.socialresearchmethods.net/tutorial/Brown/lauratp.htm. Hendry, J. (1999). Other people's worlds: An introduction to cultural and social anthroplogy. New York, New York: Washington Square University Press. Kumar, K. (1996). Using direct observation techniques [ CDIE Tips Series,4]. Retrieved from USAID Evaluation Publications website: http://pdf.usaid.gov/pdf_docs/PNABY208.pdf. Lee, R.M. (2000). Introduction to unobtrusive methods. In Unobtrusive methods in social research, pp. 1-16. Norlin, E. (2000). Reference evaluation: a three strep approach – surveys, unobtrusive observations, and focus groups. College and Research Libraries, 61(6),pp. 546-553. Paul, B.D. (1953). Interview technigues and field relationships. In Kroeber, A.L. (1953). Anthropology Today: an encyclopedic inventory. Chicago, Illinois: University of Chicago Press. Radin, P. (1966). The method and theory of ethnology: An essay in criticism[2 nd ed.] New york and London, England: Basic Books. Reece, R.D. & Siegal, H. A. (1986). Studying people: A primer in the ethics of social research. Macon, Georgia: Mercer University Press. Romanczyk, R.G.,Kent, R.N., Diament, C., & O'Leary, D. (1973). Measuring the reliability of observational data: a reactive process. Journal of Applied Behavior Analysis, 6(1), 175-184. Trochim, W.M. The research methods knowledge base, [2nd ed.]. Retrieved from: http://www.socialresearchmethods.net/kb/. Last revised: 10/20/2010. Usability studies at Johnson County Libraries (n.d.) Retrieved from: http://jocolibrary.org/templates/JCL_InfoPage.aspx?id=3063&epslanguages=EN. References