Hannah Marshall January 2015 Preliminary Findings: A Comparative Study Of User- And Cataloger- Assigned Subject Terms

1.What is the level of correspondence between users and catalogers in the subject analysis of images?  Based on evidence that description is a strong indicator of search behavior, this will lend insight into our users’ search behaviors and the accessibility of subject metadata in our image collections 2.What is the level of correspondence in the type of subject analysis being performed by each group?  When users’ and catalogers’ terms differ, do the terms that they assign reflect similar or differing types of interpretation and analysis?  The subject analysis of images has traditionally been viewed as a graduated scale that identifies different levels of interpretation and analysis. We are interested to see the degree to which each group uses them. 3.Does coaching users on performing subject analysis change their search behaviors?  Findings can be leveraged into outreach efforts and in-house cataloging best practice RESEARCH QUESTION(S)

 December Initial proposal  Literature review  Methodology  Schedule  Value of research  June 16- June 26 th Institute for Research Design in Librarianship (IRDL)  July – October Study Design  Selected images and exported metadata  Designed surveys (Qualtrics)  Drafted recruitment and informed consent form  Identified the study population, recruitment method, data analysis techniques  Pre-testing survey  Drafting and submitting Request for Exemption from IRB Review  October 5, IRB Exemption granted  October 24 th Data collection begins  December 8 th 2014 – Data collection ends  (Spring 2015 – Data collection begins again) RESEARCH PROCESS TIMELINE

 Data collection instrument: Qualtrics survey  10 images  9 subject terms per image  Control Group  “Please review the image and then enter terms that you think describe the image in the spaces provided.”  Variable Group  “What is the image of?”  “What is the image about?”  “What is the image a good example of?’ RESEARCH PROCESS METHODOLOGY

 Survey active October 24th – December 8th  80 responses – 20% response rate  Distributed to all undergraduates enrolled in art history or classics courses  Approximately 400 students  Randomly divided into a control and variable group  39 responses in control group  Completion rate = 31.7%  41 responses in variable group  Completion rate = 35.9% RESPONSES

 Comparison of literal values  Research Question #1: What is the level of correspondence between users and catalogers in the subject analysis of artworks?  Participants’ responses vs. existing subject terms  Control group vs. variable group  Research Question #3: Does coaching users on performing subject analysis change their search behaviors?  Comparison of coded values  Research Question # 2: What is the level of correspondence in the type of subject analysis being performed by each group?  Participants’ responses vs. existing subject terms  Control group vs. variable group  Research Question #3: Does coaching users on performing subject analysis change their search behaviors? ANALYSIS

 Comparing existing metadata to participants’ responses  31% correspondence (for control and variable groups)  for each of the existing terms assigned to an image, at least one participant assigned the exact same term  Equal rate of correspondence between control and variable groups  Comparison of control and variable groups (no great differences)  Participants in the variable group supplied more terms per image  Average # of terms provided by variable group = 104  Average # of terms provided by control group = 88 PRELIMINARY FINDINGS LITERAL VALUES

StrataExisting Metadata Participants (all) Participants (control) Participants (variable) Primary72%20%22%18% Secondary9%11%10%12% Tertiary13%3%4%3% PRELIMINARY FINDINGS CODED VALUES Strata: Primary – perception of the work’s pure form “What is the work/image of?” Secondary – incorporates cultural and iconographic knowledge “What is the work/image about?” Tertiary – looks at art as the product of a historical environment “What is the work/image a good example of?” *Note – roughly 66% of responses included terms that related to non-subject metadata like culture, period, materials and techniques, and work type. These were not coded

 Finish coding responses and complete the comparisons  Examine differences in the literal and coded responses for each image to detect differences between different types of images  Figurative vs. abstract  Western vs. non-western  Cultural objects and artifacts vs. expressive works  Analyze the non-subject related terms that were provided  Next Steps:  Continued analysis  Spring 2015 data collection FURTHER ANALYSIS TO BE DONE

THANK YOU!