1. Facilitating Access to Video Oral Histories through Informedia Technologies and a Multimedia Web Portal October 29, 2010 Presented at Oral History Association Meeting, Atlanta, GA Mike Christel christel@cs.cmu.edu Entertainment Technology Center Carnegie Mellon University

2. Talk Outline (Heavy on demonstrations, light on slides….) Introduction to oral history concerns/opportunitiesIntroduction to oral history concerns/opportunities Quick overview of CMU Informedia researchQuick overview of CMU Informedia research The HistoryMakers African American oral history archiveThe HistoryMakers African American oral history archive Field tests with The HistoryMakersField tests with The HistoryMakers Lessons LearnedLessons Learned Flash Web SiteFlash Web Site Next StepsNext Steps Reference: http://www.idvl.org For more info or to express interest: Email info@idvl.org

3. Why Oral Histories? Oral history has been embraced as a useful tool for: “…giving voice to those who have been excluded from the historical record” “…adding richness and personal perspective to the historical record and engaging students and scholars in a lively study of history” “…adding richness and personal perspective to the historical record and engaging students and scholars in a lively study of history” Contextual inquiry has found that oral history recordings are considered a central historical artifactContextual inquiry has found that oral history recordings are considered a central historical artifact Jong, F. D., Oard, D. W., et al. 2008. Access to recorded interviews: A research agenda. Journal on Computing and Cultural Heritage 1, 1 (Jun. 2008), 1-27Significant hurdles in oral history access are brought upon by the linear nature of speech and the extended length of some interviews: Jong, F. D., Oard, D. W., et al. 2008. Access to recorded interviews: A research agenda. Journal on Computing and Cultural Heritage 1, 1 (Jun. 2008), 1-27

4. CMU Informedia Digital Video Research 1994-2007 details at: http://www.informedia.cs.cmu.edu New work for oral histories at http://www.idvl.org Speech recognition and alignment Image processing Named entity tagging Synchronized metadata for search and navigation Fast, direct video access to oral histories, news, etc. Demonstrated successes (NIST TRECVID) with broadcast news, documentaries, surveillance video in nursing homes, oral histories – over 45 TB Oral history work has included work with Harrisburg, PA 150 and The HistoryMakers (over 1000 hours and 20,000 stories) – funded via NSF IIS-0705491

5. The HistoryMakers Oral History Archive http://www.thehistorymakers.comhttp://www.thehistorymakers.com Funded in part by IMLS Grant LG-03-03-0048-03Funded in part by IMLS Grant LG-03-03-0048-03 The world’s largest African American oral history archive with accomplished African AmericansThe world’s largest African American oral history archive with accomplished African Americans Purpose:Purpose: To educate and show the breadth and depth of this important American history as told by the first person To educate and show the breadth and depth of this important American history as told by the first person To highlight the accomplishments of individual African Americans across a variety of disciplines To highlight the accomplishments of individual African Americans across a variety of disciplines To preserve this material for generations to come To preserve this material for generations to come Committed to exposing the archive to the widest audience possible, making use of new technologies as appropriateCommitted to exposing the archive to the widest audience possible, making use of new technologies as appropriate

6. Screen Shot 1

7. Screen Shot 2

8. Screen Shot 3

9. Statistics from Field Tests, 2008-2009 Field-tested at Drexel University, The HistoryMakers office, 3 campuses of the University of IllinoisField-tested at Drexel University, The HistoryMakers office, 3 campuses of the University of Illinois 266 participants (158 female, 108 male) used system266 participants (158 female, 108 male) used system 210 people used it for only one logged session, 34 participants used it twice, 18 people used it three to five times, with 4 people using it six or more times210 people used it for only one logged session, 34 participants used it twice, 18 people used it three to five times, with 4 people using it six or more times 381 sessions were logged, representing 329 hours of use, based on recorded session start and end times.381 sessions were logged, representing 329 hours of use, based on recorded session start and end times.

10. Participant Demographics

11. Transaction Log Results 381 Total Sessions First Time Sessions (n=266) Return Sessions (n=115) Avg. Session Time (as mm:ss)44:1352:43 Number of Text Queries Issued4.203.39 Avg. Answer Size per Query413318 Number of Played Video Stories10.210.7 Time Spent Playing Video (mm:ss) 32:3935:58 Use of Query Preview Browser0.180.32 Use of Advanced Search Options0.610.97 Use of Movie Table of Contents0.640.12 Use of Timeline View0.210.10 Use of Common Text View0.150.18 Use of Storyboard View0.140.19 Use of Tree Browser View0.140.30 Use of Geographic Map View0.060.13

12. Query Types to The HistoryMakers

13. Lessons Learned Improved delivery infrastructure (web client vs. stand- alone lab-only access)Improved delivery infrastructure (web client vs. stand- alone lab-only access) Clean, simple accessClean, simple access Intuitive “standard” use of text searchIntuitive “standard” use of text search Intuitive filteringIntuitive filtering A web portal (in Flash) was developed for The HistoryMakers, and then tested by 27 participants (13 female, average age 25) in which users were asked to do two tasks, one a direct lookup “treasure hunt” and the other open-ended and exploratory.

14. Web Portal Empirical Study Results (27 Participants)Treasure Hunt TaskExploratory Task Session Time (as mm:ss) 20:00 Number of Text Queries Issued 45.214.1 Avg. Answer Size per Query 52139 Number of Played Video Stories 18.48.7 Time Spent Playing Video (mm:ss) 7:0813:41

15. Empirical Study, Post-Task Survey Data 5-Point Scale (left = 1, right = 5) Treasure Hunt Task Exploratory Task Unsatisfying – Satisfying4.0 Not Inspiring – Inspiring3.64.0 Not Entertaining – Entertaining3.74.0 Inaccurate – Accurate4.24.0 Text Search: Not Useful – Useful3.8 Map Search: Not Useful – Useful2.42.8 Search filtering: Not Useful – Useful 3.33.7 Video player: Not Useful – Useful 3.44.5 Text transcript: Not Useful – Useful 4.34.2

16. Value of Video Varies by Task The value of representation modality can be masked by the task: If only direct lookup (treasure hunt) is studied, the video modality may be seen as unnecessary over searchable transcript text. By considering exploratory tasks, the value of video for life oral histories is acknowledged. Interestingly, even for open-ended exploratory tasks, an aligned text transcript modality remains highly valued for oral histories. More details at: Christel, M., Stevens, S., Maher, B., and Richardson, J. Enhanced Exploration of Oral History Archives through Processed Video and Synchronized Text Transcripts. Proc. ACM Multimedia Conference (Florence, Italy, October 2010).

17. Web Site Demo Adobe Flex used to build Flash interface to oral histories providing:Adobe Flex used to build Flash interface to oral histories providing: Text search (using Lemur open source indexing) Text search (using Lemur open source indexing) Map search (using Yahoo! Maps) Map search (using Yahoo! Maps) Tag search Tag search Video play with synchronized transcripts Video play with synchronized transcripts Filtering (based on provenance data such as narrator gender and birth year) Filtering (based on provenance data such as narrator gender and birth year) Try it yourself at www.idvl.orgTry it yourself at www.idvl.org

18. Future Work Faceted search (e.g., showing at a glance the distribution of stories across categories, allowing for quick interactive filtering and review) Simple to use menu bar (as is popular on e-commerce sites) Emphasis on aesthetics (improved interface coming soon!) Further iterations based on web metrics Streamlining on the processing side (currently a text xml file and a video file are the input)