1. ECDL 2006 An Exploration of Space-Time Constraints on Contextual Information in Image-based Testing Interfaces Unmil Karadkar, Marlo Nordt Richard Furuta Cody Lee Christopher Quick Texas A&M University Center for the Study of Digital Libraries The Department of Computer Science Michael E. DeBakey Institute Veterinary Physiology & Pharmacology

2. ECDL 2006 Outline Cardiovascular Research Research Skill Testing Interfaces Design of the Evaluation Results and Analysis Future Work

3. ECDL 2006 Outline Cardiovascular Research Research Skill Testing Interfaces Design of the Evaluation Results and Analysis Future Work

4. ECDL 2006 Digital Image Collections Collections –Museums The Louvre –Historical image archives New York Times Supported tasks –Searching and browsing Typical interfaces –Table/grid

5. ECDL 2006 Specialized Tasks Analytical tasks –Examine image content –Interpret visible detail Assessment tasks –Estimate dimensions of objects within images Associative tasks –Compare and contrast multiple images

6. ECDL 2006 Bat Cardiovascular Research Observe bat wings under a microscope –Pallid Bats – transparent wings –Non-destructive, in-vivo studies Learning curve of one semester How can we train researchers faster?

7. ECDL 2006 Images in Cardiovascular Research Artery / Vein recognition –Analytical task Estimate order of blood vessels –Assessment task Identification of Lymphatic vessels –Associative task

8. ECDL 2006 Working with videos Single layout of images Montage –Facilitates comparison of consecutive images Motion cues But experienced researchers often don’t need these cues

9. ECDL 2006 Still Images for Testing Skills Validated by a pilot subject Properties of image layouts Spatial organization Temporal presentation Constrain or facilitate contextual information Comparison of images –Images of blood vessels at different locations –Images of a specific location over time

10. ECDL 2006 Outline Cardiovascular Research Research Skill Testing Interfaces Design of the Evaluation Results and Analysis Future Work

11. ECDL 2006 Montage Layout Spatial Dimensions (0) Context (user dependent) Temporal Dimension (used) Slowed down video

12. ECDL 2006 Scrolling Layout Spatial Dimensions (1) Context (4 images) Temporal Dimension (used) Smooth scrolling

13. ECDL 2006 Thumbnail Layout Spatial Dimensions (2) Context (all images in set) Temporal Dimension (unused)

14. ECDL 2006 Properties of Image Layouts MontageThumbnailScrolling Spatial Dim. Temporal Dim. Context 0 Used User-dependent 1 Used 4 images 2 Unused 16 images

15. ECDL 2006 Outline Cardiovascular Research Research Skill Testing Interfaces Design of the Evaluation Results and Analysis Future Work

16. ECDL 2006 Experimental Design 3 image layouts –Thumbnail, Scrolling, Montage 16 images per set 3 tasks –Artery/Vein recognition –Size Estimation –Lymphatic vessel wall identification Balanced across subjects –Ordering of tasks (AEL, ELA, LAE) –Ordering of image layouts (TMS, MST, STM)

17. ECDL 2006 Subjects Trained researchers in the bat lab –Available subject pool of 30 9 experts –Research experience of more than 2 semesters 6 novices –Research experience of 1 semester only Age group 18 to 35 4 female, 11 male

18. ECDL 2006 Data Collection Responses to image-based questions –Verbal and drawn –Validated by experts –Instrumental measurements Demographic questionnaire –Self evaluation of expertise Task reflection questionnaire –For each task Preferred layout Perceived best performance

19. ECDL 2006 Outline Cardiovascular Research Research Skill Testing Interfaces Design of the Evaluation Results and Analysis Future Work

20. ECDL 2006 Did not improve between sets Performance constrained by knowledge Improved between sets (p=0.0004) Performance constrained by layout Artery/Vein Recognition 89.6% 91.4% Performance differed significantly between groups Both groups preferred the Thumbnail layout (p=0.005)

21. ECDL 2006 N=16, x=9.75 σ 2 =9.73 No learning Constrained by knowledge Preferred layout T N=16, x=13.59 σ 2 =2.13 Learning (p=0.0004) Constrained by layout Preferred layout T Artery/Vein Recognition (p=0.005) 89.6% 91.4% Validated conventional wisdom

22. ECDL 2006 Size Estimation General understanding of the order of blood vessels does not translate into accurate estimation of size (p<0.002) Judged at various levels of accuracy (10% - 40%) Performance differed between groups at all levels Both groups preferred the Thumbnail layout

23. ECDL 2006 Size Estimation N=16, x=9.03 σ 2 =14.11 Preferred layout T N=16, x=4.06 σ 2 =12.51 Preferred layout T General understanding of the order of blood vessels does not translate into accurate estimation of size (p<0.002)

24. ECDL 2006 Lymphatic Vessel Wall Identification Preferred the Montage layout Also worked best with the Montage layout (p=0.035) (p=0.03) 87% 55% Experts and Novices use different strategies for applying contextual information Preferred the Montage layout But worked best with the Thumbnail layout

25. ECDL 2006 Outline Cardiovascular Research Research Skill Testing Interfaces Design of the Evaluation Results and Analysis Future Work

26. ECDL 2006 Continuing Work User studies with Summer research students –21 novices (9 male, 12 female) Comparison of training strategies –10-week intensive vs. 14-week sporadic training Results –Consistent with those of the 6 novices presented here (not statistically different) –Differences in training style did not affect performance

27. ECDL 2006 Lessons Learned Digital Image Collections enable discipline- specific scholarly tasks –Typical collections provide only a thumbnail layout view Performance on these tasks is affected by –Image layout properties –Task characteristics –User experience and expertise Collections should provide different layouts

28. ECDL 2006 Future Work Investigate mechanisms for conveying dimensions of objects within images Explore other image layouts –Static and dynamic collages Train subjects on new interfaces –Improve learning time –Increase productivity

29. ECDL 2006 http://ebat.tamu.edu Center for the Study of Digital Libraries Michael E. DeBakey Institute Further Information Unmil Karadkar Marlo Nordt Richard Furuta unmil@csdl.tamu.edu mnordt@csdl.tamu.edu furuta@csdl.tamu.edu Christopher Quickcquick@tamu.edu