1. Ben Shneiderman & Catherine Plaisant
Strategies for evaluating information visualization tools: Multi-dimensional In-depth Long-term Case studies (MILCs) (BELIV 2006 at AVI Conference)
Ben Shneiderman & Catherine Plaisant
University of Maryland
Human-Computer Interaction Lab
Test of Time Award from BELIV Workshop 2016

3. Multi-Dimensional In-Depth Long-term Case Studies (MILCs)
Domain experts (+ our research team)
Exploring their own data
To produce their research results
Using new interactive tools
Time-timited structured evaluation (4-8 weeks)
Observations, interviews, surveys, etc.
Replicated case studies = Hypothesis testing
(Shneiderman & Plaisant, BELIV 2006)

4. TimeSearcher: PhD 2003 2 case studies Genome biology: DNA expression
Nucleotide sequences

5. Hierarchical Clustering Explorer: TVCG 2006
3 case studies
Biologist, statistician, meteorologist
Survey results from 57 users
94 citations

6. SocialAction: CHI 2008 4 case studies Pushback from reviewers
Business consultant, medical librarian
Journalist, terrorism analyst
Pushback from reviewers
But we succeed
162 citations

7. Treeversity PHD: 2013
13 case studies

8. Treeversity PHD: 2013

9. Multi-Dimensional In-Depth Long-term Case Studies (MILCs)
Other PhD students who conducted MILCs
David Wang
Krist Wongsuphasawat
Cody Dunne
Megan Monroe
Sana Malik
Fan Du

10. The New ABCs of Research (Oxford, 2016)
Guide for Junior researchers
Manifesto for
Senior researchers
Academic administrators
Business leaders
Funding agencies
   

12. Research Methods Method Definition Pros Cons
Controlled
Experiments
Defined task, hypotheses, alter IVs (treatments), measure DVs, run stats, prove hypoths
Scientific, reproducible
Biases can undermine results
Usability
Testing
Evaluate users efficiency & subjective reaction in use of tool for specific tasks, list of changes
Fast, insight to user thought processes
Small user sample, short-term, limited scope
Case Studies
(Design Study Methodology)
Longer term, with domain experts, design tool, write up reflections/experiences
insight to user thought processes, guidance to refine tool
Takes a long time, questionable generalizability