1 Instructor: Vincent Duffy, Ph.D. Associate Professor of IE Lab 1 – Part 1 Review - Lifelong Learning Fri. Feb. 2, 2007 IE 486 Work Analysis & Design II

2 10 ways to look at a research paper systematically for critical analysis v 1. research ideas/question -what is the purpose? v 2. what background support? - literature review v 3. theoretical basis for analyzing question/hypothesis? v 4. applicability-practical contribution? v 5. theoretical contribution? v 6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability) v 7. appropriate statistical analyses and assumptions? v 8. presentation of results: what do they really mean? v 9. conclusions drawn: are they reasonable? v 10. future work/research directions: any possibilities?

3 10 ways to look at a research paper systematically for critical analysis v 1. research ideas/question -what is the purpose? v Recognize that more than just translation issues when understanding differences in culture (social- psychology) v Many U.S. researchers didn’t consider whether some of the past research was applicable (generalizable) beyond the subjects that were tested. v Limited research available to discuss the ‘cross-cultural’ aspects of human-human interaction (social- psychology), HCI (w/in a culture; then across cultures), Computers as Social Actors Paradigm

4 10 ways to look at a research paper systematically for critical analysis v 2. what background support? - literature review v Relationship shown among U.S. participants in past study v Validated over 100 social norms from social psychology…in HCI v Nass, 1996, etc.. ; Moon, Reeves, Fogg, …..

5 10 ways to look at a research paper systematically for critical analysis v 3. theoretical basis for analyzing question/hypothesis? v Computers as Social Actors Paradigm - as an extension of the social-psychology literature (human – human) v Go to social-psychology lit finding how human treats human v Re-write to sub computer v Design and run experiment to demonstrate the social rule that has incorporated the human-computer aspects v Reciprocity – differences in culture (individualist vs. collectivist)

6 10 ways to look at a research paper systematically for critical analysis v 4. applicability-practical contribution? v Internationalization of interfaces, internet, e-business, larger computer access across world. v Can be useful to know about this ….since a large percentage of the population are interacting with computers. v Help designers of hardware/software to tailor to different cultures (business, marketing etc.).

7 10 ways to look at a research paper systematically for critical analysis v 5. theoretical contribution? v casts some doubt or challenge to prior studies that were assumed to be universal. v Contributes to ‘systematic’ approach that can be used in further study. v Can be used to support current theories related to some cultural differences – and gives specific examples for how it affects social/behavioral interaction. v Computers as social actors paradigm helping to verify some prior human-human interaction findings.

8 10 ways to look at a research paper systematically for critical analysis v 6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability)’ v Relate ‘brand’ to ‘group’ in order to carry out the experiment. v ?, ?,? v Appears to be justified based on the expected cultural difference (reciprocity). v

9 10 ways to look at a research paper systematically for critical analysis v 7. appropriate statistical analyses and assumptions? v Single blind – not knowing purpose of study… v Experiments identical – ‘between subjects’ test.. v Subjects/participants not tested in all conditions v F test; v Post-hoc test…rely on this to draw conclusions later.

10 10 ways to look at a research paper systematically for critical analysis v 8. presentation of results: what do they really mean?

11 10 ways to look at a research paper systematically for critical analysis v 9. conclusions drawn: are they reasonable? v Cultural differences exist in social-behavior between human-computer in U.S. and Japan… v Suggests more work could be done… v People search for relationship (commonality) between self and technology. (?) … v Behavior and attitude….supported v Be careful before generalizing…field could be examined further…

12 10 ways to look at a research paper systematically for critical analysis v 10. future work/research directions: any possibilities? v Idiosynchrasies/differences between cultures v “It is possible that…”