1. 1 Instructor: Vincent Duffy, Ph.D. Associate Professor of IE Lab 1 – Part 1 Review - Lifelong Learning Fri. Feb. 2, 2007 IE 486 Work Analysis & Design II

2. 2 10 ways to look at a research paper systematically for critical analysis v 1. research ideas/question -what is the purpose? v 2. what background support? - literature review v 3. theoretical basis for analyzing question/hypothesis? v 4. applicability-practical contribution? v 5. theoretical contribution? v 6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability) v 7. appropriate statistical analyses and assumptions? v 8. presentation of results: what do they really mean? v 9. conclusions drawn: are they reasonable? v 10. future work/research directions: any possibilities?

3. 3 10 ways to look at a research paper systematically for critical analysis v 1. research ideas/question -what is the purpose? v Conduct systematic research…cultural differences not so readily available. v Social norms…and transferrability across cultures (based on authors view: available?) v (gestures, pictures, translation of language – simpler communication) v For human-human & human-computer interaction v reciprocity

4. 4 10 ways to look at a research paper systematically for critical analysis v 2. what background support? - literature review v Nass’ previous study re: how people treat technologies… (1996); other colleagues Fogg (1997) … v Reeves… v Social-psychology literature v How humans treat other humans v Go to this literature v Re-write hypothesis substituting computer v Redesign experiment to test v Done for over 100 social rules – reviews available in Nass and Moon 2000. v Reciprocity (section 2 citations back to 1960)

5. 5 10 ways to look at a research paper systematically for critical analysis v 3. theoretical basis for analyzing question/hypothesis? v Reciprocity idea; differences (individualist vs. collectivist). v Computers as Social Actors paradigm

6. 6 10 ways to look at a research paper systematically for critical analysis v 4. applicability-practical contribution? v Benefits; interfaces/international issues – software/marketing; awareness of what/how to focus development resources for products that may be used in different countries.

7. 7 10 ways to look at a research paper systematically for critical analysis v 5. theoretical contribution? v Challenges those who may try to extrapolate results from one country to another, considering that this study demonstrates some of the cultural differences (as evidenced in behavior). v Methodology by which you can identify other social rules that can be considered in future studies v Showed how they did this study similar to others when establishing the CASA paradigm using U.S. participants

8. 8 10 ways to look at a research paper systematically for critical analysis v 6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability) v Validity of measures? ; reliability of measures? v Wonder how sample/participants were chosen? v ‘between subjects’ test…participants didn’t participate in all conditions. (different participants were identified to participate in certain conditions – for instance… v Appears to be lab experiment… v Limits to space.. v Obvious difference (p values) F test & compared to a critical value….alpha value (likely.05) v Type I error – likelihood that I didn’t find the result incorrectly

9. 9 10 ways to look at a research paper systematically for critical analysis v 7. appropriate statistical analyses and assumptions? v Signifcant difference…. Which test? v F test…does this show (or allow them to draw conclusions) v Tukey/Posthoc test….

10. 10 10 ways to look at a research paper systematically for critical analysis v 8. presentation of results: what do they really mean?

11. 11 10 ways to look at a research paper systematically for critical analysis v 9. conclusions drawn: are they reasonable?

12. 12 10 ways to look at a research paper systematically for critical analysis v 10. future work/research directions: any possibilities? v See p.1561 “It is possible that…” gives you some idea about authors view of what was/wasn’t done in their study. v The reference to ‘It is possible that…” could give idea about future work.