1. Quantitative Methods for Researchers Paul Cairns paul.cairns@york.ac.uk

2. Your objectives  Pretty general! – Landscape/area of experiments 2

3. My objectives  Importance of experimental method  Experiments as evidence  Validity  Scrutability  Statistics as model comparison 3

4. Preliminary question? 4

5. Experiments as evidence  Randomness removes certainty  Experiments frame data  Without frame, no point 5

6. Experimental argument  Theory:X causes Y  Test: change X and measure Y  But: – variation (people, stochastic) – other things affect Y – hard to measure Y  Statistics pierce through the murk! 6

7. Theories in computing  Thin on the ground – Name one?  Low relevance to applications  So experiments are pointless? 7

8. Experimental Computing  Experiments have own value  Experiments inform theory  Narrative context  We create the objects of study QUAN, Paul Cairns

9. Experimental argument  Belief:X causes Y – A reason for looking  Try: change X and measure Y  Analyse carefully  Produce evidence 9

10. Variables  Independent variable (IV, X)  Dependent variable (DV, Y) – quantitative  Confounding variables 10

11. Devising an experiment  Research question (disposable)  One sentence  May use jargon  Answer is “yes/no” but probably “maybe”  Question suggests how to answer it QUAN, Paul Cairns

12. Revise your research question In groups of three or four, each have a go at a research question. Take turns to explain and be criticised. Be happy to be wrong/stupid. RQs are disposable. QUAN, Paul Cairns

13. Evidence  If – X has really changed – Y has been properly measured – Nothing else has changed – The result was significant  Then – Evidence that X causes Y 13

14. Value?  Modest but cumulative  Opportunity for falsification  Evidence  Isolation of phenomena 14

15. Not black and white  Experiments are not proof – Validity – Assumptions  Experiments have a frame – Eg speed of gravity 15

16. Write up  Title and abstract  Aims = lit review  Method  Results  Discussion 16

17. Why do we do a literature review? 17

18. Literature  Previous research  Defines the community  What and who  Implicit standard  Implicit style QUAN, Paul Cairns

19. Using literature  Importance  Interest  Originality  Insight

20. What’s the purpose of a write up? 20

21. Method section  Aim and hypothesis  Participants  Variables  Design  Materials  Procedure 21

22. Writing as a tool  Necessary headings: – write them! – Before you do the expt  What will sig show?  Is it valid?  Forces a dialogue – With self or supervisor 22

23. Fantasy abstract  Write an abstract for your experiment (150-250 words) specifying: 1.What the question is 2.Why it is interesting/important 3.What was done in the experiment What IV and DV are 4.What significant results (would) show 5.What this means 23

24. Swap abstracts – “homework”  Do you know what the question is?  Why is it interesting/important?  What is the experimental argument?  Do you believe it?  What would make it better? 24

25. Reading  Abelson, Statistics as Principled Argument  Hacking, Representing and Intervening  Cairns, Cox, Research Methods for HCI: chaps 1, 6, 10  Harris, Designing and reporting experiments in psychology, 3rd edn 25