1. Methodology and Explanation XX50125 Lecture 2: Experiments Dr. Danaë Stanton Fraser

2. Methodology and Explanation 2005 Lectures 1.Introduction to Methods 2.Experiments 3.Interviews/questionnaires/surveys 4.Usability testing 5.Ethnography

3. Methodology and Explanation 2005 Aim To answer a question or test an hypothesis that predicts a relationship between two or more events, known as variables. E.g. Will people be quicker and more accurate at completing a task collaborating at a distance using just verbal instructions or with verbal instructions with gesture input? Will spatial knowledge be superior following exploration of a VR simulation of a building or from exploring a model of a building?

4. Methodology and Explanation 2005 Variables Such hypotheses are tested by manipulating one or more of the variables. The variable that is manipulated is called the independent variable (the conditions to test this variable are setup independently before the experiment starts). In the study mentioned media type (VR vs model) is the independent measure The dependent variable would be accuracy of spatial information (ie time to reach point A from point B) because the time to carry out the task depends on the media explored. What are the IV and DV for the gesture study?

5. Methodology and Explanation 2005 Variables and Conditions In order to test an hypothesis the experimenter sets up the experimental conditions e.g Condition 1 Pretest Explore model Post test Condition 2 Pretest Explore VR simulation Post test Control condition – against which to compare the results What are the conditions in the gesture study?

6. Methodology and Explanation 2005 Participants Within participants design Counterbalancing required Between participants design Two drawbacks – no. of participants needed, individual differences Advantage – no order effects Matched pairs design (on characteristics such as gender and expertise) Difficult to match across all variables

7. Methodology and Explanation 2005 Practicalities Where will the experiment be carried out? How will the equipment be set-up? How does one introduce participants to study? What scripts are need to standardise the procedure? Always include pilot study

8. Methodology and Explanation 2005 Data Collection and Analysis Data – performance measures are taken e.g response times, no. of errors, angle estimation… Use graphs The data should be averaged across conditions to examine any differences Statistical tests such as t-tests and ANOVAs can reveal if the differences are significant Software packages such as SPSS are often used If there is no significant difference then the hypothesis is refuted.

9. Methodology and Explanation 2005 Data captured Record quantative results However often also able to gain qualitative data e.g dialogue turns, gesture and non verbal behaviour Good practice to collect both qualitative and quantative data

10. Methodology and Explanation 2005 Summary of Experimental Design State a causal hypothesis Manipulate independent variable Assign participants randomly to groups Use systematic procedures to test hypothesised causal relationships Use specific controls to ensure validity

11. Methodology and Explanation 2005 Quasi-Experimental Design State a causal hypothesis Separate participants based on some characteristic e.g level of mobility as child, gender, occupation Cannot assign participants randomly to groups

12. Methodology and Explanation 2005 A Variety of Experimental Designs (Intervention) - Post test Pre test - (Intervention) - Post test Pretest - (Intervention) - Post test - Delayed Post Test Interrupted time-series Cross-over

13. Methodology and Explanation 2005 Preparation for Practical 2/3 Topic: Examining the use of gestures in co-located collaboration. We will design and run a small experimental study to examine gesture in co-located collaboration In preparation for the practical please: Read up on experimental design e.g Robson, C. (1994). Experimental Design and Statistics in Psychology. Aylesbury, UK. Penguin Psychology. Coursework 1: Write up the experiment carried out in Practicals 2 and 3. Deadline for report Friday 18 th November.