1. Introduction

2.  We do experiments in Human-Computer Interaction because we want to know...  Is product A better than product B?  What is good and bad about X?  Testing design principles and methods  Etc. etc.

3.  Experimentation in HCI is all about people  As they will use the products we develop  But we also – less often - do experiments without human involvement  e.g. testing software capabilities  Strictly speaking this is not HCI, but usually a people-oriented aim

4.  Raw materials for experiments:  People  On their own horribly complex and varied things to test ... And we usually run tests with groups of people!  Computer interfaces  And software, experiences, designs, art, etc. etc.

5.  People as objects of study:  People are different  Skills, knowledge, expertise  Tiredness, illness, motivation  They think and learn  => high variability in experimental results  => hard to obtain significant results

6.  People are also subject to complex effects, that are hard to control for (measure the effect of) in experiments  Time of day effects  Tiredness, post-lunch dip, etc.  Transfer effects  Learning and interference

7.  Other problem is that of context: Experiments can be done in the field or the laboratory  Each their own strengths and weaknesses  Since we usually involve groups of people, we have problems with accounting for the effect of social dynamics ... and group relationships – how do they impact on what we want to measure?

8.  Finding subjects for experiments is (also) challenging  Nearly always, we have specific criteria that we would like participants to fulfill  Females, age 30+, driving a powder-blue prius, who likes liqourice  Often we do not have the money to pay people, so hard to get the right ones  This leads to the problem of most Psychology and HCI experimental research being done with Psychology and Computer Science undergraduate students  But how representative are they of the target population we are interested in?

9. ”Statistics is the least of your problems!” Alan Dix, ”Avoiding Damned Lies”

10.  Statistics is a tool for analyzing data from experiments and deriving meaning from them  Statistics is a logical process – each type of problem has one or more statistical methods that can be employed  If you can identify the problem, you can find the statistical test to use  Finding help/guides for statistical tests is pretty easy

11.  Statistics is primarily used when we are looking for ”broad and shallow” results  Using surveys, data logging, large experiments  When using quantitative methods (i.e. Getting numbers as data)  If we want meaning – in-debt knowledge about just a few subjects, we use qualitative methods (numbers as data)  Video logs, not post-task walkthroughs, anecdotal evidence, etc.

12.  If we want to conclude... ”95% of users had problem X” - we use statistics ”Problem X happens for this reason...” - we use qualitative methods Ideally both! Backup the quantitative data with qualitative – give meaning to the numbers! When I grow up, I want to be a HMW

13.  Statistics are an incredibly powerful tool for an HCI person (interaction design, usability, whatever...)  In this course, focus on applying statistical methods to analyze experimental data  Some qualitative methods also, but mostly this is in the course Target Group Analysis

14. A powder-blue prius

15.  Practical information about the course  Course objectives  Course textbooks  Course plan  Exercise:  Table-top hockey experiment

16.  Center for Computer Games Research  Mostly teaches at DDK-line  Empirical researcher: Science by experimentation  Mostly focused on experiments with humans (annoying bastards!)  User experience analysis in interactive applications  Games, websites, etc.

17.  Lectures Wednesday 10-12 in room: 4A22  Exercises Wednesdays 13-15 in room: 4A58  Exercises starts at 13.00 – ends at 15.00 (you can stay longer if you wish!)  Handouts for exercises on the course website (generally the week before): http://experimentdesign.wordpress.com

18.  Read the course handbook carefully – it contains important information (it is available on the website)  On the website you will find handouts, exercise guides and other documents used in the course, as well as updates and messages from the course convener: http://experimentdesign.wordpress.com

19.  Basic grounding in research skills and research methodology  Designing and running experiments  Data analysis using statistics, SPSS and Excel  Writing up studies using standard presentation conventions  Designing questionnaires and fielding surveys  Ethics in research  Laws of interaction design

23. Course textbook: Field and Hole (2003). Sage publications. Sage, 2006 Will also be used: Field (2005). Sage publications

24.  You will be using it throughout the course

25. Other good statistics textbooks: Pearson / Prentice Hall 2005Pearson / Prentice Hall 2004

26.  The course will be assessed 100% via the final exam  Exam is written, with aids, on a PC, but minus internet access.  Exam will focus on testing your understanding of the principles taught in the course  It will focus on problem solving and thinking, not remembering the curriculum word by word  Note that changes may happen …  During the course there will be an assortment of assignments, some to be handed in, some to present, during the semester  These do not count towards your grade  Without doing them you will learn nothing …

27.  This is a method course, which can be intimidating  If you need help, get help – problems are easier to fix early on  Primary help: Ask you co-students and the people in your group  Secondary: Contact the course convener during office hours  Office hours: Thursday 10.30-12.00, Monday 10-30- 12. Room 4B06.  DO NOT disturb outside office hours

28. Course weekDateLectureExerciseNotes 127/8Introduction to the course Tabletop hockey experiment 22/9NO LECTURENO EXERCISE Start reading for Week 3 39/9 How to write a scientific report WORKSHOP (lectures and exercises intermingled, 10-17) Analyzing a scientific paper Writing a lab report 416/9Planning and designing experiments Introduction to SPSS Problem solving in groups Hand in assignments 523/9Descriptive statistics Descriptive data analysis in SPSS Problem solving in groups TBA 6 28/9 and 29/9 28/9 lecture, room 4A22 10-12: The normal distribution and hypothesis testing 29/9 Exercise, room 2A52 13-15: Creating graphs in Excel Problem solving in groups TBA 77/10Parametric statistics Performing ANOVA in SPSS & other fun tasks TBA

29. Course weekDateLectureExerciseNotes 8 14/10FALL BREAK - NO LECTURENO EXERCISETBA 921/10Non-parametric statistics Yet even more problem solving in groups TBA 1028/10Correlation Some really cute problems to be solved in groups TBA 1104711Linear regression Starting the free experiment (groups) + problem solving Prepare experiment I 1211/11 Survey-based methods and questionnaire design Running experiment + constructing surveys Run experiment 1318/11 Principles of interaction design: Fitt´s law and the Power Law of Practice Fitt´s law experiment (groups) Prep. presentation of experiment 14 02/12 Ethics in research Introduction to the exam Presentations of experiment results TBA

30.  Each week there will be some core reading  From Field & Hole  Or from the compendium  Some weeks there is also optional reading suggested – strongly encouraged that you read this  (I will be watching you...)

31.  Plagiarism: Passing of someone else´s work or ideas as your own.  Don´t do it – risk being expelled or taking the course again  Collusion: Working with someone else and claiming that the jointly-produced work is entirely your own  Important point: When NOT working in groups, your work must be unique to you

34.  Aims:  To show you how experiments work in practice  The de-mystify the process

35.  Testing how far an improvised hockey puck travels under different conditions  Two factors (or conditions) are involved:  Shot type  Puck placement along stick  Each factor has two levels (or values):  Shot type: Wrist shot, slap shot  Puck placement: Near end of stick, middle of stick

36.  So we have 2 factors with 2 levels: This is called a ”two level factorial design” – a very traditional experiment design in engineering sciences  The aim is to test all possible combinations of factors and levels – here 4: Value AValue B Factor 1Short end of stickLong end of stick Factor 2Slap shotWrist shot

37.  In order to make sure our results are valid, we need to run each combination multiple times  Do 10 shots with each combination. Record distance travelled for each shot  Make sure you set up each shot exactly according to the guidelines – otherwise you introduce experimental error

38.  Follow the experimental procedure in the handout  The handout is on the course website: www.experimentdesign.wordpress.com  Follow the guidelines for how to analyze the experimental data + answer the questions given  When everyone are done we will discuss the results jointly in class