1. Some Notes on the Design and Analysis of Experiments

2. Formal experiments are … n Cons –extremely expensive (time & money) –usually not representative of the real world (cf. natural observation, field studies, surveys) n Pros –highly controlled –replicable –sometimes the only way to measure small effects or to identify interactions

3. Designed experiments are used to … n address a research question n to test a hypothesis or a model

4. Some Definitions: n Independent Variable- the variable which the experimenter has direct control over and is purposely manipulated to test a hypothesis (presence vs. absence, amount, type) n Dependent Variable- what’s being measured

5. Definitions part 2 n Factor, Treatment- a controlled variable in an experiment (fixed & random) n Level- a particular setting of a factor n Main effect- the effect of a independent variable on experiment n randomize- errr, random?

6. Definitions part 3 n within subjects experiment - all subjects receive the same treatments n between subjects experiment - subject are randomly divided into groups, and different groups receive different treatments n asymmetrical transfer - when the effect of doing A then B is different then doing B then A

7. Definitions part 4 n confounding - where the effect of variable has not been separated from the effect of another a variable n control group - a group that does not receive a treatment n factorial design - a designed experiment where two or more independent variables are studied simultaneously

8. Fractional Factorial Designs n Number of trials gets very large as one increases the number of factors & levels n higher order interactions are actually quite rare n therefore, it makes sense to confound the higher order interactions n example: 2 5-1 fractional factorial design

9. SOURCE: grand mean AA LA N MEAN SD SE 560 123.9450 18.4217 0.7785 SOURCE: AA AA LA N MEAN SD SE 30 140 140.5812 12.8250 1.0839 45 140 132.4786 11.8065 0.9978 60 140 123.1333 9.7705 0.8258 90 140 99.5869 3.8830 0.3282 SOURCE: LA AA LA N MEAN SD SE 0 140 124.6045 18.6456 1.5758 30 140 124.8262 18.6570 1.5768 60 140 123.9802 18.5886 1.5710 90 140 122.3690 17.8819 1.5113

10. SOURCE: AA LA AA LA N MEAN SD SE 30 0 35 141.1324 12.9617 2.1909 30 30 35 142.0048 13.8070 2.3338 30 60 35 140.4829 12.8396 2.1703 30 90 35 138.7048 11.9545 2.0207 45 0 35 132.7524 12.3086 2.0805 45 30 35 133.5381 11.3830 1.9241 45 60 35 133.1476 12.7154 2.1493 45 90 35 130.4762 11.0136 1.8616 60 0 35 124.5429 10.5796 1.7883 60 30 35 123.5714 8.9024 1.5048 60 60 35 122.5571 9.7806 1.6532 60 90 35 121.8619 9.9590 1.6834 90 0 35 99.9905 4.0815 0.6899 90 30 35 100.1905 3.9102 0.6609 90 60 35 99.7333 4.0352 0.6821 90 90 35 98.4333 3.3876 0.5726

11. FACTOR : Subject AA LA Res LEVELS : 35 4 4 560 TYPE : RANDOM WITHIN WITHIN DATA SOURCE SS df MS F p ================================================ mean 8602923.2895 1 8602923.2895 7253.990 0.000 *** S/ 40322.5543 34 1185.9575 AA 132098.4395 3 44032.8132 442.606 0.000 *** AS/ 10147.5084 102 99.4854 LA 517.4867 3 172.4956 6.137 0.001 *** LS/ 2866.7308 102 28.1052 AL 95.7929 9 10.6437 0.891 0.533 ALS/ 3653.5404 306 11.9397

15. Interaction An interaction exist when the effect of one variable depends on the level of another variable n Example: 2x2 factorial design has 7 possibilities for significant effects

18. A nice way to specify a design: “The experiment was a within subjects 5 X 3 X 3 factorial, repeated measures design 10 subjects X 5 limb conditions X 5 limb conditions X 3 target amplitudes X 3 target amplitudes X 3 target widths X 3 target widths X 5 blocks X 5 blocks X 20 trials per amplitude-width condition X = 45,000 total trials”

19. Some basic rules … n You should always think you know what you’re going to find BEFORE you run the experiment (which doesn’t mean that you are always right, only that you have a hypothesis) n Everything that is tested statistically should also be graphed n If your graphs and your stat analysis don’t CLEARLY agree, something is wrong

20. Some basic rules part.2 n You should always know exactly how you are going to analyze your data BEFORE you collect it. (the statistical methods) n Remember the difference between statistical significance and the magnitude of the effect