Basics of Experimentation (1) Experimental Design: Which to Choose and Why?

Why Conduct on Experiment? Goal of experimentation: –To determine cause-and-effect relations in nature –“Factor A causes factor B” What constitutes a good experiment? –Validity of the results: how true the results are Internal Validity –Extent to which one can make causal statements about the relationship between variables –Experiments aim to maximize their internal validity

Variables Variable –An event or behavior that can assume two or more values –Ex: temperature; speed of stimuli presentation; medication Independent Variable (IV) –Selected and manipulated by the experimenter –Levels: must have at least 2 levels for comparison Ex: Cell phone use: no cell phone, hand held phone, hands free phone Dependent Variable (DV) –Response or behavior that is measured by the experimenter –Ex: reaction time to pressing the brake pedal; # of words recalled; symptoms of anxiety

Experimental Designs How will participants be assigned to the levels of the IV? Between-Subjects Design –Different sets of participants are assigned to only 1 level of the IV Within-Subjects Design –Each participant is assigned to all levels of the IV Level 1 Bill John Sally Level 2 Jack Sam Lisa Level 3 Eric Peter Sandy Level 1 Larry Bob Jenny Level 2 Larry Bob Jenny Level 3 Larry Bob Jenny

Example The effects of cell phone use while driving on the time for the driver to realize the car in front of him/her is stopping. –Independent variable cell phone use 3 levels: no cell phone, hand held phone, hands free phone –Dependent variable Time it takes participants to press the brake when the car in front of them brakes.

Between-Subjects Design Considered “safer” – No chances of contamination from one treatment condition to the other. –The same participant never gets more than one treatment. Experimenter must check for as few differences as possible to exist between participants across all treatment conditions before testing. –Reduces chances of confounded experiment –Maximizes cause-and-effect relations

Between-Subjects Design No Cell Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Hands Free Phone Condition Participant 13 Participant 14 Participant 15 Participant 16 Participant 17 Participant 18 Hand Held Phone Condition Participant 7 Participant 8 Participant 9 Participant 10 Participant 11 Participant 12 Total of 18 participants needed in this experiment

Two ways to increase internal validity 1- Matching –Select a variable(s) other than the IV that could mostly likely affect performance on DV. –Ex: vision, hours of video game playing, reaction time to pressing a pedal, age –Create triads of participants who are equal on all these variable. –Randomly assign each member of the triads to a level of IV. Problems: –Cannot match participants for everything –Difficult to know which is the most important variable to match

Two ways to increase internal validity 2- Randomization –Each participants has an equal chance of being in any of the conditions of the experiment. –Does not guarantee groups will always be equal. –Preferred method when basis of matching is unsure.

Between-Subjects Design Advantages Simpler to conduct. No chances of contamination across treatment conditions. Disadvantages Requires more participants. Results may be confounded if groups are not equated by randomization. Difficult to determine which factors are important to match participants.

Within-Subjects Design All subjects receive all levels of the IV The performance of each subject is compared across the different experimental conditions. Reduces the effects of individual differences across conditions. Carryover Effects –Performance on one condition affects performance on a follow-up condition(s). –Reduces internal validity of experiment.

Within-Subjects Design No Cell Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Hands Free Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Hand Held Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Total of 6 participants are needed in this experiment

To reduce carryover effects Counterbalancing –Technique used to counter the effects of presenting conditions in a particular order/sequence. Requirements: –Each condition must the presented to each participant an equal number of times. –Each condition must occur an equal number of times in each treatment order. –Each condition must precede and follow each of the other treatments and equal number of times.

How to know the number of sequences How do you know the number of sequences that you need to run to meet these requirements? Formula: n! (n factorial) –n = number of conditions; n = 3 –3! = 3 x 2 x 1 = 6 sequences –5 conditions? 6 conditions? As the number of conditions increase, the number of sequences becomes too large. Alternative procedure: Balanced Latin Square

Balanced Latin Square –Each condition precedes and follows every other condition an equal number of times. –Preferred method of counterbalancing –For even number of conditions first row formula = 1, 2, n, 3, n -1, 4, n - 2… Order of conditions 1st2nd3rd4th Participant 1ABDC Participant 2BCAD Participant 3CDBA Participant 4DACB Need to run participants in multiples of n conditions, or 4

Order of conditions 1st2nd3rd Participant 1ABC Participant 2BCA Participant 3CAB Odd number of conditions - uses 2 Latin Squares The 2 nd Latin square is the mirror image of the 1 st. Need to run subjects in multiples of 2n, or 2 x3 = 6. Order of conditions 1st2nd3rd Participant 4CBA Participant 5ACB Participant 6BAC

Order of conditions 1st2nd3rd Participant 1No cell phoneHand heldHands free Participant 2Hand heldHands freeNo cell phone Participant 3Hands freeNo cell phoneHand held Order of conditions 1st2nd3rd Participant 4Hands freeHand heldNo cell phone Participant 5No cell phoneHands freeHand held Participant 6Hand heldNo cell phoneHands free Odd number of conditions - uses 2 Latin Squares

Within-Subjects Design Advantages Requires fewer participants. Reduces individual differences. Disadvantages Potential for carryover effects if conditions are not counterbalanced.

Which Design Works Best? It depends! If you have a large sample size and predict IV will have a large effect: –Use a between-subjects design. If you have a limited sample size and predict IV will have a small chance of carryover effects: –Use a within-subjects design.

Cross-sectional designs Study individuals of different ages at the same time. –Ex: effect of age on cognitive abilities (memory) –Test 7- yr olds, 25 yr olds and 60 yr olds Advantage –A wide variety of ages can be studied at the same time –Can collect data quickly, maybe all at once. Disadvantage –Individuals are born at different times and raised with different generations; different educational systems –Cohort effect: era in which individuals are born affects how they respond in a study.

Longitudinal Design Same participants are studied repeatedly over time as they age. –Ex: test the same participants every 3 years: 9 yr old, 12 yr old, 15 yr old… Advantage: no cohort effect Disadvantage –Attrition rate increases: participants drop out of study –Expensive and time consuming –Participants who remain may not be a representative sample

Sequential Design Combination of longitudinal and cross-sectional designs Test 5 yr olds, 15 yr olds and 45 yr olds ever 3 years Advantages –Allow to examine age effects without taking as much times as a longitudinal study. Disadvantages –Expensive and time consuming –Attrition rates