Presentation on theme: "Repeated Measures ANOVA"— Presentation transcript:
1 Repeated Measures ANOVA Quantitative Methods in HPELS440:210
2 Agenda Introduction The Repeated Measures ANOVA Hypothesis Tests with Repeated Measures ANOVAPost Hoc AnalysisInstatAssumptions
3 IntroductionRecall There are two possible scenarios when obtaining two sets of data for comparison:Independent samples: The data in the first sample is completely INDEPENDENT from the data in the second sample.Dependent/Related samples: The two sets of data are DEPENDENT on one another. There is a relationship between the two sets of data.
4 Introduction Three or more data sets? If the three or more sets of data are independent of one another Analysis of Variance (ANOVA)If the three or more sets of data are dependent on one another Repeated Measures ANOVA
5 Agenda Introduction The Repeated Measures ANOVA Hypothesis Tests with Repeated Measures ANOVAPost Hoc AnalysisInstatAssumptions
6 Repeated Measures ANOVA Statistical Notation Recall for ANOVA:k = number of treatment conditions (levels)nx = number of samples per treatment levelN = total number of samplesN = kn if sample sizes are equalTx = SX for any given treatment levelG = STMS = mean square = variance
7 Repeated Measures ANOVA Additional Statistical Notation:P = total score for each subject (personal total)Example: If a subject was assessed three times and had scores of 3, 4, 5 P = 12
8 Repeated Measures ANOVA Formula Considerations Recall for ANOVA:SSbetween = ST2/n – G2/NSSwithin = SSSinside each treatmentSStotal = SSwithin + SSbetweenSStotal = SX2 – G2/N
9 ANOVA Formula Considerations: dftotal = N – 1 dfbetween = k – 1 dfwithin = S(n – 1)dfwithin = Sdfin each treatment
14 Repeated Samples Designs One-group pretest posttest (repeated measures) design:Perform pretest on all subjectsAdminister treatments followed by posttestsCompare pretest to posttest scores and posttest to posttest scoresO X O X O
15 Agenda Introduction The Repeated Measures ANOVA Hypothesis Tests with Repeated Measures ANOVAPost Hoc AnalysisInstatAssumptions
16 Hypothesis Test: Repeated Measuers ANOVA Example 14.1 (p 457)Overview:Researchers are interested in a behavior modification technique on outbursts in unruly childrenFour students (n=4) are pretested on the # of outbursts during the course of one dayTeachers begin using “cost-response” techniqueStudents are posttested one week later, one month later and 6 months later
17 Hypothesis Test: ANOVA Questions:What is the experimental design?What is the independent variable/factor?How many levels are there?What is the dependent variable?
18 Step 1: State Hypotheses Non-DirectionalH0: µpre = µ1week = µ1month = µ6monthsH1: At least one mean is different than the othersTable B.4 (p 693)Critical value = 3.86Step 2: Set CriteriaAlpha (a) = 0.05Critical Value:Use F Distribution TableAppendix B.4 (p 693)Information Needed:dfbetween treatments = k – 1 = 4 – 1 = 3dferror = (N-k)-(n-1) = (16-4)-(4-1) = 9
19 Step 3: Collect Data and Calculate Statistic Total Sum of SquaresSStotal = SX2 – G2/NSStotal = 222 – 442/20SStotal =SStotal = 101Sum of Squares Between each TreatmentSSbetween treatment = ST2/n – SG2/NSSbetween treatment = 262/4+82/4+62/4+42/4 – 442/20SSbetween treatment = ( ) - 121SSbetween treatment = 77Sum of Squares Within each TreatmentSSwithin = SSSinside each treatmentSSwithin =SSwithin = 24Sum of Squares ErrorSSerror = SSwithin treatments – SSbetween subjectsSSerror =SSwithin = 11Sum of Squares Between each SubjectSSbetween subjects = SP2/k – SG2/NSSbetween subjects = (122/4+62/4+102/4+162/4) - 442/16SSbetween subjects = ( ) – 121SSbetween subjects = 13Raw data can be found in Table14.3 (p 457)
20 Step 3: Collect Data and Calculate Statistic F-Ratio F = MSbetween treatment / MSerrorF = / 1.22F = 21.04Mean Square Between each TreatmentMSbetween treatment = SSbetween treatment / dfbetween treatmentMSbetween treatment = 77 / 3MSbetween = 25.67Step 4: Make DecisionMean Square ErrorMSerrorn = SSerror / dferrorMSerror = 11 / 9MSwithin = 1.22
21 Agenda Introduction Repeated Measures ANOVA Hypothesis Tests with Repeated Measures ANOVAPost Hoc AnalysisInstatAssumptions
22 Post Hoc Analysis What ANOVA tells us: What ANOVA doesn’t tell us: Rejection of the H0 tells you that there is a high PROBABILITY that AT LEAST ONE difference exists SOMEWHEREWhat ANOVA doesn’t tell us:Where the differences liePost hoc analysis is needed to determine which mean(s) is(are) different
23 Post Hoc AnalysisPost Hoc Tests: Additional hypothesis tests performed after a significant ANOVA test to determine where the differences lie.Post hoc analysis IS NOT PERFORMED unless the initial ANOVA H0 was rejected!
24 Post Hoc Analysis Type I Error Type I error: Rejection of a true H0Pairwise comparisons: Multiple post hoc tests comparing the means of all “pairwise combinations”Problem: Each post hoc hypothesis test has chance of type I errorAs multiple tests are performed, the chance of type I error accumulatesExperimentwise alpha level: Overall probability of type I error that accumulates over a series of pairwise post hoc hypothesis testsHow is this accumulation of type I error controlled?
25 Two Methods Bonferonni or Dunn’s Method: Specific post hoc tests: Perform multiple t-tests of desired comparisons or contrastsMake decision relative to a / # of testsThis reduction of alpha will control for the inflation of type I errorSpecific post hoc tests:Note: There are many different post hoc tests that can be usedOur book only covers two (Tukey and Scheffe)
26 Repeated Measures ANOVA Bonferronni/Dunn’s method is appropriate with following consideration:Use related-samples T-testsTukey’s and Scheffe is appropriate with following considerations:Replace MSwithin with MSerror in all formulasReplace dfwithin with dferror in all formulasNote: Statisticians are not in agreement with post hoc analysis for Repeated Measures ANOVA
27 Agenda Introduction The Repeated Measures ANOVA Hypothesis Tests with Repeated Measures ANOVAPost Hoc AnalysisInstatAssumptions
28 Instat Label three columns as follows: Block: This groups your data by each subject.Example: If you conducted a pretest and 2 posttests (3 total) on 5 subjects, the block column will look like:Treatment: This tells you which treatment level/condition occurred for each data point.Example: If each subject (n=5) received three treatments, the treatment column will look like:Response: The data for each subject and treatment condition
29 Instat Convert the “Block” and “Treatment” columns into “factors”: Choose “Manage”Choose “Column Properties”Choose “Factor”Select the appropriate column to be convertedIndicate the number of levels in the factorExample: Block (5 levels, n = 5), Treatment (3 levels, k = 3)Click OK
30 Instat Choose “Statistics” Choose “General” Response variable: Choose “Analysis of Variance”Choose “General”Response variable:Choose the Response variableTreatment factor:Choose the Treatment variableBlocking factor:Choose the Block variableClick OK.Interpret the p-value!!!
31 Instat Post hoc analysis: Perform multiple related samples t-Tests with the Bonferonni/Dunn correction method
32 Reporting ANOVA Results Information to include:Value of the F statisticDegrees of freedom:Between treatments: k – 1Error: (N – k) – (n – 1)p-valueExamples:A significant treatment effect was observed (F(3, 9) = 21.03, p = 0.002)
33 Reporting ANOVA Results An ANOVA summary table is often includedSourceSSdfMSBetween77325.67F = 21.03Within Treatments2412Between subjects13Error1191.22Total10115
34 Agenda Introduction The Analysis of Variance (ANOVA) Hypothesis Tests with ANOVAPost Hoc AnalysisInstatAssumptions
35 Assumptions of ANOVA Independent Observations Normal Distribution Scale of MeasurementInterval or ratioEqual variances (homogeneity)Equal covariances (sphericity)If violated a penalty is incurred
36 Violation of Assumptions Nonparametric Version Friedman Test (Not covered)When to use the Friedman Test:Related-samples design with three or more groupsScale of measurement assumption violation:Ordinal dataNormality assumption violation:Regardless of scale of measurement
37 Textbook AssignmentProblems: 5, 7, 10, 23 (with post hoc)