Download presentation
Presentation is loading. Please wait.
Published bySpencer Burns Modified over 8 years ago
1
Experimental Design, Statistical Analysis CSCI 4800/6800 University of Georgia March 7, 2002 Eileen Kraemer
2
Research Design Elements: Observations/Measures Treatments/Programs Groups Assignment to Group Time
3
Observations/Measure Notation: ‘O’ Examples: Body weight Time to complete Number of correct response Multiple measures: O 1, O 2, …
4
Treatments or Programs Notation: ‘X’ Use of medication Use of visualization Use of audio feedback Etc. Sometimes see X+, X-
5
Groups Each group is assigned a line in the design notation
6
Assignment to Group R = random N = non-equivalent groups C = assignment by cutoffs
7
Time Moves from left to right in diagram
8
Types of experiments True experiment – random assignment to groups Quasi experiment – no random assignment, but has a control group or multiple measures Non-experiment – no random assignment, no control, no multiple measures
9
Design Notation Example RO1O1 XO 1,2 RO1O1 Pretest-posttest treatment comparison group randomized experiment
10
Design Notation Example NOXO NOO Pretest-posttest Non-Equivalent Groups Quasi-experiment
11
Design Notation Example XO Posttest Only Non-experiment
12
Goals of design.. Goal:to be able to show causality First step: internal validity: If x, then y AND If not X, then not Y
13
Two-group Designs Two-group, posttest only, randomized experiment RXO RO Compare by testing for differences between means of groups, using t-test or one-way Analysis of Variance(ANOVA) Note: 2 groups, post-only measure, two distributions each with mean and variance, statistical (non-chance) difference between groups
14
To analyze … What do we mean by a difference?
15
Possible Outcomes:
16
Measuring Differences …
17
Three ways to estimate effect Independent t-test One-way Analysis of Variance (ANOVA) Regression Analysis (most general) equivalent
18
Computing the t-value
19
Computing the variance
20
Regression Analysis Solve overdetermined system of equations for β 0 and β 1, while minimizing sum of e-terms
21
Regression Analysis
22
ANOVA Compares differences within group to differences between groups For 2 populations, 1 treatment, same as t-test Statistic used is F value, same as square of t-value from t-test
23
Other Experimental Designs Signal enhancers Factorial designs Noise reducers Covariance designs Blocking designs
24
Factorial Designs
25
Factorial Design Factor – major independent variable Setting, time_on_task Level – subdivision of a factor Setting= in_class, pull-out Time_on_task = 1 hour, 4 hours
26
Factorial Design Design notation as shown 2x2 factorial design (2 levels of one factor X 2 levels of second factor)
27
Outcomes of Factorial Design Experiments Null case Main effect Interaction Effect
28
The Null Case
30
Main Effect - Time
31
Main Effect - Setting
32
Main Effect - Both
33
Interaction effects
34
Interaction Effects
35
Statistical Methods for Factorial Design Regression Analysis ANOVA
36
Analysis of variance – tests hypotheses about differences between two or more means Could do pairwise comparison using t- tests, but can lead to true hypothesis being rejected (Type I error) (higher probability than with ANOVA)
37
Between-subjects design Example: Effect of intensity of background noise on reading comprehension Group 1: 30 minutes reading, no background noise Group 2: 30 minutes reading, moderate level of noise Group 3: 30 minutes reading, loud background noise
38
Experimental Design One factor (noise), three levels(a=3) Null hypothesis: 1 = 2 = 3 NoiseNoneModerateHigh ROOO
39
Notation If all sample sizes same, use n, and total N = a * n Else N = n 1 + n 2 + n 3
40
Assumptions Normal distributions Homogeneity of variance Variance is equal in each of the populations Random, independent sampling Still works well when assumptions not quite true(“robust” to violations)
41
ANOVA Compares two estimates of variance MSE – Mean Square Error, variances within samples MSB – Mean Square Between, variance of the sample means If null hypothesis is true, then MSE approx = MSB, since both are estimates of same quantity Is false, the MSB sufficiently > MSE
42
MSE
43
MSB Use sample means to calculate sampling distribution of the mean, = 1
44
MSB Sampling distribution of the mean * n In example, MSB = (n)(sampling dist) = (4) (1) = 4
45
Is it significant? Depends on ratio of MSB to MSE F = MSB/MSE Probability value computed based on F value, F value has sampling distribution based on degrees of freedom numerator (a-1) and degrees of freedom denominator (N-a) Lookup up F-value in table, find p value For one degree of freedom, F == t^2
46
Factorial Between-Subjects ANOVA, Two factors Three significance tests Main factor 1 Main factor 2 interaction
47
Example Experiment Two factors (dosage, task) 3 levels of dosage (0, 100, 200 mg) 2 levels of task (simple, complex) 2x3 factorial design, 8 subjects/group
48
Summary table SOURCE df Sum of Squares Mean Square F p Task 1 47125.3333 47125.3333 384.174 0.000 Dosage 2 42.6667 21.3333 0.174 0.841 TD 2 1418.6667 709.3333 5.783 0.006 ERROR 42 5152.0000 122.6667 TOTAL 47 53738.6667 Sources of variation: Task Dosage Interaction Error
49
Results Sum of squares (as before) Mean Squares = (sum of squares) / degrees of freedom F ratios = mean square effect / mean square error P value : Given F value and degrees of freedom, look up p value
50
Results - example Mean time to complete task was higher for complex task than for simple Effect of dosage not significant Interaction exists between dosage and task: increase in dosage decreases performance on complex while increasing performance on simple
51
Results
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.