Presentation is loading. Please wait.

Presentation is loading. Please wait.

Experimental design ITS class December 2, 2004 ITS class December 2, 2004.

Similar presentations


Presentation on theme: "Experimental design ITS class December 2, 2004 ITS class December 2, 2004."— Presentation transcript:

1 Experimental design ITS class December 2, 2004 ITS class December 2, 2004

2 Quantitative research  Quantify relationships between variables  Measured on a sample of subjects  Relationships expressed through effect statistics  Correlations, relative frequencies, diff. between means  Quantify relationships between variables  Measured on a sample of subjects  Relationships expressed through effect statistics  Correlations, relative frequencies, diff. between means

3 Types of Study  Descriptive  Measure things as they are  Experimental  Take measurements  Try an intervention (treatment)  Take measurements  Descriptive  Measure things as they are  Experimental  Take measurements  Try an intervention (treatment)  Take measurements

4 Descriptive studies  Case  Case series  Cross-sectional  Cohort or prospective or longitudinal  Case-control or retrospective  Case  Case series  Cross-sectional  Cohort or prospective or longitudinal  Case-control or retrospective

5 Experimental studies  AKA longitudinal, repeated- measures, interventions  Without a control group  Time series  Crossover  With a control group  AKA longitudinal, repeated- measures, interventions  Without a control group  Time series  Crossover  With a control group

6 Control group studies  Necessary if treatment effect cannot be “washed out”  Random assignment: min. chance that either group is not typical of population  Single-blind: subjects  Double-blind: experimenter too  Best studies: data analyzed blind  Necessary if treatment effect cannot be “washed out”  Random assignment: min. chance that either group is not typical of population  Single-blind: subjects  Double-blind: experimenter too  Best studies: data analyzed blind

7 Ethical issues  Randomized controlled study may not be ethical  E.g.: Heart disease study over 10 years  Access to treatment  Inform subjects of randomization  Randomized controlled study may not be ethical  E.g.: Heart disease study over 10 years  Access to treatment  Inform subjects of randomization

8 Quality of designs  Quality of evidence for causal relationships  Least for case, case-series  Cross-sectional, case-control  Prospective  Experimental best evidence  Quality of evidence for causal relationships  Least for case, case-series  Cross-sectional, case-control  Prospective  Experimental best evidence

9 Confounding  Two variables may be related for other reasons  Control for potential confounding factors  Two variables may be related for other reasons  Control for potential confounding factors

10 Samples  Generalizability  Representative  Safest: Random sample  Balance:  Stratified random (%s)  Balance on pretest!  Selection bias:  Not representative  Sources: age, socioeconomic status  Self-selection into groups  High compliance  Generalizability  Representative  Safest: Random sample  Balance:  Stratified random (%s)  Balance on pretest!  Selection bias:  Not representative  Sources: age, socioeconomic status  Self-selection into groups  High compliance

11 Sample size  Statistical significance  Big enough to be sure you will detect the smallest worthwhile effect  To be sure =  detect 80% of time  Detect =  95% of time, expect a smaller value if there is no effect, p<0.05  Smallest worthwhile effect =  Smallest to make a diff. to subjects’ lives  Statistical significance  Big enough to be sure you will detect the smallest worthwhile effect  To be sure =  detect 80% of time  Detect =  95% of time, expect a smaller value if there is no effect, p<0.05  Smallest worthwhile effect =  Smallest to make a diff. to subjects’ lives

12 Statistical significance  P-value:  the probability of getting something more extreme than your result, when there is no effect in the population  P-value:  the probability of getting something more extreme than your result, when there is no effect in the population

13 Sources  esign.html: Good resource for basic stats esign.html:  : Website for EASy-D: a database of evaluation of adaptive systems, also links to research-based usability guidelines  esign.html: Good resource for basic stats esign.html:  : Website for EASy-D: a database of evaluation of adaptive systems, also links to research-based usability guidelines


Download ppt "Experimental design ITS class December 2, 2004 ITS class December 2, 2004."

Similar presentations


Ads by Google