5 Experimental design So .. The hypothesized relationship is Independent variable -> dependent variable
6 Experimental designAn independent variable is one that the researcher studies as a possible cause of something else.Dependent variable is a variable that is potentially influenced by the independent variable. Manipulation: Researcher manipulates the independent variable ( treatment – intervention)
7 Experimental design So .. The hypothesized relationship is Independent variable -> dependent variable
8 Experimental design So .. The hypothesized relationship is Independent variable -> dependent variableCertain Drug impact on cancer
9 Experimental designA researchers can convincingly identify cause-and-effect relationships by experimental designMany possible factors that might cause or influence a particular condition or phenomena.The researcher attempts to control for all influential factors except the study factor.
10 Controlling for confounding variables 1- keep something constant2- Include a control group3- Randomly assign people to group4- Assess equivalence before the treatment with one or more pretests .5- Expose participants to all experimental conditions6- statistically control for confounding variables.
11 Internal ValidityRegarding cause-effect or causal relationships, internal Validity is the approximate truth about inferences.It is relevant in studies that try to establish a causal relationship.The key question in internal validity is whether observed changes can be attributed to your program or intervention (i.e., the cause) and not to other possible causes.Did the treatment cause the outcome to occur? Or were there other confounding factors that caused the outcome?shorted
12 External ValidityAre the findings unique to just participants we studied or could they apply to other groups?Refers to the extent to which the results of an experiment can be generalized across populations, time and settings.shorted
13 Single Group ThreatsImagine that we are studying the effects of an education program in mathematics for first grade students on a measure of math performance such as a standardized math achievement test.In the post-only design, we would give the first graders the program and then give a math achievement posttest.Consider what would happen if you observe a certain level of posttest math achievement or a change or gain from pretest to posttest.You want to conclude that the outcome is due to your math program. How could you be wrong?
14 Single Group ThreatsHistory Threat: It's not your math program that caused the outcome, it's something else, some historical event that occurred. Maturation Threat: The children would have had the exact same outcome even if they had never had your special math training program. All you are doing is measuring normal maturation or growth in understanding that occurs as part of growing up. Testing Threat: This threat only occurs in the pre-post design. What if taking the pretest made some of the children more aware of that kind of math problem -- it "primed" them for the program so that when you began the math training they were ready for it in a way that they wouldn't have been without the pretest.
15 Single Group ThreadsInstrumentation Threat: Like the testing threat, this one only operates in the pretest-posttest situation. What if the change from pretest to posttest is due not to your math program but rather to a change in the test that was used? Mortality Threat: It means that people are "dying" with respect to your study.Regression Threat: Also known as a "regression artifact" or "regression to the mean" is a statistical phenomenon that occurs whenever you have a nonrandom sample from a population and two measures that are imperfectly correlated.
16 Design Categories Pre-experimental designs True Experimental designs Pre-experimental designsTrue Experimental designsQuasi-experimental designsEx post facto designsFactorial designs
17 Design CategoriesSome are true experimental designs as such they allow us to identify cause-and-effect Some give alternative explanations of an observed change.
18 Design CategoriesAll of the designs have one thing in common : clearly identify independent and dependent variable.
19 Design CategoriesWe illustrate the designs using tables that have this general format "Table"Group Time ->Group 1Group 2
20 Pre- Experimental Design The cells have one of four notations:Tx: Indicates that a treatment is presentedObs: Indicates that observation is made: Indicates that nothing occurs during a particular time period.Exp: Indicates a previous experience
21 Pre- Experimental Design: One-shot Experimental Case Study The most primitive type of experiment that might be termed "research“The design has low internal validity 1- the characteristics or the behavior observed after the treatment existed before the treatment as well.2- influenced by other factors 3- a single measurement or observation doesn't guarantee that situation has change or not. Group 1TxObs
22 Pre- Experimental Design: One-shot Experimental Case Study Example ..The design will be something like this :Exposure to cold + Damp ground (TX)-> Child has a cold (Obs)One-shot experimental case study is simple to carry out, its results are meaningless.
23 Pre- Experimental Design: One- Group Pretest-posttest Design We know that a change has taken a place. But we have not ruled out other possible explanation for the change. Group 1ObsTx
24 Pre- Experimental Design: Static Group Comparison Involves both an experimental group and a control group.No attempt is made to check wither they are similar or not before the treatment so no way to know if the treatment actually causes any differences between the two groups. Group 1TxObsGroup 2
25 Types of DesignIf random assignment is used, we call the design a randomized experiment or true experiment.If random assignment is not used, then we have to ask a second question: Does the design use either multiple groups or multiple waves of measurement?If the answer is yes, we would label it a quasi-experimental design. If no, we would call it a non-experimental design.
26 True Experimental Design True experimental design is regarded as the most accurate form of experimental research, in that it tries to prove or disprove a hypothesis mathematically, with statistical analysis.For an experiment to be classed as a true experimental design, it must fit all of the following criteria:The sample groups must be assigned randomly.There must be a viable control group.Only one variable can be manipulated and tested. The tested subjects must be randomly assigned to either control or experimental groups.
27 True Experimental Design: Pretest-Posttest Control Group Design The pretest-posttest equivalent groups design provides for both a control group and a measure of change but also adds a pretest.It is important that the two groups be treated in a similar manner.Random AssignmentGroup 1ObsTxGroup 2
28 True Experimental Design: The Solomon Four-Group Design The Solomon Four-Group Design is designed to deal with a potential testing thread. This design has four groups. Two of the groups receive the treatment and two do not. Random AssignmentGroup 1ObsTxGroup 2-Group 3Group 4clip
29 Quasi-Experimental Designs Sometimes, randomness is either impossible or impractical. In those situations use quasi-experimental design.However, has features that can eliminate many threats to internal validity.Used frequently in evaluation because:Often randomization is impossible or difficultEthical/legal prohibitions against randomizationNo viable control group availableInadequate resources to conduct randomizationclip
30 Quasi-Experimental Designs: Nonrandomized Control Group Pretest-Posttest Design To show that two groups are equivalent with respect to the dependent variable prior to the treatment, thus eliminating initial group differences as an explanation for post-treatment differences.Differs from experimental designs because test and control groups are not totally equivalent; equivalence on the pretest ensures equivalence only for variables that have specifically been measured.clipGroup 1ObsTxGroup 2-
31 Quasi-Experimental Designs: Nonrandomized Control Group Pretest-Posttest Design Threats to validity:Partly controls for history threat(external event would affect both groups, provided groups, provided groups are similar), maturation, instrument threats and instrumentation threats.However, even if groups are statistically very similar, if the intervention is given to volunteers they may behave differently than control group(due to self-selection).Regression threat.clip
32 Quasi-Experimental Designs: Simple time-series experiment To show that, for a single group change occurs during a lengthy period only after the treatment has been administered.Provides a stronger alternative to “One group pretest-posttest design; external validity can be increased by repeating the experiment in different places under different condition.Group 1ObsTxclip
33 Quasi-Experimental Designs: Simple time-series experiment Threats to validity:Extension of pretest-posttest design but reduces maturation, testing, regression threatsNo control group so no selection threat.History threat is controlled partially.if measurement changes around time of program instrumentation threat may be presentclip
34 Quasi-Experimental Designs: Control group, time-series design Bolstering the internal validity of the preceding design with the addition of a control group.Involves conducting parallel series of observations for experiment and control groups.Group 1ObsTxGroup 2-clip
35 Quasi-Experimental Designs: Reversal, time-series design Showing , in a single group or individual, that a treatment consistently leads to a particular effect.Is an on-again, off-again design in which the experimental treatment is sometimes present, sometimes absent.Group 1TxObs-clip
36 Quasi-Experimental Designs: Alternating treatments design Showing , in a single group or individual, that different treatments have different effects.Involves sequentially administrating different treatments at different times and comparing their effects against the possible consequent of non-treatment.Group1TxaObs-Txbclip
37 Quasi-Experimental Designs: Multiple-baseline design Showing, the effect of a treatment by initiating at different times for different groups or individuals, or perhaps in different setting for a single individual.Involves tracking two or more groups or individuals over time, or tracking a single individual in two or more settings, for a lengthy period of time, as well as initiating the treatment at different times for different groups, individuals, or settings.clipGroup1-ObsTxGroup2
38 Quasi-Experimental Designs: Alternating treatments design Showing , the effect of a treatment by initiating it at different times forInvolves sequentially administrating different treatments at different times and comparing their effects against the possible consequent of non-treatment.Group1TxaObs-Txbclip
39 Ex Post Facto Designs Sometimes we can’t manipulate some variables Impossible, e.g.: CharacteristicsUnethical, e.g.: VirusEx post facto = after the factAlready happened in the past
40 Ex Post Facto: Simple Design Possible effect of an experience/condition that occurred in the pastMay show difference but not conclusiveGroup Time ->Group 1ExpObsGroup 2-
41 Factorial Designs 2+ independent variables Simultaneously or sequentialTime ->
42 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
43 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
44 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
45 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
46 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
47 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
48 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
49 Factorial Design: Two-factor experimental design Group Time ->Random AssignmentGroup 1Tx1Tx2ObsGroup 2-Group 3Group 4
50 Factorial Design: Two-factor experimental design Example: Clustering using K-meansTx1: A different method to choose initial centroidsTx2: A different formula to calculate distance
51 Factorial Design: Combined experimental and ex post facto design Group Time ->Group1ExpaRandom AssignmentGroup 1aTxaObsGroup 1bTxbGroup2ExpbGroup 2aGroup 2b
52 Factorial Design: Combined experimental and ex post facto design Example: Clustering using K-meansExpa, Expa: 2 different kinds of datasetsTxa, Txb: 2 different formulas to calculate distance