 Understanding the Variability of Your Data: Dependent Variable.

Presentation on theme: "Understanding the Variability of Your Data: Dependent Variable."— Presentation transcript:

Understanding the Variability of Your Data: Dependent Variable

Two "Sources" of Variability

Understanding the Variability of Your Data: Dependent Variable Two "Sources" of Variability –Independent (Predictor/Explanatory) Variable(s)

Understanding the Variability of Your Data: Dependent Variable Two "Sources" of Variability –Independent (Predictor/Explanatory) Variable(s) –Extraneous Variables

Understanding the Variability of Your Data: Dependent Variable Two Types of Variability

Understanding the Variability of Your Data: Dependent Variable Two Types of Variability –Unsystematic

Understanding the Variability of Your Data: Dependent Variable Two Types of Variability –Unsystematic –Systematic

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Error Variability - unsystematic due to extraneous variables

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Error Variability - unsystematic due to extraneous variables Within conditions variability

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Error Variability - unsystematic due to extraneous variables Within conditions variability Individuals in same condition affected differently

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Error Variability - unsystematic due to extraneous variables Within conditions variability Individuals in same condition affected differently Affects standard deviation, not mean, in long term

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Error Variability - unsystematic due to extraneous variables Common sources individual differences procedural variations measurement error

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Primary Variability – systematic due to independent variable

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Primary Variability – systematic due to independent variable Between conditions variability

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Primary Variability – systematic due to independent variable Between conditions variability Individuals in same condition affected similarly

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Primary Variability – systematic due to independent variable Between conditions variability Individuals in same condition affected similarly Individuals in different conditions affected differently

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Primary Variability – systematic due to independent variable Between conditions variability Individuals in same condition affected similarly Individuals in different conditions affected differently Affects mean, not standard deviation, in long term

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Secondary Variability – systematic due to extraneous variable

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Secondary Variability – systematic due to extraneous variable Between conditions variability

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Secondary Variability – systematic due to extraneous variable Between conditions variability Individuals in same condition affected similarly

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Secondary Variability – systematic due to extraneous variable Between conditions variability Individuals in same condition affected similarly Individuals in different conditions affected differently

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Secondary Variability – systematic due to extraneous variable Between conditions variability Individuals in same condition affected similarly Individuals in different conditions affected differently Affects mean, not standard deviation, in long term

Understanding the Variability of Your Data: Dependent Variable Roles played in the Research Situation –Error Variability A nuisance – the ‘noise’ in the research situation

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Error Variability A nuisance – the ‘noise’ in the research situation –Primary Variability The focus – the potentially meaningful effect

Understanding the Variability of Your Data: Dependent Variable Three "labels" for the variability –Error Variability A nuisance – the ‘noise’ in the research situation –Primary Variability The focus – the potentially meaningful effect –Secondary Variability The ‘evil’ – confounds the results

Example Two sections of the same course

Example Two sections of the same course Individual’s score as combination of ‘sources’

Statistical decision-making The logic behind inferential statistics Deciding if there is ‘systematic variability’ –primary vs. secondary What do the data tell us? What decisions should we make?

Statistical decision-making A Research Example –Research Hypothesis –IF students chant the “Statistician’s Mantra” before taking their Methods exam THEN they will earn higher scores on the exam.

Statistical decision-making A Research Example Your Class (M = 80, SD = 15, n = 25) compared to a known population Mean (M = 70) for a standardized exam

Statistical decision-making A Research Example Can estimate the Sampling Distribution See if Population mean ‘fits’ Cause effect relationship not clear

Statistical decision-making A Research Example using experimental approach –Research Hypothesis –IF students chant the “Statistician’s Mantra” (vs. not chanting) before taking their Methods exam THEN they will earn higher scores on the exam.

Statistical decision-making Procedure –Randomly divide class into two groups Chanters – are taught the “Statistician’s Chant” and chant together for 5 minutes before the exam

Statistical decision-making Procedure –Randomly divide class into two groups Chanters – are taught the “Statistician’s Chant” and chant together for 5 minutes before the exam Non-chanters – sing Kumbaya together for 5 minutes before the exam

Statistical decision-making Results –Compute exam scores for all students and organize by ‘condition’ (levels of IV).

Show ‘changing’ distribution

Statistical decision-making Results –Compute exam scores for all students and organize by ‘condition’ (levels of IV). –Compare Mean Exam Scores for two Conditions

Statistical decision-making Results –Compute exam scores for all students and organize by ‘condition’ (levels of IV). –Compare Means Exam Scores for two Conditions –What will you find?

Statistical decision-making Research Hypotheses generally imprecise –Predictions are not specific –So “testing” the Research Hypothesis, using the available data, not reasonable

Statistical decision-making Null Hypothesis – a precise alternative –Identifies outcome expected when NO systematic variability is present

Statistical decision-making Null Hypothesis – a precise alternative –Identifies outcome expected when NO systematic variability is present –But still must decide how close to the predicted outcome you must be to ‘believe’ in the Null Hypothesis

Statistical decision-making The Null Hypothesis Sampling Distribution

Statistical decision-making The Null Hypothesis Sampling Distribution –All possible outcomes when the Null Hypothesis is true (when there is no ‘systematic’ variability present in the data)

Statistical decision-making The Null Hypothesis Sampling Distribution –All possible outcomes when the Null Hypothesis is true –Finding all the possible outcomes?

Statistical decision-making The Null Hypothesis Sampling Distribution –All possible outcomes when the Null Hypothesis is true –Finding all the possible outcomes? –Seeing where your results fit into the Null Hypothesis Sampling Distribution

Statistical decision-making Deciding what to conclude based on the ‘fit’

Statistical decision-making Deciding what to conclude based on the ‘fit’ “True” State of the World Ho TrueHo False Reject Ho Error Correct Rejection Decision Not Reject HoCorrect Error Nonreject

Statistical decision-making Deciding what to conclude based on the ‘fit’ “True” State of the World Ho TrueHo False Reject Ho Type 1 (p) Correct Rejection Decision (power = 1 – Type 2) Not Reject HoCorrect Type 2 Nonrejection Deciding what confidence you want to have that you have not made any errors

Statistical decision-making Trade-offs between Types of Errors –I believe I can fly?

Statistical decision-making Trade-offs between Types of Errors Factors affecting Type 2 Errors (Power) –“Real” systematic variability (size of effect) –Choice of Type 1 probability –Precision of estimates (sample size)

Statistical decision-making Trade-offs between Types of Errors Factors affecting Type 2 Errors (Power) –“Real” systematic variability (size of effect) Assume.5 * SD, a moderate size effect is good –Choice of Type 1 probability Use traditional.05 –Precision of estimates (sample size) Sample of 50 (2 groups of 25)

Statistical decision-making Factors affecting Type 2 Errors (Power) –Type 2 error probability =.59 –Power =.41

Statistical decision-making Each ‘Decision” has an associated ‘error’ Can only make Type 1 if “Reject” Can only make Type 2 if “Not Reject”

Statistical decision-making Interpreting “Significant” Statistical Results Having decided to “reject” the Null Hypothesis you can: –State probability of Type 1 error –State confidence interval for population value –State percent of variability in DV ‘accounted for’

Statistical decision-making Interpreting “Significant” Statistical Results For Chant vs. No Chant example –State probability of Type 1 error.05 –State confidence interval for population value 95% CI is approximately +2 * SE Point estimate of 10 + 8 (Real difference between 2 and 18) –State percent of variability in DV ‘accounted for’ Eta 2 =.20, or 20%

Statistical decision-making Interpreting “Significant” Statistical Results Statistical Significance vs. Practical Significance How unlikely is the event in these circumstance –versus How much of an effect was there

Statistical decision-making Interpreting “Non-significant” Statistical Results Having decide you cannot reject the Ho State the estimated ‘power’ of your design