Understanding Variability Unraveling the Mystery of the Data’s Message Becoming a “Data Whisperer”

Overview of Research Issues and Goals I.Idea – hypothesis Conceptualization – Nominal Definitions – Operational Definitions – Measurement Procedures II.Design (framework/procedure) – Strategy -- Implementation – Data Collection -- Measures III.Statistics – Exploratory – Descriptive – Inferential – Decisions IV.Conclusions in context – Interpretation -- Meaning -- Generalization

Search for Cause – Effect connections Information needed to establish a cause is a cause (J. S. Mill) Cause and Effect are related – they covary Cause comes before the Effect in time All alternative Causes are eliminated

Cause and Effect Connections can be Necessary and Sufficient – single clear cause Necessary, not Sufficient – combinations of or interactions of variables needed to get effect Sufficient, not Necessary – alternative independent causes exist

Goal of Research – to understand variability, how do variables vary (descriptive), when do they vary (covariation), and why do variables vary (causes) Assume that variables cannot vary without cause Research Strategies – depend upon research goals/purposes/constraints

Descriptive Research what variables exist and how do variables vary structure of a variable – qualitative or quantitative does the variable exist separately from others one-dimensional or multidimensional possible values and range of values variability within a sample/population

Covariation Research (when do variables vary - context) when variables vary, with what do they covary “response variable - response variable” covariation assess strength and direction of relationship “predictor variable – response variable” covariation use one variable to ‘explain’ or ‘predict’ variability in the other variable limitations on ‘causal’ interpretations

Search for Cause – Effect connections Information needed to establish a cause is a cause Cause and Effect are related – they covary Cause comes before the Effect in time Logic may support direction - All alternative Causes are eliminated

Group Difference Research - focus on typical behavior Quasi-experimental research – covariation related to ‘existing’ differences (grouping) “manipulation” by selection Experimental research – creating groups to control variation of some variables manipulation by control

Data Collection Issues Measurement scale – what type of information is provided qualitative and/or quantitative operations that can be performed on measures Source of measures direct physiological measures behavioral observations indirect self reports inferred secondary collected by others

Research Setting Controlled conditions Uncontrolled conditions reducing the noise to improve ability to hear the signal allowing noise to see if signal still has impact

Evaluating Research Strategies Internal Validity – can outcome be interpreted clearly enhanced by control over all variables and precision of activities External Validity – can outcome be generalized enhanced by inclusion of representative variables and activities

Research involves solving a mystery (Must decide on Guilt or Innocence – and hope that decision matches the unknown ‘truth’) understanding why things happen – search for TRUTH! How strong a case can you make, but NEVER 100% confident TRUTH DECISION Innocent Guilty Guilt wrong right Innocence right wrong

Research involves solving a mystery (Must decide on Guilt or Innocence – and hope that decision matches the unknown ‘truth’) understanding why things happen – search for TRUTH! (You are like CSI and Prosecutor) Research Strategy – collect the evidence… the data upon which your case will be based Statistics – organize, summarize, and evaluate the strength of the evidence Decide on your ‘interpretation’ of the evidence (Decide on your version of the “Truth” – guilt or innocence) Persuade others of the validity of your interpretation (Other Researchers are like Defense attorney – need only raise doubts!) validity of your research appropriateness of evaluation

The ‘Case” a body is found in the library – “who” and/or “what” caused the death? Law & Order: Psychological Research Unit Descriptive Research Covariation Research Causal Evidence Meaningful level of Explanation?

Main Characters – all the variables that might be involved Dependent Variable(s) – the ‘response’ variable always present, the basis for the data used to answer the question Independent Variable(s) – explanatory/predictor variable the suspected cause or variable related to the cause Extraneous Variables – all variables not cast in either of the roles above nuisance or confound – possible impacts (evil)(Orient Express) best dealt with in the context of the research strategy – design and procedures (Constant, Control, Random) alternatively, can be dealt with indirectly, using statistical adjustments

Qualities of the Main Characters – any variables which will be ‘measured’ Actually are Qualities of Operational Definitions Used reliability – precision and consistency validity – appropriateness and accuracy

Recap Research involves solving a mystery – understanding why things happen Research Strategy – collect the evidence… the data upon which your case will be based Statistics – organize, summarize, and evaluate the strength of the evidence Decide on your ‘interpretation’ of the evidence the logic of statistical decision-making Persuade others of the validity of your interpretation validity of your research appropriateness of evaluation

Research involves solving a mystery – understanding why things happen Research Strategy – collect the evidence… the data upon which your case will be base Issues: correct variables, measured well (operational definitions) design is appropriate for questions descriptive, covariation, causal absence of contamination (internal validity) can generalize beyond specifics (external validity)

Research involves solving a mystery – understanding why things happen Research Strategy – collect the evidence… the data upon which your case will be based Statistics – organize, summarize, and evaluate the strength of the evidence Issues: organize/summarize data (correct descriptive statistics) isolate signal from noise (correct inferential statistics) evaluate the signal (understand what the message is)

Research involves solving a mystery – understanding why things happen Research Strategy – collect the evidence… the data upon which your case will be based Statistics – organize, summarize, and evaluate the strength of the evidence Decide on your ‘interpretation’ of the evidence Issues: what do the results mean what decision will you make (guilt or innocence) how confident are you (probability you are wrong)

Research involves solving a mystery – understanding why things happen Research Strategy – collect the evidence… the data upon which your case will be based Statistics – organize, summarize, and evaluate the strength of the evidence Decide on your ‘interpretation’ of the evidence Persuade others of the validity of your interpretation Issues: validity of your research validity of your analyses of the data validity of your interpretation of the analyses

Mystery Solved