Chapter 2 Research Methods in Organizational Psychology SOP6669 Dr. Steve

Methods:  Experiment / Quasi-Experiment  Questionnaire/Survey  Naturalistic Observation  Case Study  Meta-Analysis Research Methods

Research Methods Experiment Study conducted in a contrived environment  Benefits: Provides more safety Provides more safety Cause and effect relationships Cause and effect relationships Manipulate I.V. (e.g., leadership style)Manipulate I.V. (e.g., leadership style) Measure D.V. (e.g., task performance)Measure D.V. (e.g., task performance) Control extraneous variables (e.g., experience)Control extraneous variables (e.g., experience)  Disadvantages: Time consuming Time consuming Quasi-Experiment – not randomized or unable to manipulate IV (e.g., gender)

Self-report to obtain data on attitudes/behaviors conducted by phone, mail, interviews, electronically  Benefits: Can collect a large quantity of data Can collect a large quantity of data  Disadvantages: Accuracy of reporting Accuracy of reporting Representativeness of sample Representativeness of sample Return rate Return rate Research Methods Questionnaire/Survey

Observe overt behaviors over time Systematic sampling at various times Systematic sampling at various times Representative sample Representative sample  Benefits: Use to generate hypotheses Use to generate hypotheses  Disadvantages: Experimenter bias Experimenter bias Obtrusiveness Obtrusiveness Frequency of behavior occurring Frequency of behavior occurring Research Methods Naturalistic Observation

In depth view of past events using interviews and archival records  Benefits: Detailed account of why particular event occurred Detailed account of why particular event occurred  Disadvantages: Little generalizability Little generalizability Research Methods Case Study

Meta-analysis – statistical procedure that combines the results of many independent research findings on a single topic  Used to estimate true relationship  Measures effect size of findings  Uses archival data Data Analysis Meta-analysis

Descriptive vs. Inferential Statistics  Descriptive stats merely describe data Frequency Frequency Central tendency Central tendency Variability Variability  Inferential stats used to test hypotheses T-Test T-Test Analysis of variance Analysis of variance Correlation Correlation Regression Regression Non-parametrics Non-parametrics Research Steps Statistical Analysis

Data Analysis Central Tendency 1.Mean – average: X = ∑X / N Mean = 72 / 8 = 9 2. Median – middle score (when placed in order) -use when outliers exaggerate the mean Median = Mode – most often occurring score Mode = 6 _ example scores = 5, 6, 6, 8, 9, 10, 11, 17 * In a normal distribution, Mean = Median = Mode

 Range - distance between highest and lowest score (Range = High score – Low score) (Range = High score – Low score) Range = 17 – 5 = 12 Range = 17 – 5 = 12  Standard Deviation – average distance from the mean S=  Σ(x – x) 2 / n – 1 S=  Σ(x – x) 2 / n – 1 Data Analysis Variability S =  (5-9) 2 + (6-9) 2 + (6-9) 2 + (8-9) 2 + (9-9) 2 + (10-9) 2 + (11-9) 2 + (17-9) 2 / 7 S = 3.85

Positively Skewed Distribution Negatively Skewed Distribution Normal or Bell-shaped Distribution Data Analysis Skewed Frequency Distributions

Correlation ( r ) – Degree of relationship between two variables Used for prediction Used for prediction Cannot be used to infer causation Cannot be used to infer causation Range from –1 to +1 Range from –1 to +1 Negative r – as one variable increases the other decreases Negative r – as one variable increases the other decreases Positive r – as one variable increases so does the other Positive r – as one variable increases so does the other Zero r – no relationship between the two variables Zero r – no relationship between the two variables Data Analysis Correlation rABC A1.0 B C

Positive Correlation Negative Correlation * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * * * * * ** * * * * * * * * * * * ** * * * * * * * * * * ** * * * * * * * * * ** * * * * * * * * * * ** * * * * * * * * * * ** * * * * ** * * * * * * * * * ** * * * * * * * * * * * ** * * * * * * * * * * ** * * * * * * * * * ** * * * * * * * * * * ** * * * * * * * * * * ** * * * * ** * *

 IQ scores of identical twins: r = +.86  Phases of the moon & # acts of violence: r =.00  Economic conditions & # lynchings: r = -.43  Amount of ice cream sold & # drownings: r = +.60  Price of rum in Cuba & priests salaries in New England: r = +.38  Number of cigarettes smoked per day & incidence of lung cancer: r = ??? Correlation Examples

Regression Variables (used for prediction) Y i = ß 0 + ß 1 X i1 + ß 2 X i2 (Y = a + b1X1)  Predictor Variable (X) – measure used to predict an outcome (similar to independent variable) Example: selection test scores, years of experience, education level Example: selection test scores, years of experience, education level  Criterion Variable (Y) – outcome to be predicted Example: work performance, turnover, sales, absenteeism, promotion, etc. Example: work performance, turnover, sales, absenteeism, promotion, etc.  Example: AFOQT scores as predictors of pilot training performance Statistical Methods Regression

Statistical Pitfalls: Bias  Representative Sampling Selecting a sample that parallels the population Selecting a sample that parallels the population Might use covariates to account for differences Might use covariates to account for differences  Statistical Assumptions ANOVA assumes a normal distribution and independence ANOVA assumes a normal distribution and independence Lack of normality is only minor problem, but may want to identify distribution shape and whyLack of normality is only minor problem, but may want to identify distribution shape and why Observations may not be independent, may need to aggregate (e.g., class instead of student)Observations may not be independent, may need to aggregate (e.g., class instead of student)

Statistical Pitfalls: Errors in Methodology  Statistical Power – probability of detecting a true difference of a particular size Type I error – falsely reject null hypothesis when a true difference does not exist Type I error – falsely reject null hypothesis when a true difference does not exist Type II error – fail to reject null hypothesis when a true difference does exist Type II error – fail to reject null hypothesis when a true difference does exist Power affected by Power affected by Sample sizeSample size Effect size (e.g., Cohen’s D)Effect size (e.g., Cohen’s D) Type I error rate selected (alpha)Type I error rate selected (alpha) Variability of sampleVariability of sample (F ratio = var between group / var within group) (F ratio = var between group / var within group)

Statistical Pitfalls: Errors in Methodology  Multiple Comparisons – if you compare enough variables, will find a relationship by chance alone Bonferroni correction – family-wise adjustment Bonferroni correction – family-wise adjustment (alpha =.05 / #comparisons) Replicate Replicate Cross-validate (holdout sample) Cross-validate (holdout sample)  Measurement Errors Reliability: Consistency of Measure Reliability: Consistency of Measure Validity: Measures what it was designed to measure Validity: Measures what it was designed to measure

Statistical Pitfalls: Problems with Interpretation  Confusion over significance P value does not reflect effect size – could have a small effect, but a lot of power P value does not reflect effect size – could have a small effect, but a lot of power  Precision vs. Accuracy More decimals not necessarily more accurate More decimals not necessarily more accurate  Causality Correlations are not causal, but ANOVA may not be either Correlations are not causal, but ANOVA may not be either

Statistical Pitfalls: Problems with Interpretation  Graphs May not provide accurate portrayal of data May not provide accurate portrayal of data

Always think critically about the research you read Who were the participants in the study? Who were the participants in the study? How strong of a relationship was found? How strong of a relationship was found? Was it causal or correlational? Was it causal or correlational? Was it a field study or laboratory study? Was it a field study or laboratory study? How was data collected and analyzed? How was data collected and analyzed? Do you agree with the conclusions based on the analyses provided? Do you agree with the conclusions based on the analyses provided? Research Critical Thinking

Ethical Principles of Research  Privacy: Participants have the right to limit the amount of information they reveal about themselves. If they decide to withdraw from the experiment at any time, they have the right to do so Participants have the right to limit the amount of information they reveal about themselves. If they decide to withdraw from the experiment at any time, they have the right to do so  Confidentiality: Participants have the right to decide to whom they reveal confidential information. By ensuring confidentiality, researchers may be able to obtain more honest responses Participants have the right to decide to whom they reveal confidential information. By ensuring confidentiality, researchers may be able to obtain more honest responses  Protection from Deception: Deception can only be used if the value of the research must outweigh the harm imposed on participants and the phenomenon cannot be measured any other way Deception can only be used if the value of the research must outweigh the harm imposed on participants and the phenomenon cannot be measured any other way