1. Diversity Awareness Training Sanchez & Medkik Hypothesis Nature of quasi-experimental design Measures used & their validity Tests of Hypotheses Alternative explanations for results Learning Points

2. Hypothesis Diversity Awareness Training Differential Treatment of Culturally Different Individuals Cultural Awareness

3. Method Participants –125 supervisors/mgrs in a county government –125 Raters of supervisors/mg above Are no raters evaluating two supervisors? ConditionNumber of Participants Diversity Training69 Control56

4. Method Design –No random assignment to conditions Participants in experimental group were chosen bec.. –they were one of 4 employees with the longest tenure in each of their departments & –had not received diversity awareness training Participants in control group were matched on tenure with those in the experimental group –Control group Ps would have been eligible for training –Did Ps in control group receive training before?

5. Measures Pre-training performance ratings –Relevant to training dimensions (e.g., Coworker contact, communication skills) –Extracted for the year immediately before training –5-point rating scales (# of items not specified) –Anchors used poor to excellent Issues –Reliability not given

6. Method Matched control and experimental groups on tenure Control variables –Pre-training performance rating –Demographics Gender, ethnicity, tenure, educational level –Demographics of coworkers who rated Ps in control & experimental groups Gender & ethnicity

7. Establishing equivalence Tested for mean differences between experimental and control group on –Matching and Control variables –Do not present appropriate statistical test results for means but present sds Present means for categorical variables(!) –Present correlational information

8. VariableControlTraining Gender.66 Ethnicity.67.76 Tenure2.592.69 Educational level3.823.99 Rater’s ethnicity.38.33 Rater’s gender.56.50 Means on Demographic Var

9. Means on Continuous Control Variables Pre-training Performance ControlExperimental Coworker contact4.144.10 Communication4.174.12 Diversity training is not significantly correlated with any of these variables

10. Training Outcome Measures Trainee reactions –6-items –5-point Likert rating scales –Reliability=.98 –Completed immediately after training –Only completed by experimental group Usefulness of mean data

11. Training Outcome Measures Cultural Awareness –Correctly pair nine-terms with their meanings –Completed 1 year after training –Previously developed scale called CAI –Reliability=.75

12. Training Outcome Measures Differential Treatment Ratings –Coworkers’ ratings of how Ps treated those who were culturally different from Ps –1 year after training –Previously developed discrimination scale –10 items rated on 5 point scale –Reliability=.98

13. Means on Outcome Variables VariableControlExperimental Cultural Awareness5.666.24 Differential Treatment 1.261.44 Diversity training is not significantly correlated with any of these variables

14. Validity of Rater Sample No differences between participant and rater sample on –Proportions of men & women –Proportions of Whites & VMs Correlation b/w post-training measures and performance ratings, between supervisor and peer performance ratings –Did raters know whether target was in the experimental vs. control group? Higher expectations

15. Preliminary Analyses Significant correlations between (control) demographic variables –Tenure & Educational level= -.26 –Ethnicity and Educational Level= -.26 –Gender & Ethnicity= -.39 –Coding issues? 1=male, 0=female 1=White, 0=VM 1=less than 5 years, 5=21 years or more tenure Educational level coding not provided

16. Preliminary Analyses Significant correlations between pre- training performance & demographic variables –Coworker Contact & Communication=.50 –Coworker contact & gender=-.24 –Communication & gender=-.22 –Communication & education=.25 –Coding: 1=male, 0=female

17. Validity of Outcome Variable Trainee reactions not related to any variable –Usefulness of trainee reactions –Statistical Power issues

18. Validity of Outcome Variable Significant correlations with b/w Cultural awareness & control variable –Coworker contact performance =.27 Criterion validity of outcome variable –Ethnicity=.30 1=White, 0=VM –Education level=.50

19. Significant correlations between Differential treatment & control variables –Gender=.20 ( 1=male, 0=female) –Rater ethnicity=.30 (1=White, 0=VM OR 1=VM 0=White) –But no correlation b/w DT & pre-training performance rating Implications for –Using type of raters –Criterion validity of differential treatment ratings »Do supervisors have opportunity to notice differential treatment? Validity of Outcome Variable

20. Hypothesis Testing Regression analyses to test for mediation effects requires –Independent and dependent variable to be related –Mediator variable to be related to both independent & dependent variables –Criteria not met for… IV=Training Mediator=Cultural Awareness DV=Differential treatment BUT….forging ahead!

21. Hypothesis Testing of fake data? Step PredictorR2R2 ΔR2ΔR2 β 1Participant Ethnicity13 -17 Rater Ethnicity25* 2Training190625* 3Training19025* Cultural Awareness12 3eRater Ethnicity x Training260744*

22. Graph of Interaction

23. Discussion Lack of support for hypotheses –Diversity training did not have any effect on social perception biases Educational level & participant ethnicity predicted cultural awareness Trainee reactions were positive(!) –Uselessness of these types of measures

24. Alternative Explanations Diffusion of treatment among controls –Not supported by higher differential treatment ratings given to trained participants Selection bias –Lack of differences on control variables including pre-training performance ratings Trainees held to higher standard by non-white raters

25. Alternative Explanations Qualitative analyses of interviews with non-white raters of trainees –Possible backlash due to Lack of information re: purpose of training Timing of post-test= need for post-training support Pre-training beliefs & feelings –Usefulness of non-white raters who interact w/diversity trainee

26. Learning Points from Article Writing up unexpected results Presentation of statistical results –Means vs. frequencies depends on type of variable Double check statistical results –Discrepancy between correlational and regression tables Analyses should also be guided by hypotheses