Presentation is loading. Please wait.

Presentation is loading. Please wait.

Institute for Cross-Cultural Management

Similar presentations


Presentation on theme: "Institute for Cross-Cultural Management"— Presentation transcript:

1   Assessing Cross-Cultural Competence: How Good Are the Available Instruments?
Institute for Cross-Cultural Management Florida Institute of Technology

2 Authors William Gabrenya, PhD - Cross-Cultural Psychology (USA)
Rana Moukarzel, MS - Industrial/Organizational Psychology (Lebanon) Marne Pomerance, MS – I/O (USA) Richard Griffith, PhD – I/O (USA)

3 Support DEOMI Contract FA2521-10-T-0087 Dan McDonald, PhD
Patrice Reid, PhD

4 Cross-Cultural Competence and the U.S. Military
Cross-Cultural Psychology watches Iraq disintegrate Case study in worst practices Sidelined culture experts How could this happen?

5 Culture Expertise Rediscovered
Defense Language Office (DLO) DEOMI Army Research Institute Services DLO project to try to keep track of it all…

6 Three Issues in Cross-Cultural Competence
Identify competencies Measure competencies Train competencies Utilize capabilities acquired at considerable cost

7 DLO Framework “Framework for Cross-Cultural Competency”
Competency model Organized set of requirements for acceptable performance Higher level generality to lower level, hierarchy Expressed behaviorally Two parts 3C competencies Antecedent variables (enablers) Revised

8 Competencies Name (# elements) Example C1
Culture-General Concepts and Knowledge (3) Comprehends and navigates intercultural dynamics C3 Cultural Perspective-Taking (3) Understands how one’s own group is viewed by members of another group C4 Communication (2) Acquires and applies knowledge and concepts of intercultural communication skills C5 Interpersonal Skills (2) Builds relationships in support of mission performance C6 Cultural Adaptability (2) Understands the implications of one’s actions and adjusts approach to maintain relationships with other groups, or cultures

9 Antecdents (Enablers)
Name (# elements) Example E1 Cognitive Bias Resilience (3) Accepts, or does not feel threatened by, ambiguous situations and uncertainty. E2 Emotional Resilience (4) Tolerates emotionally shocking, frustrating, or exhausting circumstances… E3 Self-Identity Resilience (3) Demonstrates ability to maintain personal values independent of situational factors E4 Learning Motivation (4) Is motivated to make sense of inconsistent information about social rules and norms… E5 Social Interaction (4) Actively seeks out and explores unfamiliar cross-cultural interactions …

10 Validation* of the Framework
Content validity The right competencies? Criterion validity Are these competencies related to performance and adjustment? *Analogously

11 Content Evaluation of the DLO Framework
Developed using subject matter experts (SMEs): Returned soldiers Experts in expatriate assignments Military psychologists We compared it to: Military models Civilian models New SME research Short answer: content is good

12 Criterion Evaluation Difficult… Imprecise…
The research is civilian The measurement is poor Imprecise… Mapping a competency model to common constructs Short answer: Middling support

13 This Report: Evaluate the quality of instruments that can assess Framework competencies
Using “off the shelf” instruments Using non-cultural instruments Personality, cognitive, etc.

14 Overview of the Procedure
Deconstruct the competency model to elemental form Map elements to commonly studied constructs Find “all” plausibly useful instruments Evaluate the quality of the instruments Map instruments to Framework elements First: All instruments Second: Only the good ones Tally it up

15 1. Deconstruct the Framework
Element Aka “component” in competency modeling C5: Interpersonal Skills “[C5.1] Develops and maintains rapport. Builds relationships in support of mission performance. // [C5.2] Manage and resolve conflict in support of mission objectives.”

16 2. Map the elements to constructs
What existing, researched construct does this competency seem to be related to?

17 Framework Constructs Measures
Competency 1 Competency element 1.1 Personality Construct 1 Personality Construct 2 Instrument/Subscale 1 Instrument/Subscale 2 Competency element 1.2 Competency 2 Competency element 2.1 Attitude Construct 1 Attitude Construct 2 Instrument/Subscale 1 Instrument/Subscale 2 Competency element 2.2 Enabler 1 Cognition Construct 1 Cognition Construct 2 Instrument/Subscale 1 Instrument/Subscale 2 Enabler element 1.2 Enabler element 1.1 Enabler 2 Abilities Construct 1 Abilities Construct 2 Enabler element 2.1 Instrument/Subscale 1 Instrument/Subscale 2 Enabler element 2.2

18 Competency 3: Cultural Perspective Taking
MAKSS-Awareness ICC-Awareness SEE-Empathic perspective taking SEE-Empathic Awareness BEVI-Sociocultural Closure (Perspective taking) Competency element 3.1: Demonstrates an awareness of one’s own worldview… and how one’s own group is viewed… Knowledge of attributed stereotypes Self-insight Perspective taking Competency element 3.3: Takes the cultural context into consideration when interpreting situational cues. Metacognition Situational Awareness CQS-Metacognition

19 3. Find ALL Instruments Hunting and gathering in a vast literature
Two kinds: Single-construct Batteries Three business models (why this matters) Open Restricted Proprietary

20 Many Instruments Don’t trust the published lists
We unfairly divided scales into “primary” and “secondary”

21 Primary Instruments Name Subscales CCAI
Cross-Cultural Adaptability Inventory Flexibility/Openness (FO); Emotional Resilience (ER); Perceptual Acuity (PAC); Personal Autonomy (PA) CQS Cultural Intelligence Scale Metacognition; Cognition; Motivation; Behavior ICAPS Intercultural Adjustment Potential Scale Emotion Regulation (ER); Openness (OP); Flexibility (FL); Creativity (CT) IDI Intercultural Development Inventory Denial/Defense (DD); Reversal [R]; Minimization (M); Acceptance/Adaptation (AA); Cultural Disengagement (CD) MPQ Multicultural Personality Questionnaire Cultural Empathy; Openmindedness; Social Initiative; Emotional Stability; Flexibility ICC Intercultural Communicative Competence Complex structure: 3 domains; 5 dimensions; 4 developmental levels INCA Intercultural Competence Assessment 6 subscales

22 Name Subscales CCAI Cross-Cultural Adaptability Inventory
Flexibility/Openness (FO); Emotional Resilience (ER); Perceptual Acuity (PAC); Personal Autonomy (PA) IES Intercultural Effectiveness Scale Continuous Learning; Interpersonal Engagement; Hardiness ISS Intercultural Sensitivity Scale Interaction Engagement; Respect for Cultural Differences; Interaction Confidence; Interaction Enjoyment; Interaction Attentiveness SEE Scale of Ethnocultural Empathy Empathic Feeling and Expression (EFE); Empathic Perspective Taking (EP); Acceptance of Cultural Differences (AC); Empathic Awareness (EA)

23 Secondary Instruments
ADS Adjustment Difficulties Subscale AIC Beliefs, Events, and Values Inventory ASSIS Acculturative Stress Scale for International Students BASIC Behavioral Assessment Scale for Intercultural Communication Effectiveness BEVI CCSI Cross-Cultural Social Intelligence CGAIC Culture-Generic Approach to Intercultural Competence CWQ The Culture in the Workplace Questionnaire EMMIC European Multidimensional Models of Intercultural Competence GAP Test Global Awareness Profile ICSI Intercultural Sensitivity Inventory Instrument IRC Intercultural Readiness Checklist IRI Interpersonal Reactivity Index ISAS The Inventory of Student Adjustment Strain MAKSS Multicultural Competence Scale MASQUE Munroe Multicultural Attitude Scale Questionnaire MCI MCKAS PCAT Peterson Cultural Awareness Test PCSI Peterson Cultural Style Indicator SCS Social Connectedness Scale WDS Workplace Diversity Survey

24 Validation of the Instruments
Primary Face Construct Criterion Secondary

25 Face Validity Good: Most instruments
Bad: ICAPS (Intercultural Adjustment Potential Scale) Moderate: CQS (Cultural Intelligence Scale)

26 Construct Validity Convergent validity Divergent validity Structure
Similar to what it should be similar to? Divergent validity Different than what it should be different than? Structure If subscales: support for dimensionality?

27 Construct Validity Findings
(Primary scales only) Missing information Diverse outcomes, moderate validity Mushy Method variance not accounted for

28 Construct Validity Findings
Good: MPQ (Multicultural Personality Questionnaire) Mixed: most instruments Bad: ICAPS, ISS (Intercultural Sensitivity Scale) Unknown: GCI, IES, INCA, ICC (mainly proprietary)

29 Criterion Validity Does the scale predict something real? Known-groups
Adjustment: psychological or sociocultural Performance Response to training in experiments Known-groups Do groups differ on the scale the way they should?

30 Criterion Validity Findings
Good: MPQ (Multicultural Personality Questionnaire) Moderate: CCAI, CQS, SEE Mixed: ICAPS Poor: ISS Unknown: GCI, IES, INCA

31 Conclusion: Validity Few scales with overall high validity
Many highly desirable scales have little support

32 5. Map (good) instruments to Framework elements
So: can the Framework be assessed? Strategy: Estimate extent to which each element is assessed using high and moderate validity instruments

33 Framework Constructs Measures
Competency 1 Competency element 1.1 Personality Construct 1 Personality Construct 2 Instrument/Subscale 1 Instrument/Subscale 2 Competency element 1.2 Competency 2 Competency element 2.1 Attitude Construct 1 Attitude Construct 2 Instrument/Subscale 1 Instrument/Subscale 2 Competency element 2.2 Enabler 1 Cognition Construct 1 Cognition Construct 2 Instrument/Subscale 1 Instrument/Subscale 2 Enabler element 1.2 Enabler element 1.1 Enabler 2 Abilities Construct 1 Abilities Construct 2 Enabler element 2.1 Instrument/Subscale 1 Instrument/Subscale 2 Enabler element 2.2

34 Competency 3: Cultural Perspective Taking
MAKSS-Awareness ICC-Awareness SEE-Empathic perspective taking SEE-Empathic Awareness BEVI-Sociocultural Closure (Perspective taking) Competency element 3.1: Demonstrates an awareness of one’s own worldview… and how one’s own group is viewed… Knowledge of attributed stereotypes Self-insight Perspective taking Competency element 3.3: Takes the cultural context into consideration when interpreting situational cues. Metacognition Situational Awareness CQS-Metacognition

35 Score Card: Competencies
Competency Name (# elements) Rating C1 Knowledge (3) * C3 Perspective-Taking (3) *** C4 Communication (2) ** C5 Interpersonal Skills (2) C6 Cultural Adaptability (2) ****

36 Score Card: Enablers Enabler Name Rating E1.1 Ambiguity tolerance
(****) E1.2 Need for closure E1.3 Suspending judgment * E1.4 Inclusiveness ** E2.1 Stress resilience E2.2 Emotion regulation **** E3.1 Self-confidence E3.2 Self-identity E3.3 Optimism (**)

37 More Enablers Enabler Name (# elements) Rating E4.1
Learning through observation (3) E4.2 Inquisitiveness (*) E5.1 Social flexibility (3) **** E5.2 Willingness to engage ***

38 So How Do We Measure a Competency?
Plan A: off the shelf self-report measures Mixed value Various problems with self-report measures Faking, response biases, cognitive/declarative knowledge, etc. Plan B: behavior-based assessment centers Designed for each competency Difficult, expensive, slow

39 Another Way… 1. Rewrite the Framework as a blended competency/causal model Competency model: Just the right competencies for the job Causal model: Related competencies and enablers in models Add moderators; mediation

40 2. Assess the model, not just the competencies
For each competency, assess several variables in the model “Triangulate” on the individual’s competency Relieves the pressure on assessing the competency itself

41 Blended Model C1.1a: Culture-General Knowledge Openness
E4.1.2: Learning Through Observation E4.2: Culture Inquisitiveness Multicultural Attitudes Need for Cognition C: Competencies Requiring Knowledge Situational & External Antecedents

42 fin


Download ppt "Institute for Cross-Cultural Management"

Similar presentations


Ads by Google