Presentation on theme: "Quantitative Resilience Research across Cultures and Contexts"— Presentation transcript:
1 Quantitative Resilience Research across Cultures and Contexts Fons J. R. van de Vijver
2 Outline 4. Acculturation 5. Test adaptations 1. General introduction Tertium comparationisApproaches: Absolutism/relativism/universalismIdentity of meaning2. Common problems of cross-cultural studies (and their solutions)3. Establishing similarity of meaning:3a. Bias and equivalence: Taxonomies3b. Examples4. AcculturationConcepts and Models / Assessment5. Test adaptationsConcepts / Example
3 General Introduction Conceptual core of cross-cultural studies Aim is to compare constructs or scoresIs resilience the same across the globe?Is Country A more/less resilient than Country B?Comparison always implies some shared quality (tertium comparationis): If a comparison visualizes an action, state, quality, object, or a person by means of a parallel which is drawn to a different entity, the two things which are being compared do not necessarily have to be identical. However, they must possess at least one quality in common. This common quality has traditionally been referred to as tertium comparationis (Source:
4 Views on the Relation between Resilience and Culture 1. Absolutism (“etic”)Resilience refers to a universal set of characteristics that individuals use to cope with and thrive despite adversity2. RelativismResilience refers to a concept (dealing with coping and thriving) that is universally applicable; however, its manifestations may differ across culturesExample: Zimmerman & Brenner (2010, referring to Ungar, 2006)The conceptual foundation of resiliency theory can be applicable across cultures; the extent to which resources and assets are applied by youth in their experiences of adversity, however, may not be consistent across all contexts.3. Relativism (“emic”)Resilience refers to basic concept of coping and thriving; however, link between resilience and cultural context is so close that cross-cultural comparisons of resilience are futile and superficial
5 Choice between models is often made on an ideological basis However, more productive to see absolutism and relativism as extremes along a continuumEmpirical studies possible of adequacy of these viewpointsCross-cultural evidence is vital for determining which viewpoint holds for a particular measure/construct
6 Part 2 What are common problems in comparative studies? Central problem:Identity of meaning6
7 Common methodological problems of cross-cultural research and their solutions
8 Problem 1Cross cultural differences in scores cannot be interpreted due to rival hypothesesParticularly salient in two-culture studies that do not consider contextual factorsSolution:Anticipate on rival hypotheses by including more cultures or measuring contextual factors
9 Problem 2Cross-cultural similarities and differences are visually (and not statistically) testedA common example is the absence of a test of similarities of internal consistency coefficientsSolutionExplicit tests of cross-cultural similarities and differences; e.g., simple test of similarity of independent reliabilities available
11 Problem 3 Samples show confounding differences Solution: Particularly salient in convenience samplingSolution:Adaptation of study design and assessment of confounding differences
12 Problem 4Means of different cultural groups are compared without assessing the equivalenceParticularly salient when studying new instruments or working with cultures in which instrument has not been usedSolution:Assessment of structural and metric equivalence; assessment of structural equivalence/differential item functioning should be a routine part of analysis, similar to routine assessment of internal consistency
13 Problem 5Cultural characteristics are attributed to all individuals of that culture (ecological fallacy)Particularly common in studies of individualism—collectivismSolution:Awareness of distinction between individual-and culture-level characteristicsAssessment of relevant characteristics, such as individualism—collectivism, at individual level
14 Problem 6 No check on quality of translation/ adaptation Solution: Check is often not reported or procedure is poorly specified (e.g., translation back translation has been used, but results of procedure are not reported)Solution:Awareness that translation back translation is not always the best possible method; other approaches, such as committee approach, may be more suitableMore detail in reports about translation/adaptation procedure
15 Problem 7 Lack of rationale for selecting cultures Solution: Convenience sampling of cultures is by far the most common procedure in cross-cultural psychology; most common comparison is between Japan and the USSolution:Explain why the culture was chosen
16 Problem 8 There is a verification bias in studies of common paradigms Particularly salient in studies of individualism –collectivismSolution:More critical appreciation of the boundaries of the construct, more focus on falsification
17 Problem 9There is a focus on the statistical significance of cross-cultural differencesIn the first and two related problems:Implicit goal of cross-cultural psychology is not the establishment of cross-cultural differencesFocus on significance detracts attention from effect sizesSolution:Balanced treatment of similarities and differences; differences easier to interpret against a backdrop of similaritiesMore effect sizes should be reported, such as Cohen’s d and (partial) eta squares.
18 Problem 10Results are generalized to large populations, often complete populations of countries, although no probability sampling has been employed to recruit participantsParticularly salient in convenience sampling of participants (often student samples)SolutionMore attention in reports for sampling frame and for consequences on external validity
19 Part 3aBias and equivalence:Definitions of conceptsA framework
20 (a) Bias and Equivalence Does the test measure the same attributes for all cultural groups?Can scores be compared across ethnic groups?
21 Bias: Taxonomy What is internal bias? General: dissimilarity of psychological meaning across cultural groupsPractical: when cross-cultural differences do not involve target construct measured by the testTheoretical: a cross-cultural comparison is biased when observed cross-cultural differences (in structure or level) cannot be fully interpreted in terms of the domain of interest
23 Construct Bias Partial nonoverlap of behaviors defining construct González Castro & Murray (2010): Criteria for resilience are based on studies with U.S. youth and adults, and one important cross-cultural issue involves how these criteria, as Westernized aspects of resilience, may or may not relate to resilience that is manifest in underdeveloped and/or non-Western countries.Examples:Ho (1996): filial piety: Chinese concept is broader than Western conceptSocioeconomic status: upbringing, education, income and professionGender bias in a study of adolescents in the Netherlands about body perception: sexual attractiveness was related to physical attractiveness for women and to physical strength for men8
24 Definition of happiness in individualistic and collectivistic countries? Example: Uchida, Norasakkunkit and Kitayama (2004):
25 Types and Sources of Method Bias Method bias tends to have a global influence on cross-cultural score differences (e.g., increment due to social desirability)
26 Item Bias (also known as differential item functioning, DIF) Informal descriptionDifferences in psychological meaning of stimuli, due to anomalies at item levelMore formal definition:An item of a scale (e.g., measuring anxiety) is said to be biased if persons with the same trait anxiety, but coming from different cultures, are not equally likely to endorse the item.
29 Analysis of Variance and Item Bias Item behavior examined per itemWe do not test for cultural differences, but we test whether scores are identical for persons from different groups with an equal proficiencyNote: regression approach quite similar (illustrated later)
30 Taxonomy of Equivalence Refers to level of comparabilityIs related to bias:Highest level of equivalence obtained for bias-free measurementExamples:Underrepresentation of own region will lead to an underestimation of geographical knowledgeKnowledge of the word “bacon” may well point to level of acculturation (adaptation)Will strongly reflect legal system or subgroup normsInfluence of social desirability4
31 Types of Equivalence Three types: 1. “Structural” or “functional equivalence”2. “Metric equivalence” or “measurement unit equivalence”3. “Scalar equivalence” or “full score equivalence”9
32 (a) “Structural” or “Functional Equivalence” Measurement of the same traitsVarious statistical tools available, e.g.,exploratory factor analysis (with target rotation)confirmatory factor analysisnomological networks (particularly relevant when items/questions are not identical across cultures)Qualitative equivalence can be firmly establishedInterest is here NOT in a comparison of scores across cultures, but in the question of underlying structure: Is it reasonable to assume that the same construct is measured in each cultural group?Differences in scores of various cultural groups may be due factors like stimulus familiarity and social desirability (assuming that these affect all items in more or less the same way)10
33 (b) “Metric Equivalence”, “Measurement Unit Equivalence” Difference in offset of scales of cultural groups, equal measurement unitsIndividual differences have a different meaning within and across cultures:no problems with offset in intra-cultural comparison, offset has to be added in cross-cultural comparisonStatistical tool: structural equation modeling (confirmatory factor analysis)Example:Measurement of temperature in degrees of Celsius and Kelvin (consistent difference of 273 degrees)In psychology such an offset can be created by factors like social desirability or stimulus familiarity13
34 (c) “Scalar Equivalence” or “Full Score Equivalence” Complete comparability of scores, both within and across cultures; seamless transfer of scores across culturesFrequently taken as the aim of cross-cultural researchFor physical variables full score equivalence is not hard to achieve (e.g., measurement of height and blood pressure)To demonstrate that a psychological test shows full score equivalence is usually quite complicated14
35 Comparability and Equivalence Levels StructuralUnderlying constructMetricSame plus score metricScalarSame plus origin of scale
36 Part 3b Establishing similarity of meaning How to determine equivalence?How to determine item bias?36
37 Many statistical procedures available for testing structural equivalence Common approach:Apply dimensionality-reduction techniqueCompare underlying dimensions across culturesSimilarity of underlying dimensions is criterion for similarity of meaning
39 Two procedures explained 1. Pairwise comparisonsCompare all cultures in a pairwise manner2. “One to all” comparisonCompare all cultures to a global, pooled solution
40 Characteristics of pairwise comparisons Strong point: much detail, all pairs comparedWeak point: computationally cumbersome, difficult to integrateCharacteristics of pooled comparisonsStrong point: maintains overview, integrationWeak point: can conceal subgroups of countries
41 Example PairwiseData set: WISC-III administered in Canada and Netherlands/Flanders41
46 1. Determining Number of Factors Scree plot suggests the extraction of a single factorLiterature:Debate about 3 or 4 factorsHierarchical model of correlated factorsHere: 4 factors46
47 2. Factor Analyses per group: Oblimin-Rotated Solution 47
48 2. Factor Analyses per group: Oblimin-Rotated Solution 48
49 3. Compare Factors across Groups Rotate one solution to the otherTarget rotations to deal with rotational freedom in factor analysisEvaluation by means of Tucker’s phi (factor congruence coefficient):similarity of factors up to multiplying (positive) constant (correct for differences in eigenvalues across cultures)49
50 3. Compare Factors across Groups Formula (x and y are loadings after target rotation of one to the other):50
52 3. Compare Factors across Groups Values above .90 are usually considered to be adequate and values above .95 to be excellentSuch high values point to similarity of factors structural equivalence52
53 3. Compare Factors across Groups Dedicated software needed to compute Tucker’s phiSPSS routine available53
55 PROPORTIONALITY COEFFICIENT per Factor: .99 .98 .97 .91 55
56 Conclusion Strong evidence for similarity of first two factors Less convincing for third and fourth factor5656
57 Example “One to All” Steps in analysis: 1. Exploratory factor analysis on the total data set;Two procedures (note: correct for mean differences between groups):“quick and dirty”: standardize scores per cultural groups and factor analyze the standardized scoresmore adequate solution: compute the weighted average of the covariance matrices of the cultural groups (weight by sample size)this factor analysis provides the “pooled solution”
58 “One-to-all” procedure 2. Carry out a factor analysis in each cultural group3. Compute agreement of the pooled solution and each of the country solutionsSource: Van de Vijver, F.J.R. & Poortinga, Y.H. (2002). Structural Equivalence in Multilevel Research. Journal of Cross-Cultural Psychology.
59 Example 1990-1991 World Values Survey (Inglehart, 1993, 1997) 47,871 respondents from the following 39 “regions” (number of respondents in parentheses): Austria (1355), Belarus (912), Belgium (2318), Brazil (1672), Bulgaria (877), Canada (1545), Chile (1368), China (960), (the former) Czechoslovakia (1384), Denmark (892), (the former) East Germany (1226), Estonia (864), Finland (416), France (902), Hungary (886), Iceland (659), India (2150), Ireland (976), Italy (1810), Japan (655), Latvia (720), Lithuania (847), Mexico (1193), Moscow (894), Netherlands (935), Nigeria (954), Northern Ireland (283), Norway (1111), Poland (850), Portugal (976), Russia (1642), South Africa (2480), South Korea (1210), Spain (3408), Sweden (901), Turkey (886), United Kingdom (1356), United States (1688), and (the former) West Germany (1710).
61 Pooled solution(Sign of loadings in line with expectation)
62 Stem-and-Leaf Display of Agreement Pooled Loadings and Factor Loadings per Country
63 Correlations of GNP and the Loadings per Region on the Postmaterialism Scale Conclusion: Postmaterialism concept more salient in more affluent countries
64 Metric Equivalence at Scale Level: Structural Equation Modeling64
65 Difference with Exploratory Factor Analyses Starts from covariance matricesUse metric informationMore parameters tested for cross-cultural similarity; examplesFactor loadingsFactor correlations/covariancesError component of latent variablesError component of observed variablesEnables the testing of a hierarchy of models
66 Example of AMOSModel tested: one factor of verbal comprehension factor in two countries (Belgium/Netherlands and Canada)Models tested:Identical factor loadings across countriesFree factor loadingsIdem with a correlated errorFor diagram and output: see AMOS files66
69 Measurement weights: regression weights in the measurement part of the model. In the case of a factor analysis model, these are the "factor loadings".Structural residuals: variances and covariances of residual (error) variables in the structural part of the model.Measurement residuals: variances and covariances of residual (error) variables in the measurement part of the model.69
79 Hundreds of statistical procedures available Assumption: Equal observed scores on global instrument (scale) in different cultures have the same meaningAlmost all techniques start from unidimensional scalesProcedures test whether, given equal total scores, patterns of observed scores are the same across culturesOften applied proceduresANOVA (example follows)Item Response Theory(in education) Mantel-Haenszel (equivalent to testing applicability of Rasch model)
80 How to Determine Item Bias? Analysis of varianceINPUT: a data matrix with interval-level dependent variables (e.g., Likert-scale), one variable indicating culture.80
81 Step 1: Compute Total Score Compute total test score (or mean item score) (so, a unifactorial scale is assumed).COMPUTE sumscore = i_acad_1 + i_cult_1 + i_groo_1 + i_infl_1 + i_inte_1 + i_like_1 + i_look_1 .EXECUTE .81
82 Step 2: Determine Cutoffs (here three groups; percentiles 33 and 67).EXAMINEVARIABLES=sumscore /PLOT BOXPLOT STEMLEAF/COMPARE GROUP /PERCENTILES(33, 67) HAVERAGE/STATISTICS DESCRIPTIVES /CINTERVAL 95/MISSING LISTWISE /NOTOTAL.ORFREQUENCIESVARIABLES=sumscore/NTILES= 3/ORDER= ANALYSIS .82
84 Step 4: Carry out ANOVAs Significant main effect of level: irrelevant UNIANOVAi_acad_1 i_cult_1 i_groo_1 i_infl_1 i_inte_1 i_like_1 i_look_1 BY group level/METHOD = SSTYPE(3)/INTERCEPT = INCLUDE/PRINT = DESCRIPTIVE ETASQ/CRITERIA = ALPHA(.05)/DESIGN = group level group*level .Significant main effect of level: irrelevantSignificant main effect of culture: uniform biasSignificant interaction between culture and level: nonuniform biasNOTE: in large samples effect sizes can be used (eta squared > .06: Cohen’s medium effect size)84
85 Regression DESCRIPTIVES VARIABLES=sumscore cult /STATISTICS=MEAN STDDEV MIN MAX.
86 compute predictor values for these new variables * compute predictor values for these new variables. compute dev_mean=sumscore compute dev_cult=cult EXECUTE . compute interaction = dev_mean*dev_cult.
88 Part 4. AcculturationDefinition: Acculturation refers to changes that take place as a result of continuous first-hand contact between individuals of different cultural origins (Redfield, Linton, & Herskovits, 1936).Psychological acculturation refers to psychological aspects of process
89 Acculturation research traditions: Stress and coping Social learning Social cognition (more recent)
90 Framework of Acculturation: Acculturation Variables Acculturation ConditionsAcculturation OrientationsAcculturation OutcomesCharacteristics of the receiving society (e.g., discrimination, opportunity structures)CulturaladoptionPsychologicalwell-being(psychological distress, mood states, feelings of acceptance, and satisfaction)Characteristics of the society of origin (objective, perceived)Socioculturalcompetence in ethnic culture(interaction with conationals, maintenance of culturally appropriate skills and behaviors)CulturalmaintenanceCharacteristics of the immigrant group (objective, perceived)Personal characteristicsSocioculturalcompetence in mainstream culture(interaction with hosts, acquisition of culturally appropriate skills and behaviors)
91 Features Compare S-O-R model Mediation model with feedback loops Feedback almost never studiedCausality usually inferred (so, some arbitrariness)Implicit schemedistal—proximal—outputTerm adaptation used in literature to refer to adjustment/outputProblem: adaptation can refer to both product and process
92 Resilience-Related Pathways for Immigrants (González Castro & Murray, 2010)
93 Studies of Acculturation Conditions Personality often studiedMPQ, Big FiveUsually: extraversion +, neuroticism –Intelligence not studiedMulticulturalism policies presumably unrelated to acculturation outcomes in Western societiesESS (Schalk-Soekar et al., 2007)ICSEY (Berry et al., 2006)
101 Dimensionality of Cultural Distance Psychological measures of distance (perceived cultural distance) load on a single factorNote: models of cross-cultural distance models tend to be multidimensional (e.g., Hofstede)
102 Acculturation Orientations Notes on terminology:1. Various terms used, e.g.,Strategies, styles, orientations2. Adaptation usually reserved for output/adjustment; here: adoption, adoptingin original formulation: does the immigrant want to establish relationships with new culture?Problem: Narrow conceptualization
103 Cultural adoption Cultural maintenance maintaining characteristics of own (heritage) cultureCultural adoptionadopting characteristics of the culture of the society of settlement
104 Acculturation Models Unidimensional model Bidimensional model Cultural Cultural maintenance adoptionCultural maintenanceCultural adoption
105 Berry’s Bidimensional Model YesSeparationIntegrationCulturalmaintenance?NoMarginalizationAssimilationNoYesCultural adoption?
106 FeaturesCorrelations of dimensions often varyConceptually independentEmpirically often negatively relatedDimensions or orientations more important?Methodologically: dimensions often easier to deal withConceptually: orientations prevailNote that integration refers to biculturalism in psychology and to sociocultural outcomes in sociology (a well integrated immigrant is a person who speaks the mainstream language, has a paid job, etc.)
108 Domain Specificity Conceptually domains independent Empirically not always the caseWill depend on a host of factors, such as cultural distance, perceived pressure to assimilate, …Often slightly negative correlationsExample: we found a clear negative corelation in the evaluations of Dutch and Turkish culture in a group of Turkish-Dutch
110 Assessment of Acculturation: Recurrent Problems Acculturation variables (conditions, orientations, and outcomes) are mixedReliance on ‘Proxy’ measures of acculturation, such as length of stay (poor validity)Reliance on single-index measures (do not fully account for construct)
111 Assessment of Acculturation: Recurrent Problems (cont’d) Measure of only adoption dimension, not of maintenance dimensionAcculturation aspects (e.g., cognition, values, attitudes) are often combined.Sound and meaningful?No psychometric properties reportedOften emphasis on actual behavior and language proficiencyMeasures often assess sociocultural outcomes that are used to predict other outcomes (e.g., school performance)
112 Outcomes Focus on two kinds of outcomes Psychological adjustment (stress & coping)Sociocultural adjustment (social learning)Almost no studies of cultural maintenanceThis lack of balance absent in sociolinguistics where both acquisition of mainstream and loss of ethnic languages is studiedThis lack of balance is also absent in study of acculturation orientations
114 (1) One-Statement Method Example item (1 statement for 1 domain) only Turkish friends more Turkish than Dutch friends I find it important to have as many Turkish as Dutch friends. more Dutch than Turkish friends. only Dutch friends no Dutch and no Turkish friends. Advantages Short(est) questionnaireProblem One dimension?HeritageMainstream
115 (1) One-Statement Method Research findings Domain specificity (public, private components)publicDutchprivateTurkish Recommendation This method is often quite useful in practice, despite conceptual problems Take domains into consideration
116 (2) Two-Statement Method Example (domain friends) I think it is important to have Dutch friends I think it is important to have Turkish friends Advantages The two dimensions are measured independently Items are not complex Questionnaire is still short Disadvantages/questions Are the two dimensions really independent? How to define the four acculturation orientations?
117 How to Define the Four Acculturation Orientations? Sample-dependent coding:Mean or (more common) median splitAdvantage: optimal spread of participants across orientationsDisadvantage: validity can be problematic in groups with a shared preference (often the case for integration)
118 How to Define the Four Acculturation Orientations? (cont’d) Response scale-dependent codingMidpoint split (average scores above or below midpoint of scale)Advantage: face validityDisadvantage: what to do when scale has even number of anchors? Solutions such as random split or allocating these to a single group have an unavoidable arbitrariness
119 (2) Two-Statement Method ResultsPossible method factor, e.g., all maintenance items togetherDomain dependence: public domain (Tu, Du) private Dutch domain private Turkish domainDomain dependence does not always show up as separate factors (usually based on differences in mean scores)
120 Potential problem:Two scores are sometimes converted to four orientations (e.g., distance method), which introduces dependencies in the dataRecommendation This method can be used Take domains into consideration
122 Summary of ResultsResults of the ‘one-statement’ and the ‘two-statement’ measurement methods: domain specificityPublicDutchPrivateTurkish7654321PrivatePublicCulturalmaintenance(Turkish)Cultural adoption (Dutch)
123 (3) Four-Statement Method Example item (4 items for 1 domain) (Int) I find it important to have Dutch friends and I find it also important to have Turkish friends. (Sep) I find it not important to have Dutch friends but I find it important to have Turkish friends. Advantage The four strategies are measured independently Disadvantages (questions) Complex items (see Marginalization) Questionnaire is long (per domain 4 questions) Factors and (independent) dimensions?
124 (3) Four-Statement Method Research findings Bipolar unidimensional structure(-) Integration (+) A S M % of our immigrant Dutch samples prefer integration (one score) Advantages Method is broad Measure integration with more details
125 Summary of Results Measurement Results methods Four-statement Insufficient discrimination: integration vs not-integration One-statement Discrimination between public and private domains Two-statement More detailed information within domainsTwo-statement method often works best.
126 Questions to consider when choosing/designing an instrument 1. The clear formulation of research goals and choice of acculturation variables.What is the role of acculturation in the study? Antecedent, mediating/moderating, or outcome variable
127 2. Which acculturation aspects are dealt with? knowledge, values, attitudes, or behavior
128 3. The choice of research methodology (how to study?) “Soft” or “hard” measuresSelf-reports, observations, …
129 4. The choice of a measurement method (how to assess acculturation?) Orientations: one-, two-, and four-statement methodPerceived or actual environmental conditionsMultilevel issues may be involved when both individual and contextual variables are considered
130 5. The choice of life domains and situations to be dealt with in the items in which domains and situation to assess?
131 6. Choice of item wording.Questionnaires often in second languageUse simple language
132 An Empirical Study Methods (dimensions) of acculturation (1) One-statement method (2) Two-statement method (3) Four-statement method Domain(s) of acculturation Private domains (celebrations, child-rearing) Public domains (language, education, living)
133 Participants 293 Turkish-Dutch adolescents Gender: 144 female and 149 male Generation: 15 first and 278 second generations Age: years, M = (SD = 1.69) Education: Secondary SchoolInstrument and procedure (1) 15 items on 15 domains (7 private and 8 public) (2) 30 items on 15 domains (7 private and 8 public) (3) 36 items on 9 domains (5 private and 4 public)
135 Summary of Results Measurement methods of acculturation One- and two-statement methods: no significant influences of measurement on outcome Four-statement method: the largest influence on outcome Domain specificity Distinct but interrelated positive relationship between private and public domains