Part 5 Test Adaptations 1. Conceptualizations 1. Conceptualizations 2. Example 2. Example

What is the central issue in translations/adaptations? Producing instruments that measure target constructs adequately in target cultures Producing instruments that measure target constructs adequately in target cultures

A Note on Terminology Translation Translation Conventional term, still often usedConventional term, still often used Adaptation Adaptation Has become generic term for modern translation practicesHas become generic term for modern translation practices Based on increased sensitivity for non- linguistic factors in translations, such as cultural norms of address, relevance of thorough knowledge of target cultureBased on increased sensitivity for non- linguistic factors in translations, such as cultural norms of address, relevance of thorough knowledge of target culture

Main Applications of Translations/Adaptations Comparative Studies Comparative Studies Comparison of construct or mean scores across culturesComparison of construct or mean scores across cultures High demands on comparability of scoresHigh demands on comparability of scores Maximizing comparability Maximizing comparability Monocultural studies in target culture Monocultural studies in target culture Main issue is ensuring validity in new contextMain issue is ensuring validity in new context Few demands on comparability scoresFew demands on comparability scores Maximizing local suitability Maximizing local suitability

Translations in Historical Perspective Stage 1: Stage 1: Close translations were standard practiceClose translations were standard practice Techniques were developed (e.g., translation back translation)Techniques were developed (e.g., translation back translation) Important societal developments: Important societal developments: Globalization and migration (multi-ethnic societies)Globalization and migration (multi-ethnic societies) Stage 2: Stage 2: Increasing appreciation that close translations have problems, e.g., Grade 12 = Form 6 = ……?Increasing appreciation that close translations have problems, e.g., Grade 12 = Form 6 = ……? Need for adaptations, localizationsNeed for adaptations, localizations Need for standardization of adaptation proceduresNeed for standardization of adaptation procedures

What is a Good Translation/ Adaptation? Dependent on perspective Dependent on perspective Linguistic perspectiveLinguistic perspective Psychological perspectivePsychological perspective Mapping problem: Mapping problem: Translating/adapting can be seen as finding an optimal mapping of text in two languagesTranslating/adapting can be seen as finding an optimal mapping of text in two languages What is a good mapping? What is a good mapping? A good mapping shows equivalence of the original and translationA good mapping shows equivalence of the original and translation

Example What is the American equivalent of the Dutch item “Hoe heet de koningin van Nederland?” (Suppose that item is part of a test of crystallized intelligence) What is the American equivalent of the Dutch item “Hoe heet de koningin van Nederland?” (Suppose that item is part of a test of crystallized intelligence) Literal/close translation: What is the name of the queen of the Netherlands?” Literal/close translation: What is the name of the queen of the Netherlands?” Problem: Item more difficult for American children than for Dutch childrenProblem: Item more difficult for American children than for Dutch children Adaptation: “What is the name of the president of the USA?” Adaptation: “What is the name of the president of the USA?” Problem: Queen and president are not equally known in their respective countriesProblem: Queen and president are not equally known in their respective countries

What Does “Equivalent” Mean? Eusebius Hieronymus (St. Jerome, famous bible translator from Greek and Hebrew to Latin; ±347—419/420): Eusebius Hieronymus (St. Jerome, famous bible translator from Greek and Hebrew to Latin; ±347—419/420): 2 types of translations: “words” and “meanings” (he favored the latter)2 types of translations: “words” and “meanings” (he favored the latter) Here two types of equivalence relevant: Here two types of equivalence relevant:linguistic mapping/equivalence mapping/equivalencepsychological }

Linguistic Equivalence (Broader than similarity of words) (Broader than similarity of words) Linguistic equivalence refers to similarity of linguistic features of a text. Linguistic equivalence refers to similarity of linguistic features of a text. Examples of relevant linguistic features are: Examples of relevant linguistic features are: Lexical similarityLexical similarity Grammatical accuracyGrammatical accuracy In general: emphasis on formal-textual characteristics (cf. automatic translations)In general: emphasis on formal-textual characteristics (cf. automatic translations)

Psychological Equivalence Psychological equivalence refers to similarity of (psychological) meaning and scores Psychological equivalence refers to similarity of (psychological) meaning and scores Similarity in a broad sense: Similarity in a broad sense: Textual, e.g.,Textual, e.g., Connotation of words, implied context of text Connotation of words, implied context of text Comprehensibility Comprehensibility Metrical:Metrical: Score comparability Score comparability

Relationship between Two Perspectives Three possible relations between linguistic and psychological features, depending on the overlap: Poorly translatable b. partial Essentially non-translatable c. none Translatable a. complete psych. linguistic

Translatability A psychological test/item is Well translatable if linguistic and psychological features yield the same translation Well translatable if linguistic and psychological features yield the same translation Poorly translatable if linguistic and psychological features do not entirely converge (e.g., translation of slang: meaning is translatable, but conciseness is lost) Poorly translatable if linguistic and psychological features do not entirely converge (e.g., translation of slang: meaning is translatable, but conciseness is lost) Non-translatable if there is a complete or nearly complete nonoverlap (e.g., Jabberwocky) Non-translatable if there is a complete or nearly complete nonoverlap (e.g., Jabberwocky)

Jabberwocky (Lewis Carroll, 1871) 'Twas brillig, and the slithy toves Did gyre and gimble in the wabe; All mimsy were the borogoves, And the mome raths outgrabe. "Beware the Jabberwock, my son! The jaws that bite, the claws that catch! Beware the Jubjub bird, and shun The frumious Bandersnatch!" Illustration by John Tenniel

Framework for Translations/Adaptations Need for a theoretical— methodological framework that links all stages of a project Need for a theoretical— methodological framework that links all stages of a project Bias and equivalence as key concepts Bias and equivalence as key concepts

Steps in Designing Cross-Cultural Tests (Hambleton & Patsula, 1999) 1.Ensure t hat construct equivalence exists in the language and cultural groups of interest. 2.Decide whether to adapt an existing test or develop a new test. 3.Select well-qualified translators. 4.Translate and adapt the test. 5.Review the adapted version of the test and make necessary revisions. 6.Conduct a small tryout of the adapted version of the test. 7.Carry out a more ambitious field-test. 8.Choose a statistical design for connecting scores on the source and target language versions of the test. 9.If cross-cultural comparisons are of interest, ensure equivalence of the language versions of the test. 10.Perform validation research, as appropriate. 11.Document the process and prepare a manual for the users of the adapted tests. 12.Train users. 13.Monitor experiences with the adapted test, and make appropriate revisions.

Overview of Common Procedures to Examine Accuracy of Translations/ Adaptations Procedures as opportunities to strengthen the quality of a translation/adaptation project Procedures as opportunities to strengthen the quality of a translation/adaptation project Two taxonomies presented here: Two taxonomies presented here: common: (back) translations vs. committee approachcommon: (back) translations vs. committee approach use of existing/new materialuse of existing/new material

Theoretical and Methodological Background Crucial concept in translations is equivalence: Crucial concept in translations is equivalence: LinguisticLinguistic Mapping of linguistic meaning (word meaning, sentence meaning) Mapping of linguistic meaning (word meaning, sentence meaning) PsychologicalPsychological Mapping of psychological meaning (serves the same psychological function in all languages?) Mapping of psychological meaning (serves the same psychological function in all languages?) A good translation combines these considerations A good translation combines these considerations

Options Adoption (Close “literal” translation) Adoption (Close “literal” translation) Advantage: maintains metric equivalenceAdvantage: maintains metric equivalence Disadvantage: adequacy (too) readily assumed, should be demonstratedDisadvantage: adequacy (too) readily assumed, should be demonstrated Adaptation (changing contents of one or more items so as to increase cultural appropriateness) Adaptation (changing contents of one or more items so as to increase cultural appropriateness) Advantage: more flexible, more tailored to the contextAdvantage: more flexible, more tailored to the context Disadvantage: fewer statistical techniques available to compare scores across culturesDisadvantage: fewer statistical techniques available to compare scores across cultures Assembly (composing a new instrument) Assembly (composing a new instrument) Advantage: very flexibleAdvantage: very flexible Disadvantage: almost no comparability maintainedDisadvantage: almost no comparability maintained

A Sample of Possible Procedures (after Harkness, 2003) Translation back translation Committee approach (forward translations) Mixed approaches (e.g., independent forwards) Translation stage Think alouds, focus groups Feedback from mono- and bilinguals Comprehension and readability checks Pretesting stage (qualitative) Equivalence and bias analyses (DIF, structural equivalence) Pretesting or actual administration (quantitative)

Strength and Weakness of Translations Back Translation Main strengths Well accepted quality check; standard procedures well known in scientific community (incl. researchers, grant institutions and journal boards) No knowledge of target language required Main weaknesses Capitalizes on linguistic, cultural, and item- writing skills of (usually) a single person Can produce stilted language Readability and comprehensibility in target language may be problematic

What is the Best Option? One type is not intrinsically better or worse than another One type is not intrinsically better or worse than another Main question is Main question isNOT What is globally the best choice? What is globally the best choice?BUT What is the best choice in a specific case? What is the best choice in a specific case?

Construct equivalence Similarity of construct in source and target culture Cultural equivalence Norms about interaction (modes of address) “Cultural fact sheet” Linguistic equivalence Translation accuracy: Retention of denotation and connotation Measurement equivalence Retention of psychometric features (response styles) Similarity of factors measured by a test and comparability of scores Four Important Perspectives (Harkness & Van de Vijver, in preparation) :

Adaptation Perspectives Adaptation PerspectivesIndicator Construct Measurement Language Integration Culture

A good translation/adaptation combines equivalence perspectives A good translation/adaptation combines equivalence perspectives What is a good translation/ adaptation? What is a good translation/ adaptation? A translation or adaptation is good when it combines high levels of construct, cultural, linguistic, and measurement equivalence.A translation or adaptation is good when it combines high levels of construct, cultural, linguistic, and measurement equivalence.

Is There a Best Way to Translate an Instrument? Simple items often straightforward to translate Simple items often straightforward to translate Close translations will do well, various kinds of equivalence jointly maximizedClose translations will do well, various kinds of equivalence jointly maximized More complex items often require choices about which equivalence will be maximized: More complex items often require choices about which equivalence will be maximized: Maximizing comparability or cultural appropriateness ?Maximizing comparability or cultural appropriateness ?

Different perspectives on equivalence often, but not always compatible Different perspectives on equivalence often, but not always compatible Example: cross-cultural differences in modes of address Example: cross-cultural differences in modes of address Maximizing linguistic equivalence may challenge cultural appropriateness (e.g., requests may be too direct)Maximizing linguistic equivalence may challenge cultural appropriateness (e.g., requests may be too direct) Maximizing cultural appropriateness may challenge statistical equivalence (e.g., rephrasing may threaten comparability of scores)Maximizing cultural appropriateness may challenge statistical equivalence (e.g., rephrasing may threaten comparability of scores)

Taxonomy of Adaptations Need for adaptation can be Construct-drivenConstruct-driven Culture-driven (communication style)Culture-driven (communication style) Language-drivenLanguage-driven Measurement-driven (familiarity/recognizability)Measurement-driven (familiarity/recognizability)

Most examples come from

Kaufman Assessment Battery for Children (second edition)

Subtests: Atlantis Atlantis Number Recall Number Recall Rover Rover Triangles Triangles Word Order Word Order Pattern Reasoning Pattern Reasoning Story Completion  replaced by adaptation of Story Completion  replaced by adaptation of WISC(/-R/-III) Picture Arrangement

1. Example Construct-Driven Problems with the behaviors or attitudes associated with the construct or with communication norms pertaining to these behaviors or attitudes Problems with the behaviors or attitudes associated with the construct or with communication norms pertaining to these behaviors or attitudes Usage of somatic and psychological symptoms in depression inventories Usage of somatic and psychological symptoms in depression inventories Differential norms in allowance to express psychological symptoms across cultures Differential norms in allowance to express psychological symptoms across cultures

Patel, Abas, Broadhead, Todd, & Reeler (2001) Patel, Abas, Broadhead, Todd, & Reeler (2001) In Zimbabwe, multiple somatic complaints such as headaches and fatigue are the most common presentations of depression. On inquiry, however, most patients freely admit to cognitive and emotional symptoms. Many somatic symptoms, especially those related to the heart and the head, are cultural metaphors for fear or grief. Most depressed individuals attribute their symptoms to “thinking too much” (kufungisisa), to a supernatural cause, and to social stressors. Our data confirm the view that although depression in developing countries often presents with somatic symptoms, most patients do not attribute their symptoms to a somatic illness and cannot be said to have “pure” somatisation. This means that it is vital to understand the culture specific terminology used by patients and to assess mood in those with multiple somatic complaints.In Zimbabwe, multiple somatic complaints such as headaches and fatigue are the most common presentations of depression. On inquiry, however, most patients freely admit to cognitive and emotional symptoms. Many somatic symptoms, especially those related to the heart and the head, are cultural metaphors for fear or grief. Most depressed individuals attribute their symptoms to “thinking too much” (kufungisisa), to a supernatural cause, and to social stressors. Our data confirm the view that although depression in developing countries often presents with somatic symptoms, most patients do not attribute their symptoms to a somatic illness and cannot be said to have “pure” somatisation. This means that it is vital to understand the culture specific terminology used by patients and to assess mood in those with multiple somatic complaints. Consequence Consequence Common western measures of depression will under-diagnose depression in Shona speakers.Common western measures of depression will under-diagnose depression in Shona speakers.

33 2. Example Culture-Driven  Example: ‘Burglar’ (Picture Arrangement; adapted for use in low-SES children in Bangalore, India ) Problems: Problems: 1.Unclear whether the burglar was getting in or getting out; 2.Man not recognized as burglar; 3.Window was not recognized (vertically moving windows are uncommon in India) Malda, Van de Vijver, Srinivasan, Transler (2008): Adapting a Western Cognitive Test for a Non-Western Context: The KABC-II in Bangalore, India

Example: Do you often feel distressed? Example: Do you often feel distressed? Translation to Dutch: Translation to Dutch: “Distressed” does not have an equivalent word in Dutch“Distressed” does not have an equivalent word in Dutch Possible solutionsPossible solutions Composite of different emotions in Dutch; ask for frequency of composite (“how often do you feel X and Y?”). Problem: composite may not be recognizable Composite of different emotions in Dutch; ask for frequency of composite (“how often do you feel X and Y?”). Problem: composite may not be recognizable Choose a single emotion that is as close as possible; problem: change of item content if no close match can be found Choose a single emotion that is as close as possible; problem: change of item content if no close match can be found Describe the emotion in the item (e.g., vignette); problem: may require a similar description in English original Describe the emotion in the item (e.g., vignette); problem: may require a similar description in English original Need to check adequacy of chosen solution in statistical analysisNeed to check adequacy of chosen solution in statistical analysis Combination of judgmental and statistical evidence crucial in instruments that are more difficult to translate/adaptCombination of judgmental and statistical evidence crucial in instruments that are more difficult to translate/adapt 3. Example of Language-Driven Adaptation

Language and test content: Language and test content: Adaptation of words in subtest Atlantis: Adaptation of words in subtest Atlantis: Kannada nonsense words Kannada nonsense words (e.g., English ‘Dablee’  Kannada ‘Ribu’) Important: number of syllables Important: number of syllables Adaptation of digits in subtest Number Recall Adaptation of digits in subtest Number Recall based on number of syllables (1 in English version; first 2 and then 3 in Kannada version) based on number of syllables (1 in English version; first 2 and then 3 in Kannada version)

36 4. Example of Measurement-Driven Adaptation (Unfamiliarity) Kaufman ABC used in Bangalore (Kannada-speaking children) Kaufman ABC used in Bangalore (Kannada-speaking children) Adaptation of words in subtest Word Order based on: Adaptation of words in subtest Word Order based on: Unfamiliarity and ambiguity of objects and words Unfamiliarity and ambiguity of objects and words Number of syllables Number of syllables Original version  Kannada version  Malda, Van de Vijver, Srinivasan, Transler (in review): Adapting a Western Cognitive Test for a Non-Western Context: The KABC-II in Bangalore, India

Original version  Kannada version  Problem: word for star in Kannada is too long, English word “star” is well known but too short (monosyllabic)

Original version  Problems: (1) Key was often called ice cream; (2) English word “key” was often used, which is too short (monosyllabic) Kannada version 

Original version  Problem: original drawing was not easily recognized as house, distinguishing features added Kannada version 

Example: ‘Painting’ Problem: mirror was not recognized

Rover Test content: Test content: Additional instructions in subtest Rover Additional instructions in subtest Rover One additional instruction in subtest Pattern Reasoning One additional instruction in subtest Pattern Reasoning Slight change of subtest composition and item order in subtest Triangles Slight change of subtest composition and item order in subtest Triangles Sample item Original version  Sample item Indian version  Problem: original sample item was too difficult; this item has been added as actual test item

Background Reading (1)(2)