Presentation is loading. Please wait.

Presentation is loading. Please wait.

Asking Survey Questions in Many Tongues Paper presented at the ESS Launch Conference Brussels, November 25-26, 2003 Janet A Harkness, ZUMA Germany.

Similar presentations

Presentation on theme: "Asking Survey Questions in Many Tongues Paper presented at the ESS Launch Conference Brussels, November 25-26, 2003 Janet A Harkness, ZUMA Germany."— Presentation transcript:

1 Asking Survey Questions in Many Tongues Paper presented at the ESS Launch Conference Brussels, November 25-26, 2003 Janet A Harkness, ZUMA Germany

2 Overview English as survey lingua franca The ESS translation context ESS requirements and procedures Common survey practice and where the problems lie The ESS and lessons learned

3 English as survey lingua franca We can all speak English Source questionnaires often in English Texts anchored to language Language-driven challenges (two way)

4 The ESS Context: Facts and Figures Round One fielded in 22 countries Fielded in languages, depending on how you count All languages used as first by 5% and more 5 countries fielded in more than one language (12 languages) Most often shared languages (to date): French Italian German. 9 countries shared one or more languages

5 The WP Context: Resources & Researchers ESF EC and Partners CCT Scientific Advisory Board Participating countries Translation Workpackage Translation Expert Panel Dr. Hans Hönig, Dr. Paul Kussmaul, Beth-Ellen Pennell, MA, Dr. Alisú Schoua-Glusberg

6 Key Workpackage Component TRAPD A framework for ESS translation and assessment to deal with the complex requirements

7 Translation framework TRAPD Translation, Review, Adjudication, Pre-testing and Documentation 5 interrelated processes with protocols involved in producing a final translated version TRAPD combines translation with assessment

8 Key Activities for Round One Develop framework for ESS translation and assessment: TRAPD Overview report on TRAPD Guidelines for NCs on translation and assessment procedures, team members roles. Strategies for languages shared across countries Documentation framework and guidelines

9 Additional practical measures Basic guidelines on organising translation and assessment, selecting & training translators Documentation template for problems, divergences, sharing harmonisation Source questionnaire annotated to help translators Consultation hotline

10 Ongoing work--Round One into Round Two Probe interviews with (external) translators on the source questionnaire Analysis of (external) translators appraisals of selected questions from ESS translations Comparisons of selected translations for shared languages

11 In Sum Funds, Guidelines, Specifications, Requirements, Illustrations and Examples Templates, Annotations, Hot-line, Presentations and Reports Ongoing: cognitive-based work on the source questionnaires and translations

12 Isnt this an unusual amount of effort? Arent questionnaires easy to translate?

13 Questionnaires at first glance Easy questions, simple wording Response categories often one or two words Simple and repeated instructions Repetition of lexical and structural elements throughout Similar or identical questions in other surveys And the work team often knows English

14 Questionnaires at second glance Questionnaires are complex texts They look easier than they are Good questions not easy to construct Measurement-in-text poses special problems Cross-disciplinary requirements (e.g., measurement, translation) Interdisciplinary expertise needed A key problem: resulting challenges under- estimated

15 A Complex Text Type Text-with-blanks genre Often largely generic Texts within the text Low Context- jump from one topic to another Cross-national aims encourage general formulations Computer applications can technically complex source formats

16 One example of not-so-easy features Measurement properties lead to surveyspeak and scalespeak Surveyspeak and Scalespeak are textual traces of measurement (Harkness 1996)

17 Surveyspeak Q:All things considered, how satisfied are you with your life as a whole nowadays ? (ESS, B13) Q:Hows life?

18 Q:When did you last visit a dentist? Would you say that you are //very certain, somewhat certain, neither certain nor uncertain, somewhat uncertain or very uncertain// about the date you just gave? A: Conversational Follow-up: Uh-Huh Really ? Are you sure?

19 Scalespeak -- response categories How important or unimportant is ---- Extremely Important Somewhat important Neither important/or unimportant Somewhat unimportant Extremely unimportant

20 Scalespeak Extremely unimportant(ESS E13) Somewhat important Somewhat unimportant Somewhat difficult Somewhat ugly

21 Response scales are difficult to translate Differences across languages/ cultures in quantification structures & potential negation saliency of concept (satisfaction, justness) disclosure norms response styles numeracy and related notions scaling norms (1-6; 1-10?)

22 Known Measurement Effects vs. Allow/ Not allow Allow/ Forbid

23 So what about differences such as: vs. Agree/ Not agree Agree/ Disagree Agree/ Reject or

24 Translations may not be able to match English scale properties German: Agree / Not agree/ Reject English : Agree/ Not agree / Disagree

25 Despite these complexities views on translation translators assessment and testing are often pre-theoretical and procedures followed not grounded in research

26 Not by chance that the ESS work package focused on Survey translation process procedures Specifications, briefing, & training materials for translators and for those hiring translators Survey translation evaluation procedures Providing access to know-how to implement translation components

27 These are what the survey community in general still lacks Thus the ESS effort and activities meet real needs It is a necessary amount of effort

28 In addition No design costs, only translation and assessment Core questionnaire designed for replication Even ESS translations (in a deluxe version) are a very small part of total national survey costs.

29 Lessons learned from: Contact with national teams Documentation from national teams Discussions within the Expert Panel and CCT Cognitive interviews with external translators Related work in other projects (eg SHARE, EUYOUPART, ISSP)

30 Some lessons learned 1.Procedures and guidelines crucial Team work Iterative translation and review Sharing and harmonisation 2.More hands-on input would help more On the job training Training opportunities and materials

31 Lessons learned continued 3. Pretesting –TRApD –more needed on source questionnaires 4.Version documentation is essential – but the Ugly Duckling of the work package. Need to help it have its proper place in a user-friendly form 5.Version harmonization raises scheduling issues. No harmonization discussion raises comparability issues 6.More intensive cross-cultural input needed Questionnaire content Questionnaire formulations [Raises planning, timing and procedural questions]

32 Thank you very much for your time and attention

Download ppt "Asking Survey Questions in Many Tongues Paper presented at the ESS Launch Conference Brussels, November 25-26, 2003 Janet A Harkness, ZUMA Germany."

Similar presentations

Ads by Google