Presentation is loading. Please wait.

Presentation is loading. Please wait.

BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year Regent Teacher: Altamiro Costa Pereira Supervisor Teachers:

Similar presentations


Presentation on theme: "BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year Regent Teacher: Altamiro Costa Pereira Supervisor Teachers:"— Presentation transcript:

1 BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year Regent Teacher: Altamiro Costa Pereira Supervisor Teachers: Cristina da Costa Santos and Luís Pinto FICTION OR REALITY ?

2 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

3 The importance of questionnaires Peoples Opinion Politics Religion Sports Science Demography Medicine … Collected through Have applications in… Questionnaires (1) Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281-91

4 Sistematic mistakes or Biases: The way each question of a questionnaire is written may or may not influence the answer of the inquired person Errors related to not measuring the situation that is intended to be measured. Biased Questionnaire (2) Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34:

5 (5) Question Design Questionnaire Design Administration of Questionnaire Insensitive measure Forced choice Sensitive question Vague word Overlapping interval Ambiguous question Juxtaposed scale Horizontal response format End aversion Primacy and recency (3) Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health (8): (4) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:

6 (5) Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13. A catalog of biases in questionnaires

7 wasnt found … it wasnt found any article which proves the influence of those biases in questionnaires. There are several articles identifying possible biases. BUT… (6) Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; (7) Foddy W. Constructing questions for interviews and questionnaires: theory and practive in social research. Cambridge (UK): Cambridge University Press; (8) Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; p (9) Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63.

8 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

9 Find out if the types of biases such as: Question Design; Questionnaire Design; Administration of Questionnaire… …Influence the answers given in a questionnaire. Aim Could the way each question is made affect, in a significant way, the answers given by the inquired person?

10 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

11 Participants and Methods Utilização dos telemóveis pelos estudantes do primeiro ano do Mestrado Integrado em Medicina (MIM) da Faculdade de Medicina da Universidade do Porto (FMUP) All the students of the first year of MIM from FMUP Question Design Questionnaire DesignAdministration of Questionnaires Other questions Define the theme of the questionnaire Define the target population Types of biased questions Elaboration of two versions of a questionnaire Random distribution of questionnaires by per of classes Collect, organize and analyse A1 A2

12 Participants and Methods Collect, organize and analyse

13 Participants and Methods Website

14 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

15 Questionnaires (9) Questionnaire A2 Questionnaire A1

16 Questionnaire A2 Questionnaire A1 Questionnaires [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

17 Questionnaire A1 Questionnaire A2 Questionnaires [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

18 Questionnaire A2 Questionnaire A1 Questionnaires (10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:

19 Questionnaire A1 Questionnaire A2 Questionnaires 5. [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

20 Questionnaire A1 Questionnaire A2 Questionnaires [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

21 Questionnaire A1 Questionnaire A2 Questionnaires [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

22 Questionnaire A2 Questionnaire A1 Questionnaires [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

23 Questionnaire A2 Questionnaire A1 Questionnaires [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

24 Questionnaire A1 Questionnaires Questionnaire A2 [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

25 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

26 Expected Results Answers Questions of Questionnaires A1 Questions of Questionnaires A2

27 Expected Results The way each question is made affects, in a significant way, the answers given by the inquired person.

28 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

29 Results and Discussion Questionnaire A1Questionnaire A2 Total* of inquired students (N) 107 of of 120 Percentage of inquired students91%82% * The total number of students who should answer the questionnaires AI and A2 was 118 and 120, respectively. However, only 107 and 98 answered because the others werent present when the questionnaires were delivered.

30 Questionnaire A2 Questionnaire A1 Questionnaire A1 (N=107 ) Questionnaire A2 (N=97) p Value Quão importante é o telemóvel para ti? Median (min-max) 7,4 (0-10)8,0 (3-10)< 0,001 Tests: Mann-Whitney Results and Discussion [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

31 Questionnaire A2 Questionnaire A1 RESPOSTAS VÁLIDAS: RESPOSTAS VÁLIDAS: Questionnaire A1 (N=85) QuestionnaireA2 (N=98) p Value Conseguia fazer a vida normal sem telemóvel. * N (%) 29 (34%)32 (33%)0,834 * Neutral questions (Não sei) werent considered. Tests: Chi-square Results and Discussion 21% 21% não sei CI95%=[13%,29%] 0% Mmissing answers [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

32 Questionnaire A1 Questionnaire A2 Questionnaire A1 (N=107) QuestionnaireA2 (N=98) p Value Considera-se viciado no uso do telemóvel N (%) 21 (20%)19 (19%)0,966 Tests: Chi-square Results and Discussion [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

33 Questionnaire A2 Questionnaire A1 (10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77: Questionnaire A1 (N=106) Questionnaire A2 (N=98) p Value Quão importante/satisfeito está com as aplicações do telemóvel. Mediana (1º quartil – 3º quartil) Aplicações ImportânciaSatisfaçãoImportânciaSatisfaçãoImportância Satisfação Funcionalidades 3 (2-4)4 (4-5)3 (3-5)4 (4-5)0,0110,655 Duração da bateria 4 (4-5)3 (3-4)5(4-5)3 (2-4)0,0060,214 Tarifário 5 (4-5)4 (3-5)5(4-5)4 (3-5)0,1770,778 Design 3 (2-4)4 (3-5)3(2-4)4 (4-5)0,1040,817 Tests: Mann-Whitney Results and Discussion

34 Questionnaire A1 Questionnaire A2 Concordo/Discordo not considered Concordo/Discordo not considered Questionnaire A1 (N=45) Questionnaire A2 (N=97) p Value Concorda com o desbloqueio ilegal dos telemóveis. * N (%) Concordo 32 (71%)66 (68%)0,713 Tests: Chi-Square Results and Discussion 58% 58% Concordo/Discordo CI95%=[49%;68%] 1 Missing answer [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

35 Questionnaire A1 Questionnaire A2 Questionnaire A1 (N=107) QuestionnaireA2 (N=98) p Value Em média, frequência de envio de mensagens N (%) Nunca (0 mensagens diárias) 1 (1%)0 (0%) Raramente (1-2 mensagens diárias) 5 (5%)2 (2%) Ocasionalmente (3-10 mensagens diárias) 27 (25%)20 (20%) Frequentemente (11-30 mensagens diárias) 39 (36%)61 (62%) Constantemente (mais de 30 mensagens diárias) 35 (33%)15 (15%) Results and Discussion [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

36 Questionnaire A1 Questionnaire A2 Questionnaire A1 (N=107) Questionnaire A2 (N=98) p Value Mensagens enviadas numa aula teórica Zero 6 (6%)1 (1%) Uma a três 24 (23%)25 (26%) Quatro a dez 40 (38%)37 (38%) Mais de dez 36 (34%)35 (36%) Results and Discussion [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

37 Questionnaire A2 Questionnaire A1 Questionnaire A1 (N=107) Questionnaire A2 (N=97) p Value Uso de telemóveis prejudica as relações sociais dos jovens N (%) Sim 29 (27%)21 (22%)0,366 Tests: Chi-Square Results and Discussion [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

38 Questionnaire A2 Questionnaire A1 Questionnaire A1 (N=107) Questionnaire A2 (N=98) p Value Quando utilizas o telemóvel, o que fazes mais frequentemente? N (%) Música 4 (5%)6 (7%) Jogos 0 (0%)1 (1%) Chamadas 15 (18%)14 (16%) Internet 2 (2%)1 (1%) SMS/MMS 59 (72%)67 (75%) Fotos 0 (0%) Messenger 0 (0%) Outros 2 (2%)0 (0%) Results and Discussion 23% 23% of missing answers (IC95%= 15,0;31,0) 9% 9% of missing answers (IC95%= 3,3;14,7) [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

39 Questionnaire A1 Questionnaire A2 Questionnaire A1 (N=107) Questionnaire A2 (N=98) p Value Quais são os três tipos de pessoas com quem mais falas ao telemóvel? N (%) Pai 36 (34%)21 (23%)0,051 Mãe 78 (74%)59 (64%)0,054 Amigos 83 (79%)71 (77%)0,397 Namorado(a) 43 (41%)44 (48%)0,495 Professores 1 (1%)2 (2%)0,607 Avós 0 (0%) - Irmão/Irmã 14 (13%)18 (20%)0,298 Primos 4 (4%)2 (2%)0,685 Colegas de Faculdade 56 (53%)59 (64%)0,257 Pessoas que não conheces pessoalmente 0 (0%) - Tests: Chi-Square Results and Discussion [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13.

40 Its very hard to write a biased questionnaire with only one bias per question. Results and Discussion

41 We want to understand if there are significant differences between the questionnaire considered biased and the one considered neutral. The same inquired person both The same inquired person should answer both questionnaires In what way is that a limitation? Results and Discussion

42 Minimize selection bias randomly The questionnaires were randomly distributed to the inquired people Results and Discussion

43 The homogeneity of the population: Same age Same course Same college

44 Results and Discussion Lack of sample size in some options Some statistical tests

45 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References

46 Conclusion Could the way each question is made affect, in a significant way, the answers given by the inquired person? Yes

47 Questionnaire A1 Questionnaire A1 Conclusion Questionnaire A1 Questionnaire Design Juxtaposed Scale Visual Analogical Scale

48 Conclusion Some recommendations: visual analogical scales Use visual analogical scales ; neutral option Presence of neutral option ; Mixing concepts Mixing concepts ; order Attention to the order of the options.

49 Summary Introduction Importance of questionnaires Biases Types of biases Importance of our work Aim Participants and Methods Questionnaires Expected Results Results Discussion Conclusion References

50 [1] Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3): [2] Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34: [3] Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health (8): [4] Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77: [5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis Jan;2(1):A13. [6] Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; [7] Foddy W. Constructing questions for interviews and questionnaires: theory and practice in social research. Cambridge (UK): Cambridge University Press; [8] Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; p [9] Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63. [10] Salant P, Dillman DA. How to conduct your own survey. New York: John Wiley & Sons, Inc; 1994.

51 Class 8 Ana Raquel Roca Andreia Freitas Bárbara Vaz Bárbara Neto Joana Costa João Alves Jorge Nogueiro Lucas Lopes Margarida Capitão Maria Durães Mariana Silva Raquel Cardoso


Download ppt "BIASES IN QUESTIONNAIRES Introduction to Medicine II School Year 2010/2011 Class 8, First Year Regent Teacher: Altamiro Costa Pereira Supervisor Teachers:"

Similar presentations


Ads by Google