3/22/2011Sources of error: Information bias1 Principles of Epidemiology for Public Health (EPID600) Victor J. Schoenbach, PhD home page Department of Epidemiology Gillings School of Global Public Health University of North Carolina at Chapel Hill

11/5/2001Sources of error: Information bias2 Abort, Retry, Fail “Tips for safer drives: Never turn off a PC or accessories while the computer is on or the disk is active.” — USA Today [PC Magazine, 10/3/1996]

Chapter 1 THE HISTORIAN'S TASK: Insight into the future History, a record of things left behind by past generations, started in Thus we should try to view historical times as the behind of the present. Anders Henriiksson (ed), Non Campus Mentis, NY, Workman Publishing Co., 2003

Non Campus Mentis “History, as we know, is always bias, because human beings have to be studied by other human beings, not by independent observers of another species.” Anders Henriiksson (ed), Non Campus Mentis, NY, Workman Publishing Co., 2003, chapter 1

3/22/2011Sources of error: Information bias5 Information bias Information bias: a systematic distortion or error that arises from the procedures used for classification or measurement of the disease, the exposure, or other relevant variables.

3/22/2011Sources of error: Information bias6 Information bias Classification or measurement Differential or nondifferential bias Direction of bias Misclassification of covariables

3/22/2011Sources of error: Information bias7 Classification or measurement Data for epidemiologic studies consist of classifications (e.g., “hypertensive” vs. “normotensive”) or measurements (e.g., 120 mmHg systolic BP). Possible sources of measurement or classification error include instrumentation, laboratories, records, respondents; data collectors, managers, analysts, and interpreters.

3/22/2011Sources of error: Information bias8 Sources of measurement error Respondent (interview, questionnaire): inability to understand, recall, articulate; unwillingness to disclose social desirability influences Can be influenced by wording of questions and how they are asked.

9 Example of misunderstanding Medico – Não consigo encontrar o motivo das suas dores, meu caro. Só pode ser por causa da bebida. Paciente – Não tem importãncia, doutor. Eu volto outro dia que o senhor estiver sóbrio. De Luciana V. Paiva, Osasco - SP, en Bom Humor Nosso E Dos Leitores”, Almanaque Brasil de Cultura Popular. Maio 2001;3(26) Exemplar de quem viaja TAM.

3/22/2011Sources of error: Information bias10 How not to ask questions “Has anyone ever tried to give you the mistaken idea that sex intercourse is necessary for the health of the young man? (from a survey by the NC state health officer, circa 1926, summarized in Kinsey et al., 1948) Can you guess the right answer?

3/27/2007Sources of error: Information bias11 Respondent cognitive processes Respondent cognitive processes: interpretation, recall, judgment formation, response formatting, editing Qualitative research on response processes, e.g.: “What types of physical activity or exercise did you perform during the past month?” “What did you think we meant when we said ‘physical activity’?” “Which, if any, of the following would you (also) consider to be physical activity?

3/27/2007Sources of error: Information bias12 Cognitive testing - 2 Recalling and retrieving – Retrieval probes: Recall strategy Recall interval Search strategy (proximal, distal, anywhere) Long term recall - link to events to help remember Recall frame of reference--what kinds of things helped you remember?

3/22/2011Sources of error: Information bias13 Surveys and Questionnaires Survey validation Pretesting (wording, item sequence, time) Pilot testing (all steps - procedure, item performance) Translation validation Sources of error: Information bias13

3/27/2007Sources of error: Information bias14 Sources of measurement error Data collector: unclear or ambiguous questions, lack of a neutral demeanor, insufficiently conscientious, inaccurate transcription, fraud

11/5/2001Sources of error: Information bias15 Sources of measurement error Data managers: inaccurate transcription, mis-reading, miscoding, programming errors Data analysts: variable coding and programming errors Data interpreters: inadequate appreciation of the characteristics of the measure or of the relations being studied

11/5/2001Sources of error: Information bias16 Techniques for avoiding data collection errors Precise operational definitions of variables Detailed measurement protocols Repeated measurements on key variables Training, certification, and re-certification Data audits (of interviewers, of data centers) Data cleaning – visual, computer Re-running all analyses prior to publication

11/5/2009Sources of error: Information bias17 Validation and agreement Sensitivity and specificity – used to evaluate classifications When no validation standard, we measure agreement Measures of agreement often correct for “chance”

11/5/2001Sources of error: Information bias18 Information bias – differential or non-differential Important question for any kind of bias – are error processes different for groups being compared If no, “non-differential” If yes, “differential” Has implications for direction of bias In general, non-differential is safer

3/22/2011Sources of error: Information bias19 Direction of bias “Upward” “Downward” “Towards the null” “Away from the null” Null = 0 (for differences) 1 (for ratios)

11/5/2001Sources of error: Information bias20 Direction of bias In simple situation, information bias is towards the null IF: 1. Dichotomous exposure and disease 2. Non-differential misclassification with both sensitivity and specificity each greater than 0.5; AND 3. Errors in one variable are independent of errors in the other

11/5/2001Sources of error: Information bias21 Errors in covariables It is almost always important to control for other variables (e.g., age) Errors in measurement of these variables hamper attempts to control for them Direction of bias is generally unpredictable

3/22/2011Sources of error: Information bias22 Classic studies – 6 degrees of separation? Classic experiment by Yale psychologist Stanley Milgram asked people in Kansas to forward a letter to a target person in Massachusetts If did not know target person, then send it to someone they thought might know him Milgram’s 1967 paper reported that it only took 5 jumps, on average, for letters to arrive

3/29/2005Sources of error: Information bias23 Selection bias in a classic study According to Judith Kleinfeld, psychologist at the University of Alaska, Fairbanks, archives reveal that only 30% of the letters actually reached their destination! (Gewolb, Josh. Random samples. Science 26 October 2001;294:777. See Kleinfeld, Judith S. Society. Jan/Feb2002; 39(2):61-66)

24 Dr. Kinsey and the Institute for Sex Research Alfred S. Kinsey (photograph from Wardell B. Pomeroy, Dr. Kinsey and the Institute for Sex Research)

25 Sex research in the mid-20 th century Alfred S. Kinsey (photograph from Wardell B. Pomeroy, Dr. Kinsey and the Institute for Sex Research)

11/7/ Kinsey et al. on selection and information bias (Alfred C. Kinsey, Wardell B. Pomeroy, Clyde E. Martin. Sexual behavior in the human male, Phila, W.B. Saunders, 1948) Alfred S. Kinsey (photograph from Wardell B. Pomeroy, Dr. Kinsey and the Institute for Sex Research)

4/2/2002Sources of error: Information bias27

3/22/2011Sources of error: Information bias28 Real-life example: Quit for Life Randomized trial of smoking cessation interventions Self-reported “In the past 7 days, have you smoked a cigarette, even a puff?” Attempted (unsuccessfully) to validate with saliva cotinine People who did not give a time to be called for validation had very high quit rates!

29 Kinsey biography by Wardell Pomeroy Pomeroy, Wardell B. Dr. Kinsey and the Institute for Sex Research NY, Signet / New American Library, 1972: p136

Non Campus Mentis “Hindsight, after all, is caused by a lack of foresight.” Anders Henriiksson (ed), Non Campus Mentis, NY, Workman Publishing Co., 2003, chapter 1