1. Rapid Qualitative Research Tools Indirect Observation technology in qualitative research

2. Learning Objectives define the concept of a RAP and describe its history list recent developments of RAP instruments discuss issues of validity with RAP describe some unobtrusive measures list types of data obtained from unobtrusive observation field work

3. Class Break After the first part of RAP and before comparisons with PRA and Indirect Observation VISUAL DISPLAY EXERCISE….

4. Comments on visuals exercise just completed

5. Next Session on ANALYSIS Each group to write at least three themes on the board for the others to inspect

6. History of RAP began in questioning the questionnaire survey as a way to address broader issues of need for thinking about research among the rural poor STUDENTS REACTIONS TO QUESTIONNAIRES –Surveys? Needs assessments

7. Questionnaire History came from academics to write a thesis or publish in an academic journal question formulation did not allow –to discover systems, –nor to explore cognitive perceptions –nor to identify social problems, questions were usually formulated from outside motivation to use it came from comfort in numbers, like a child’s blanket

8. Questionnaire Method contains dependent & independent variables strict statistical analysis applied to determine if pre- formulated hypotheses are rejected or accepted random sampling & minimization of bias to maintain scientific respectability

9. Questionnaire Strengths large numbers of people could be surveyed statistical analysis with confidence limits could be achieved generalization possible

10. Questionnaire Weaknesses Read Stone, L. and J.G. Campbell, The Use and Misuse of Surveys in International Development: An Experiment from Nepal. Human Organization, 1984. 43(1): p. 27-37 –tried to measure KAP survey, and found non- sampling error was far greater than sampling error –much of it relates to different interpretations of questions than considered by designer of questionnaire

11. Example of non-sampling error " I am Pro/Anti Church?" Person A answered #2 –1. Very Prochurch (The church is the greatest institution in America) –2. Prochurch ( The church is a powerful agency for promoting social and individual welfare) –3. Antichurch ( The church serves a useful need for some people) –4. Very Antichurch ( Church services are not always completely interesting) Person B also answered #2 –1. Very Prochurch ( The church serves a useful need for some people) –2. Prochurch ( Church services are not always completely interesting) –3. Antichurch (The church seeks to impose worn-out dogmas on individuals) –4. Very Antichurch (The church represents shallowness, hypocrisy, and prejudice.) Attitudes and Persuasion: Classic and Contemporary Approaches" by Richard Petty and John Cacioppo. Wm. C. Brown Company Publishers, Dubuque, Iowa 1981; page111

12. Question wording, format, context When asked what they consider "the most important thing for children to prepare them for life" 61.5% chose "to think for themselves" when offered that alternative on a list, yet only 4.6% said such in an open-ended question. When asked how successful they have been in life, 34% reported high success (scale -5 to 5) but only 13% when 0 to 10 scale used Marital satisfaction is major or minor contribution to life satisfaction depends on order questions asked with.18 ≤ r ≤.67

13. Question wording, format, context When asked how often psychosomatic patients reported symptom frequencies more than 2x/month 62% when response scale ranged from –"twice a month or less" to –"several times a day" 39% when scale ranged from "never" to "more than twice a month"

15. Questionnaire Weaknesses Stone, L. and J.G. Campbell, The Use and Misuse of Surveys in International Development: An Experiment from Nepal. Human Organization, 1984. 43(1): p. 27-37 what people say is not what people do results are cultural and time-bound context of an activity is not revealed person asking questions introduces a bias, since deference or untrue answers can be given

17. Questionnaire Weaknesses Culture is a big issue: Self-assessed health or self- assessed well-being Sampling is tricky where people do not have phones or mail boxes, or not live on linear streets Analysis takes time

18. RAP: Basic elements of rapid surveys Low cost Quick feedback of results Smaller sample size Shorter questionnaire Computerized data capture and analysis –(MacIntyre K. Rapid assessment and sample surveys: trade-offs in precision and cost. Health Policy Plan 1999;14(4):363-73.)

19. Rapid Assessment Procedures basis in rural sociology, where called RRA, in late 1970’s, PRA was an outgrowth also used in agriculture & rural development, especially community forestry health efforts began about 10 years later, used to be more common in HIV work, others, less today –recognized that traditional surveys to assess health care were expensive, slow, and had poor validity

20. Basics of Rapid Assessment Procedures (RAP) Pre-designed study protocol –Other Terms RA (Rapid Appraisal) RRA (Rapid Rural Assessment) FES (Focused Ethnographic Study) User-friendly Interdisciplinary approach

21. Basics of Rapid Assessment Procedures (RAP) Speed & cost Focused on topic or problem –e.g Crap Rap, Rapid Assessment Procedures to Improve the Household Management of Diarrhea by Herman & Bentley –STD Rap The Manual for Targeted Intervention Research on Sexually Transmitted Illnesses with Community Members –Qualitative Methods: A field guide for applied research in sexual and reproductive health –Acute Respiratory Infections FES HAVE COPY HERE –Women’s Health FES HAVE COPY HERE Usually exploratory to learn about something

22. Student Experiences with RAP? Not much rapping going on after initial interest in the early 1990s Mostly with filiariasis, shistocomiasis, malaria, –USA: study on Baltimore youth, –San Diego youth shootings –Injury control (Crap Rap) STD/HIV rap, didn't take hold WHY?

23. Example: Women’s Health FES for India quite sophisticated including: –training syllabus, using unstructured interviews, and –computerized analysis

24. Purpose: Carrying out a community-based ethnographic study of women's health. –main goal provide techniques for the generation and analysis of data to facilitate program development and implementation in organizations working in women's health.

25. The results of the ethnographic study are intended to assist in: Developing recommendations for appropriate communication with women, and to enhance development of effective home care recommendations and health messages; Selecting appropriate forms of popular local terminology for health care workers to communicate effectively with women;

26. The results of the ethnographic study are intended to assist in: Improving household morbidity, mortality and treatment surveys by suggesting ways of adapting questions to consider/acknowledge community perceptions and practices Identifying constraints in improving the condition of women's health and suggesting locally appropriate strategies to deal with these constraints.

27. Training Exercises over a 10 day period for data collectors Direct Observation: Community Mapping & Resources Key Informant Interviews with Local Healers & Other Knowledgeable Women Free Listing of Women’s General Problems in the Community Free Listing of Women’s Health Problems in the Community, exploring domain of women’s illnesses Free Pile Sort: Grouping Women’s Illnesses

29. Key Informant Interviews with Women: Health Concepts and Health Seeking Behaviors Ethnographic Field Guide –General issues related to Health Problems of Women in the Community

30. Exploring women's health by developmental state

31. "Tell me the story of your life. What was life like for you as a girl? –What were some of the good things you remember? –What were some of the problems you remember? –Were you ever ill? –Were your brothers or sisters ever ill? How many times have you been pregnant? –How many live births have you had? –How many live children do you now have? –How have these pregnancies affected your health?

32. "Tell me the story of your life. What was life like for you with your first pregnancy? –Tell me what you understand about the process of pregnancy. –Where does the baby come from? –How did you know you were first pregnant? –Where does the baby grow in your body? –How does it grow? –What did it feel like for you to be pregnant?

33. "Tell me the story of your life. What was life like for you with your first pregnancy? –What did you think? –What did you not think about? –What did you wish you knew then, that you know now? –What are questions that you still have about pregnancy?" – You may be inspired to do a further piece of data collection, for instance, asking which of these different stages is "most joyous," "most traumatic," most important, etc.

34. Health and Health-Seeking: "How is your life going right now? How would you say your health is? –What things have made you healthy? What things have made you weak/sick? "Tell me about other illnesses you have had. –What were the first symptoms of illness X ? What caused those symptoms? What did you think when you noticed those symptoms? What did you do? –"Where do you go or whom do you consult when you are ill?"

35. Women's Status/Power: Ask for narrative accounts of what happens during meals. And what happened the last time your child was sick etc. Who eats first? Who makes decisions about cash? How are household jobs divided up (may be important to determine status and authority among women)? –What ways are men and women treated differently in your household? Who is the most important person in your household? Why?

36. Women's Status/Power: How do others treat that person? What responsibilities does each person have to the household? What benefits does that person gain from the household? Who is the second most important person? How do they address that person? What do they ask that person to do for them? What benefit does that person get from being in the household?

37. Data Analysis Developmental Stages Table Types of Practitioners Table

41. Paired Comparisons: Illness and Choice of Provider

46. Preliminary Analysis and Devising a Coding Scheme for Textual Data begin the process of analysis of the entire data set. develop a coding system for the textual data collected in this research. identify areas of further research and areas which are no longer fruitful for each of the three topic areas. evaluate which research methods worked well and which should either be dropped or modified.

47. Main Exercises, carried out in subsequent 4-6 weeks

48. Main Findings of this study in India (see Gittelsohn book, available for $19 from him) white discharge is a major complaint of rural Indian women –variation in perception of severity of problem

49. Main Findings of this study in India white discharge was commonly linked with other key health problems of women –other terms/names are used when describing their problems to health practitioners, including weakness backache –reasons are: feeling of shame and embarrassment women recognize clusters of signs and symptoms of illnesses that “go together” –from MDS, white discharge close to dysmenorrhea, abdominal pain, weakness & UTI’s

51. Main Findings of this study in India perception of these illnesses are the “lot” of women - and should be accepted as such evidence of relative power of women within the household women’s perceptions of their health problems –studies reflect guilt, embarrassment & shame women feel in relation to their reproductive health problems

52. Main Findings of this study in India white discharge carries significant social burden of guild and shame, which reduces women’s likelihood of seeking timely, effective treatment women possess a systematic framework for thinking about and organizing their illnesses, Fig 1 patterns of treatment-seeking for white discharge and other women’s health problems are based on their own illness experiences and their culture-based ethnomedical frameworks

53. Explanatory Model: A systematic set of cultural knowledge, beliefs and attitudes with respect to a particular illness or other aspect of health. The "explanatory model" for an illness includes the presumed causes, processes, preferred treatments, and expected usual outcomes.

55. Applying Data to Programs Health Education –what is needed –what is level of local knowledge –creating messages –identifying channels, media, etc. Improving Services (usually means educating providers)

56. Design issues to consider: Who gathers data? How long does it take to gather data? How much depth do we need? Who interprets data & communicates results? Who uses the results? How quickly are the results available?

57. INDIRECT OBSERVATION Study people’s behaviors without their knowing it less reactive (people don’t respond, and behave differently because they are being observed) unobtrusive observation by Bernard. ethical difficulties here.

58. Hand washing in public bathrooms (NYT 960917) 60% in Penn Station washed, up to 78% in Chicago phone survey: 94% said they washed their hands

59. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 Research Objectives The American Society for Microbiology is interested in replicating an observational study and telephone study that Wirthlin Worldwide conducted in 1996. The objectives of the research are to observe adults in public restrooms to determine the proportion who wash their hands, and also to poll the American public to determine how frequently they say they wash their hands in several situations: –After petting a dog or cat –Before handling or eating food –After coughing or sneezing –After using a public restroom –After handling money –After changing a diaper –After using the bathroom in their home The purpose of this research is to increase public awareness of the importance of handwashing.

60. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 Methodology – Observed Behavior Wirthlin Worldwide observed the behavior of 7,836 individuals in public restrooms and recorded whether or not they washed their hands after using the facilities. The research was conducted in 5 different cities: Observers discreetly watched and recorded whether or not adults using public restrooms washed their hands. Observers were instructed to groom themselves (comb their hair, put on make-up, etc.) while observing and to rotate bathrooms every hour or so to avoid counting repeat users more than once. Observers were also instructed to wash their hands no more than 10% of the time. City & SiteDate(s)MalesFemalesTotal Chicago, Navy Pier8-26106115362597 New Orleans, Treasure Chest Casino8-24, 8-278476661513 San Francisco, Golden Gate Park8-24, 9-2259294553 Atlanta, Braves Baseball Game8-27365525890 New York City, Grand Central Station & Penn Station 8-24, 8-28105712262283

61. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 Methodology – Telephone Poll Wirthlin Worldwide also interviewed a total of 1,021 U.S. adults via telephone about their handwashing behavior in several situations. Interviewing was conducted during the period of Sept. 1-4, 2000. Results are weighted by age, gender, geographic region, and ethnicity to ensure reliable and accurate representation of the 18+ population in continental U.S. Margin of error for a sample of this size is + 3% at a 95% confidence level, meaning that in 95 out of 100 cases a survey of this size will produce responses with + 3% of the result that would have been obtained had the entire universe of adults in the continental U.S. been interviewed.

62. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 Executive Summary There is a huge gap between self-reported handwashing behavior after using public restrooms and actual handwashing behavior – people are less likely to wash their hands after using public restrooms than they say they are. This is consistent with trend data collected four years ago. –More than nine in ten (95%) say they always wash their hands after using public restrooms. However, only slightly more than two-thirds (67%) are observed washing their hands after using public facilities. Among males, nine in ten (92%) report that they always wash their hands after using public restrooms, but only 58% of those observed actually did. This represents a gap of 34%. –Interestingly, compared to actual handwashing behavior in 1996, men nowadays are significantly less likely to wash their hands after using public restrooms (58% actually washed their hands in Aug. 2000 vs. 61% in Aug. 1996). Nearly all women (97%) surveyed report that they always wash their hands after using public restrooms, but only 75% observed actually did. This represents a gap of 22%. Similar to 1996, women continue to be significantly more likely than men to say that they always wash their hands after using public facilities (97% vs. 92%).

63. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 Executive Summary When asked about their handwashing behavior for a list of activities, women were also significantly more likely than men to indicate that they always wash their hands for the following activities: –after using the bathroom at home (90% women vs. 81% men) –after changing a diaper (86% women vs. 70% men) –before handling or eating food (84% women vs. 69% men) –after petting a dog or cat (54% women vs. 36% men) –after coughing or sneezing (40% women vs. 22% men) –after handling money (28% women vs. 12% men) The less affluent appear to have more stringent handwashing behavior than their more affluent peers – this is consistent with results from 1996. Those with incomes of $35,000 or less are significantly more likely than those with incomes of more than $35,000 to say they always wash their hands… –Before handling or eating food (82% vs. 73%) –After petting a dog or cat (51% vs. 40%) –After coughing or sneezing (37% vs. 26%) –After handling money (26% vs. 15%)

64. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 Executive Summary The less educated appear to be more hygienic than their counterparts in some respects – this is consistent with results from 1996. Those with high school or less education indicate that they are significantly more likely than those with some college/college education to always wash their hands… –Before handling or eating food (83% vs. 73%) –After petting a dog or cat (55% vs. 38%) –After coughing or sneezing (40% vs. 26%)

65. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 The less affluent are more likely than the affluent to report that they always wash their hands - Among U.S. Adult by Income* - Statistically significant at the 95% confidence level * National telephone study (n=1,021)

66. WIRTHLIN WORLDWIDE American Society for Microbiology Handwashing Study September 2000 The less educated are more likely than the educated to report that they always wash their hands - Among U.S. Adult by Education* - Statistically significant at the 95% confidence level * National telephone study (n=1,021)

67. Marker behaviors Easily observed/less sensitive behavior used as an indicator of another difficult to observe/sensitive behavior, –someone in Nepal carrying a lota is going to defecate –sexual behavior of a population is very difficult to study, and this is faced in current AIDS epidemic study of bathroom graffiti in Manila and Chicago –42% in Chicago dealt with homosexuality compared with 2% in Manila

68. Physical clues which indicate what a person’s behavior was (behavior trace studies) –wear spots on a floor or carpet to see where people went –graffiti NIH bathrooms –doodles on class notes –garbage (garbology), a trace measure Rathje at U. of Arizona has been studying behavior patterns by analyzing garbage from a representative sample of residents since 1973 (Bernard Pg 442 for details)

70. Other examples in Bernard Chapter 15 (2002)

71. Difficulties with Indirect Observations people may change their behavior when they learn you are doing this –in garbage project, people put out fewer liquor bottles very time consuming, expensive

72. Difficulties with Indirect Observations how far do people go to get data? Techniques of Deception –Tearoom Trade Study, Bernard pg 348 on fleeting homosexual behavior in public washrooms, Laud Humphreys observed homosexual acts, while acting as a lookout He recorded identification data on subjects, and five years later interviewed them while not revealing his identity! Would a researcher go to jail to protect data that could be incriminating? –ethnical issues discussed in Bernard, Chapter 15

73. Archival Research Kroeber’s study of women’s fashions records of births, deaths, causes of death, etc. –e.g. L A County had 82 homicides in 1953, and 2512 in 1992, while population doubled, incidence increased almost 15 times, or 1431%

74. Focus on Sexuality in Magazines US News and World Report 1962, 1972, 2000

75. Next Session on ANALYSIS Each group to write at least three themes on the board for the others to inspect

76. Comparing RAP & PRA Table 1 Table 2: The RRA-PRA Continuum

80. Principles Shared by RAP & PRA (Chambers 1992 pg 14-15) reversal of learning –learn from rural people, directly, on the site and face-to-face, gaining from local physical, technical and social knowledge learning rapidly and progressively –with conscious exploration, flexible use of methods, opportunism, improvisation, iteration and cross- checking, not following a blueprint program but being adaptable in a learning process

81. Principles Shared by RAP & PRA Off-setting biases, –especially those of rural development tourism, by being relaxed, and not rushing, listening not lecturing, probing instead of passing on to the next topic, being unimposing instead of important, seeking out the poorer people & women and learning their concerns and priorities

82. Principles Shared by RAP & PRA optimizing trade-offs triangulating seeking diversity Additional principles stressed in PRA –facilitating-they do it –self-critical awareness and responsibility –sharing of information

83. This course has all the elements of RAP, we have just been learning the techniques isolated from one another, but your projects, if they were to be disseminated for others to do, would be RAP’s

84. Differences between RAP’s and questionnaires, surveyor gets more involved in the process, rather than just checking boxes on a data collection form richness of the information obtained

85. Politics of public health work: Budgets, priorities of funders –Funds are often tied Ideologically Contractors Three questions about things: –What are the facts? –What are the presuppositions about the facts? –What are the questions you aren't supposed to ask?

86. Validity Is inductive nature of study compromised too much? Brief time frame of contact & trust issues How does multidisciplinary team affect the process? Technical lead and field worker competence Level of commitment of technical lead

87. What is “quick and clean” -- an attempt to refine RAP Intent of the tool & audience clarification Programmatic questions Explain process to “customize” Community participation vs. program managers vested Methods and data analysis