Methodological Issues 2007 CFRF Annual Workshop Jill Belsky Professor and CFRF steering committee member 1. Why care about “rigour”? 2. Some methodological “tools” 3. Some references 4. Your ideas, comments, questions

1. Why care about rigour (not rigor mortis)? (Neef 2003: 4) “Following a boom period throughout the 1990s, the theoretical, conceptual and methodological foundations of participatory approaches have attracted increasing criticism in the last years. Among the main issues noted were methodological limitations and lack of scientific rigour.” Examples? a. insufficient analysis b. too much researcher bias c. limited precision of approaches and systemic validation of results d. concealed interests of marginalized individuals who are not in organized groups

Defining rigour in PAR: Branigan 2003:37 “…rigour is evident in research when the methods used are those that can represent the fullest, most detailed, rich and expressive picture of a particular situation...Rigour is in large part dependent on the researchers themselves Swepson 2000:8 “…a more appropriate criterion of rigour is the degree of the relevance of the methodology to the problem; the one which best allows the researcher to conduct systematic inquiry in order to present a warranted assertion—that is, the methodology is fit for a given function.”

2. Some Methodological Tools 1. Relevance -what is the research question and why are you asking it? -who or what is it intended to benefit from it and how? -how are you defining “relevance”? if to increase knowledge, whose knowledge? if to increase empowerment, whose empowerment? if to increase action, who wants it to happen? -what information is most relevant and which method(s) most likely to gather it?

2. Triangulation (cross-checking results) -compares the results from either two or more different methods of data collection or two or more data sources from same method -search for patterns of convergence to develop or corroborate an interpretation -a type of test of validity because it assumes that any weakness in one method will be compensated by strengths in another (i.e., different methods as well as researchers are more or less able to “see” a part or slice of the total situation).

3. Combining and mixing methods Not all research questions, or points along the research process, require the same level of local/community participation Standards such as interviews, focus groups, surveys, participant obs., ethnography… and also mapping, transect walks, seasonal calendars, time/resource use/other trend analyses, venn diagramming (social networks, groups), matrix scoring and ranking, drama and participatory video making…. Yes, its okay and maybe even necessary to mix methods from quantitative and qualitative research!

4. Participant feedback -especially relevant to qualitative or interpretative research methods -investigator’s account is compared with those of the research subjects (eg “is that what you said?) and their reactions are incorporated into the study findings -a related technique is to use “low inference descriptors” or use of description phrased very close to participants’ accounts and researcher’s field notes, such as direct quotations

5. Clear exposition of methods of data collection and analysis; context - provide a clear account of how data were collected and analyzed, assumptions made, context (“audit trail”) -written account should include sufficient information and data to allow the reader to judge whether the interpretation offered is adequately supported by the data and is credible - a related method would be to discuss all of the above with other people, including peers who are familiar with or disinterested in the research –

6. Critical Reflection -”reflection” means “turning back” on experience to be aware of the ways in which the researcher and the research process shape the data, interpretations, and actions (especially prior assumptions and experiences) -”critical” includes questioning taken-for-granted beliefs that relate to our experiences and how we make meaning of it (including our research); question our personal and intellectual biases and make these transparant; pose new questions for ourselves as to how we can reframe our inquiry for new possibilities for thought and action

7. Attention to negative cases -in addition to providing serious exploration of alternative explanations for the data collected and interpreted, involves a search for and discussion of factors in the data that contradict the explanation provided 8. Fair dealing - attempt to incorporate a wide range of different perspectives so that the view point of one individual or group is not overstated (very relevant to com forestry!) - may suggest conventional (random) sampling techniques are warranted

Take away points -the questions to be answered and actions to be fostered are determined by the researcher and community groups -the questions determine the methods to be used no method is inherently “participatory” or not, though some are more effective than others to foster dialogue, trust, reveal power and political conflict  context -PAR can and should be rigorous based on definitions and criteria discussed here and throughout the workshop -participatory researchers build on standard research methods – alone or in combination plus others, emphasizing self-critique and reflection, innovation, resourcefulness, collaboration, respect for multiple knowledge systems, production and methods!

3. Some References Branigan, E ‘But how can you prove it? Issues of rigour in action research. Journal of the HEIA 10(3): Denzin, N. and Lincoln. Y. (Eds.) Handbook of qualitative research. Thousand Oaks, CA: Sage Publications. Finn, J.L and M. Jacobson Just Practice: A Social Justice Approach to Social Work.Eddie Bowers Pub., Iowa. Mays, N. and C. Pope Assessing quality in qualitative research. BMJ 320(50-2). [Download from bmj.com] Neef, A Participatory approaches under scrutiny: will they have a future? Quarterly Journal of International Agriculture 42(4): Sutherland, A Participatory research in natural resources. Natural Resources Institute, The University of Greenwich. Swepson, R Reconciling action research and science. In Branigan, E ‘But how can you prove it? Issues of rigour in action research. Journal of the HEIA 10(3):37-38.

4. Your ideas, observations and questions…