1. Donna M. Mertens Independent Consultant Minnesota Evaluation Studies Institute, MESI Spring Training Minneapolis MN March 2015

2.  Name, position, organization (no acronyms please).  Where do you come from?  What is your experience in evaluation?  What has been your experience with social justice?  What do you want to get out of this workshop? Mertens MESI March 2015 2 2/19/2015

3. PhilosophicalAssumptions AxiologyRespect for cultural norms; support for human rights and social justice; reciprocity OntologyIssues of power & critical interrogation of multiple realities: social, political, cultural, economic, race/ethnic, gender, age, religion and disability values to unmask those that sustain an oppressive status quo EpistemologyIssues of power & Interactive link; knowledge is socially and historically located; trusting relationship. Methodology Qualitative (dialogic)/ Quantitative / Mixed Methods; Context 2/19/2015 Mertens MESI March 2015 3

4. 2/19/2015Mertens MESI March 2015 4 Phase 1 Qual: Identify stakeholder s; build relationship s; analyze documents Quant: Identify available demographic and environmenta l data Phase 2 Qual: Interview s, focus groups, town meetings Quant: Establish baseline for pollutants and health status Phase 3 Qual: Develop interventi ons based on stakehold er input Quant/Qual: Pilot test intervention s & data collection Phase 4 Quant: process assessment of implementatio n; pretests Qual: observe and interview stakeholder groups during intervention Phase 5 Quant: Post- test measures Qual: Interviews, observations, policy planning

5.  Explicit goals?  Implicit goals?  Contradiction between explicit goals and behaviors?  Changes in stance over time; changes in personnel over time  Explicit: Deaf people in the courts  Implicit: I don’t really need to deal with these multicultural issues because we are all white and well educated/ of British extraction 2/19/2015Mertens MESI March 2015 5

6.  “ism” present: depicting intended beneficiary as “less than” member of powerful majority  assuming problem is knowledge-based – not social/cultural/oppression  Problem: Self esteem; they just don’t care; have babies to get welfare $$  Problem: Ignorance (people with DA voting in Pakistan; HIVAIDS in Botswana) 2/19/2015Mertens MESI March 2015 6

7.  Tell them what they need to know, e.g., you have a right to vote and this is how you do it; you can say no or use a condom  Acknowledge the complexity of the solution that is needed as compared to the services that are offered; develop partnerships to address complexity  Use prior research and evaluation and applicable theories (e.g., developmental psychology or change theory); best practices  Consult with community about potential solutions: Address the meaning of the experience in their lives (HIVAIDS), connect solution to experienced needs (dancing, healthy cooking) 2/19/2015Mertens MESI March 2015 7

8.  Who to involve? How to involve?  Involvement throughout entire cycle of study: questions, methods, data collection, use  Notice: who is not here? Deaf people? Only people who I am comfortable with? 2/19/2015Mertens MESI March 2015 8

9.  Use low tech radio shows to solicit community members to make contact via mobile phones; do follow up interviews  Use smartphones for short surveys  Web based recruitment; data collection and dissemination  GPS and mapping  An uncomfortable topic: sex with children; drug dealing; gang activity; sex with same sex partner 2/19/2015Mertens MESI March 2015 9

10.  Appeal to shared desire to provide effective services addressing the needs of marginalized populations: process and impact evaluation  Argue for the primacy of the needs of the community and disadvantaged clients  Find a champion  Build a coalition  They don’t want this type of service; they won’t come…. 2/19/2015Mertens MESI March 2015 10

11.  Finding higher ground  What are you going to do if you have negative findings? 2/19/2015Mertens MESI March 2015 11

12.  Most Significant Change: Davies and Dart (2005)  ‘During the last month, in your opinion, what was the most significant change that took place for participants in the program?’ p. 10  Other PAR strategies (Robert Chambers)  Know the literacy level; comfort in speaking in groups  Inuit culture: Children would not speak to the evaluator. 2/19/2015Mertens MESI March 2015 12

13.  How to keep people safe?  Sex with same sex partner is not accepted in culture, is against the law, could be punished by death 2/19/2015Mertens MESI March 2015 13

14.  Wealth and status, gender, ethnicity, age group, sexual orientation, disability, deafness, refugee, immigrant, religion  Homogeneous groups; individual interviews  Supportive accommodations  Respect and meaningful involvement 2/19/2015Mertens MESI March 2015 14

15.  Present actionable results; identify clear options for possible results and associated actions  Identify strategies to overcome barriers and support change  Share results in clear and understandable manners to full range of stakeholders  Share results with those who can advocate for change (Cook, 2014)  South Asia policy makers and program developers 2/19/2015Mertens MESI March 2015 15

16.  Form coalitions  Press releases  Transparency  Present to multiple audiences in multiple formats 2/19/2015Mertens MESI March 2015 16

17.  Advocate for good evaluation (standards); advocate for justice  Evaluators have to be objective; they can’t be advocates. 2/19/2015Mertens MESI March 2015 17

18.  Mertens, D. M. & Wilson, A. (2012). Program Evaluation Theory and Practice: A Comprehensive Guide. NY: Guilford.  Mertens, D. M. (2015). Research and evaluation in education and psychology: Integrating diversity with qual, quant and mixed methods. 4th ed. Thousand Oaks, CA: Sage.  Mertens, D. M. (2009). Transformative research & evaluation. NY: Guilford.  Mertens, D. M. & Ginsberg, P. (2009).(Eds.) Handbook of Social Research Ethics. Thousand Oaks, CA: Sage.  American Evaluation Association (2011). Public Statement on Cultural Competence in Evaluation. AEA. Mertens MESI March 2015 18 2/19/2015

19. Donna M. Mertens, PhD Independent Consultant Donna.Mertens@Gallaudet.edu Mertens MESI March 2015 19 2/19/2015