Developing Culturally Competent Evaluation Tools with Client Organizations Jeanette Treiber, Ph.D. Project Director Tobacco Control Evaluation Center Department of Public Health Sciences University of California Davis School of Medicine (with the TCEC team)

Challenge and Approach Challenge: Tobacco Control Evaluation Center (TCEC) to provide guidance and capacity building to 100 California tobacco control programs serving culturally diverse communities Solution: 1. Learn from practitioners 2. Develop evaluation guidelines for specific cultural groups 3. Make tools “living documents” in an interactive process 4. Use web-based technology to keep the capacity growing

Overview  Method  Definition Culture  Definition Cultural Competence  Definition Culturally competent Evaluation  Clashing cultures in tobacco control  Capacity Building through exchange and research  Tools  Future Work

Method  Reviewing literature on culturally competent evaluation  Analyzing California Tobacco Control campaign culture  Bringing together tobacco control evaluation practitioners to exchange experiences  Infusing practitioner observations with lit reviews and KIIs  Developing and web posting of culture specific tools that are “living documents”  Continuing dialogue among practitioners through teleconferences and web interactions

Definition Culture = racial, ethnic, religious, social and political groups’ ever changing thoughtcustoms communicationcourtesies languagerituals beliefsmanners of interacting valuesroles practices

Definition Cultural Competency = “A set of congruent behaviors, attitudes and policies that come together in a system, agency, or among professionals and enables that system, agency or those professionals to work effectively in cross- cultural situations.” (Endo et al. 2003)

What does cultural competence in evaluation mean? Culturally competent evaluation refers to:  Evaluators with cultural know-how  Inclusion of community voices  Culturally attuned evaluation design and implementation

Clashing Cultures Culture of California tobacco control strategy  based on norm change theory  Based on proven intervention techniques  Successful in reducing overall smoking rates in California  Less successful with a number of cultural groups  Critical role of evaluation in success of strategy

Role of evaluation = critical Example: Community assessment activity for smoke-free housing objective Assessment of community is critical to success of campaign Accurate assessment is not possible if there are cultural barriers to assessment approach Some cultural groups will be excluded from assessment Because of the skewed assessment the campaign will not be successful with some cultural groups

Capacity building through exchange and research  Evaluation Center brought evaluation practitioners together  Facilitated discussions on strategies and methods for a number of cultural groups  Enhanced the knowledge through research and KIIs  Developed “living tools”/guidelines and made them available on the Center’s website  Continues the dialogue through various forms of interactions and updates (teleconferences, blogs, listserves, etc.)

Tools at “Culture in Evaluation” tools on TCEC website have:  guidelines for developing data collection instruments for a specific culture  guidelines for working in and with the community  Information on the culture’s relationship to tobacco and health  Information on diversity within the culture

Future work  Case study with a TC program directed at Asian Pacific Islanders  State-wide quantitative research on evaluators’ experience with cc  Study the degree to which cc tobacco control evaluation effects higher outcomes in reducing health disparities