3 Native People for Cancer Control (NPCC) The NPCC is one of twenty-five programs nationwide funded by the Community Networks Program of the National Cancer Institute There are two ongoing types of evaluation: a centralized national external evaluation and an internal evaluation focused specifically on the NPCC
4 Community Networks Program National External Evaluation
5 National External Evaluation Comparison of all 25 Community Networks Programs Logic Model – Phase I Core Data Collection – Phase I Logic Model – Phase II Core Data Collection – Phase II Logic Model Phase III Core Data Collection – Phase III
6 First Year Outcome of the National Evaluation for the NPCC The NPCC is now a national not a regional program To quote Ken Chu (the original designer of the external evaluation process): the NPCC “was just too large compared to the other programs to be regional.” Thus, NPCC participants are supplying the specific information needed for documenting the program and its impact on Native communities and organization.
7 Regional Native American Community Networks Program Internal Evaluation
8 Five Aims - NPCC 1) Increase cancer education activities among American Indians and Alaska Natives; 2) Build the capacity of tribal colleges and universities to become partners in cancer-related investigations; 3) Enhance cancer training opportunities for Native researchers; 4) Conduct community-based research that targets key cancer disparity issues; and 5) Reduce cancer-related health disparities by increasing interventions among American Indians and Alaska Natives.
9 NPCC Internal Evaluation Data Sources Annual Staff and Colleagues Questionnaire Scheduled Interviews with Selected Staff and Colleagues Overview and of Internal and External Program Reports All Questionnaire and Interview Data are Confidential
10 Internal Evaluation Strategies Process Evaluation - assesses extent to which program plans are implemented as anticipated Impact Evaluation – assesses immediate effects of program activities Outcome Evaluation – assesses longer term effects of program implementation
11 Internal Evaluation Report Qualitative and Quantitative Data Analysis Analysis of Positive Program Accomplishments Analysis of Barriers to the Implementation of Program Goals and Objectives Suggestions of Ways to Overcome Barriers to Program Implementation Suggestions of Modifications of Program Operation
12 Selected Results of the 2006 Staff and Colleagues Questionnaire Thirty-seven (97%) of the 38 eligible participants returned the questionnaire. Seventy-three percent of those surveyed stated that they would support a continuation of the NPCC work in 2010 when the current program ends. No one said that they would not support program continuation at that time. Seventy percent rated the “overall quality of the work accomplished in the first year of the NPCC” as “good,” “very good,” or “excellent.” No one rated the overall quality or work accomplished as poor. Seventy-eight percent rated the “overall commitment of the NPCC to improving cancer education and care” as “good,” “very good,” or “excellent.” No one rated the overall commitment as poor.
13 Selected Results Continued Seventy percent of those surveyed rated the “NPCC’s collaboration with you or your organization in the past year” as “good,” “very good,” or “excellent.” No one rated the collaboration as “poor.” Fifty-one percent rated the “sharing of information” in the NPCC as “good,” “very good,” or “excellent.” No one rated information sharing as “poor.” Thirty-six percent rated “referrals to or from other agencies or organizations” as “good,” “very good,” or “excellent.” No one rated the referrals as “poor.” Fifty-two percent rated “sharing resources (e.g. equipment, supplies)” as “good,” “very good,” or “excellent.” No one rated the sharing resources as “poor.” Forty-six percent rated “coordinating services” as “good,” “very good,” or “excellent.” No one rated coordinating services as “poor.”
14 Selected Results Continued Fifty-two percent rated undertaking joint projects or co-sponsoring events“” as “good,” “very good,” or “excellent.” No one rated the undertaking of joint projects or co-sponsoring events as “poor.” Fifty-six percent rated the “overall responsiveness of the NPCC to community needs” as “good,” “very good,” or “excellent.” No one rated the NPCC responsiveness as poor. Twenty-seven percent of those surveyed stated that what they “liked most about working with the NPCC” was the opportunity for sharing, networking, team effort, and collaboration. Forty-six percent (of those who are a member of another partnership or coalition) “share information from the NPCC” with their organization’s members. Twenty-seven percent of those surveyed also “liked most” the passion and commitment to the common goal the NPCC represented: making a difference in the lives of American Indians and Alaska Natives.
15 Barriers to Achieving NPCC Goals Fourteen percent of those asked about the nature of the barriers to the implementation of the first aim (increasing cancer education activities among American Indians and Alaska Natives) noted: potentially, lack of inclusiveness, input, ownership by tribes – getting leaders engaged to see cancer as their problem. Five percent of those asked about the nature of the barriers to the implementation of the second aim (building the capacity of tribal colleges and universities to become partners in cancer-related investigations) stated that a barrier is having the time and transportation time to reach out to other tribal colleges and universities. Another five percent noted that tribal colleges and universities are not used to working close with partners and thus improved avenues to communication were needed.
16 Barriers Continued Eight percent of those asked about the nature of the barriers to the implementation of the third aim (enhancing cancer training opportunities for Native researchers) stated: a barrier is the limited time in a five-year grant cycle. Fourteen percent of those asked about the nature of the barriers to the implementation of the fourth aim (conducting community-based research that targets key cancer disparity issues) noted: communication of NPCC researchers with tribal communities and getting support for community-based research. Eight percent of those asked about the nature of the barriers to the implementation of the fifth aim (reducing cancer-related health disparities by increasing interventions among American Indians and Alaska Natives) stated: potentially, lack of inclusiveness, communication, input, ownership.
17 Ways to Overcome Barriers to Achieving NPCC Goals Fourteen percent of those asked how the program could be “more effective in achieving the five specific aims” responded that the NPCC needed to establish more formalized tribal involvement. Sixteen percent of those asked for “suggestions to improve the existing program” stated that the NPCC needed to establish more formalized tribal involvement including the initiating of programs. Fourteen percent stated that the program needed to create more opportunities for dialogue, networking, collaboration, and communication within the NPCC. Eleven percent noted that an advertising program was needed to make tribal communities aware of the NPCC.
18 Ways to Overcome Barriers Continued Eleven percent wanted to disseminate news of the program and increase community outreach and education activities to tribal communities. Eight percent wanted to establish better tracking and communication of ongoing activities matched with the five program aims. Thirty-eight percent asked how the program could “help you and your organization” achieve your objectives for your work with the NPCC responded that they needed continued creative collaborative support for outreach programs, training, and subcontracts.
19 Internal Evaluator’s Conclusions This is the fastest and most innovative start to a program I have ever experienced in my extensive history as a program evaluator in American Indian and Alaska Native communities. This National Cancer Institute funded program has the very definite potential to serve as a national model for future community-based research programs in underserved communities facing health disparities.