1. The PICO Model: Involving the Community in the Evaluation Process May 2012
Kate Lyon, MA, James Bell Associates, Inc.
Aleta Meyer, PhD, Office of Planning Research and Evaluation, Administration for Children and Families
Julie Morales, PhD, James Bell Associates, Inc.

2. PICO
P – The target population you plan to serve I – The intervention or program to be evaluated C – The comparison you will make to understand how well the program works in your community O – The intended outcomes you want to see achieved
A framework for developing a well-built evaluation question
PICO developed by Mark Testa (UNC Chapel Hill) and used extensively with the Children’s Bureau’s Permanency Innovations Initiatives to reduce long-term foster care. (find material from slides)
Adapted from: Permanency Innovations Initiative Evaluation Team (2011) Logic Model and Theory of Change. Presented at The Children’s Bureau’s PII Kickoff Meeting, Washington DC, November 2010

3. Tribal Maternal, Infant and Early Childhood Home Visiting Program
Administration for Children and Families, Office of Child Care in partnership with HRSA
5 year grants to Tribes, Tribal Organizations and Urban Indian Organizations
19 current grantees, up to 5 more
Funded to implement and identify evidence-based HV programs

4. Evaluation Component Rigorous evaluation
Lack of evidence around the effectiveness of home visiting programs in Tribal communities
Community-based participatory evaluation

5. Tribal HV Evaluation Institute
Technical assistance on rigorous evaluation, tracking and measuring benchmarks, continuous quality improvement, data systems
Intensive individualized TA and universal TA
Site visits with Federal team
Using PICO framework
Context in which TEI does PICO – during site visits to our grantees, we facilitate discussions around evaluation using PICO. The goal is to engage the entire group in thinking about what the evaluation question should be. We facilitate the discussion, but all of the content emerges from the grantee. It’s a way of allowing a participatory process in the development of the evaluation question, yet still working within a prescribed framework, as is so often the case.

6. The PICO Discussion
Brings together program leadership and staff, stakeholders, and evaluators
Collaborative discussion about community needs, program development, and evaluation
PICO builds rigor into the discussion
Allows participatory process within a prescribed framework

7. Population
Who is your target population for home visiting? What are their prioritized needs?
Needs assessment informs target population.
From broad: what is your community like?
To specific: who will you serve?

8. Intervention
Highlights the linkages between the needs of the target population, the program, and the benefits you hope to achieve.
What is the theory of change for the program(s) you have selected?
What implementation supports need to be in place to have a successful program?

9. Comparison Walk through different types of comparisons.
What is the alternative course of action that your comparison group will experience?
What is the strongest contrast that is feasible?
The strongest possible contrast will help to attribute any observed differences between the groups to your program.

10. Outcomes
What are the short- and long-term outcomes you hope to achieve?
Is there evidence that the program will impact those outcomes?
Which outcomes are most critical for the community?

11. PICO Example
Do urban American Indian children ages 0-5, living below the poverty level (P) whose families receive Parents as Teachers home visitation services (I) demonstrate greater school readiness (O) compared to children whose families receive usual services (C)?

12. PICO Example
Do women who are pregnant with their first child (P) who receive Nurse Family Partnership home visiting services (I) experience better birth outcomes (O) compared to pregnant women who gave birth at the clinic before NFP was implemented (C)?

13. PICO Example
Do home visitors (P) who receive enhanced reflective supervision (I) demonstrate greater confidence and competence in addressing difficult family issues (substance use and domestic violence) (O) as compared to home visitors who receive regular supervision (C).

14. For more information: Kate Lyon, TEI, James Bell Associates
Aleta Meyer, ACF, OPRE
Julie Morales, TEI, James Bell Associates
Anne Bergan, ACF, OPRE

15. The Tribal Evaluation Institute is funded by the Office of Planning, research and Evaluation within the Administration for Children and Families. TEI was awarded to James Bell Associates in partnership with the University of Colorado’s Centers for American Indian and Alaska Native Health and Michigan Public Health Institute. For more information, contact the individuals on this slide.
The Tribal Home Visiting Evaluation Institute (TEI) is funded by the Office of Planning, Research and Evaluation, Administration for Children and Families, Department of Health and Human Services under contract number HHSP WC.
TEI is funded to provide technical assistance to Tribal Home Visiting grantees on rigorous evaluation, performance measurement, continuous quality improvement, data systems, and ethical dissemination and translation of evaluation findings.
TEI1 was awarded to MDRC; James Bell Associates, Inc.; Johns Hopkins Bloomberg School of Public Health, Center for American Indian Health, and University of Colorado School of Public Health, Centers for American Indian and Alaska Native Health.
For more information on TEI contact:
Nicole Denmark Kate Lyon
Federal Project Officer Project Director
Office of Planning Research and Evaluation James Bell Associates, Inc.