1. Presented by: Community Planning & Advocacy Council.
Outcome Accountability Success and Impact of Services Adapted from FRIENDS, the National Resource Center for Community Based Child Abuse Prevention
Presented by:
Community Planning & Advocacy Council.

2. Why are we here??
Program evaluation—the systematic process of collecting information and providing answers to important questions about program outcomes—is an obligation, not an option.
All programs need periodic self-assessment in order to know whether they are fulfilling their overall purpose of supporting families.
Accountability involves telling stakeholders what you are going to do, doing what you said you would do, and providing evidence that you lived up to your promises.

3. Why are we here??
It is critical for programs to demonstrate that desired outcomes are occurring as a result of their services.
Good outcome evaluation is needed so that funders can see what results their money is buying and to assist programs ensure quality services.

4. Key components
over a sufficient period of time, in enough communities, family support programs (like respite and crisis care, mutual self-help parent support groups, parent education classes, home visiting, etc.) can be expected to produce a progression of outcomes beginning with short term outcomes (engagement and learning), that lead to intermediate outcomes (behavior change), and eventually resulting in long term outcomes (population wide impact such as reduced risk/incidence of child abuse and neglect).

5. Not all Agencies are Equal
A small community-based program that is not experienced in the methods of outcome evaluation should not be expected to conduct a rigorous, long term experimental model evaluation.
They should, however, be expected to conduct evaluation activities on some meaningful level.

6. Evaluation Plan
Evaluation is a process that should be built into any program that provides supportive services to children and families.
It is not a one-time-only activity, but a cyclical process that involves careful thought about the people you serve, the challenges they face and the changes that you hope your services can bring about.
Through your evaluation activities, you will measure the extent to which you were successful in facilitating the desired changes.
But the process does not end there; good evaluation requires that you reflect on what you have learned, and adjust your services, with the net result being stronger and more effective community-based child abuse prevention programs.

7. Outcome accountability is a process
Developing a Logic Model
Your program’s Logic Model is the vital conceptual tool that links the needs of the people you work with, the results you want, the way you will work together to get those results, and the methods you will use to find out if those results are being achieved.

8. Logic Model Vision: Target Population: Services and Assumptions:
Outcomes
Resources
Indicator/Result
Measurement Tool
Responsible Individual
Timeline
Short Term
(Engagement and/or learning)
Intermediate
(Implementation of new skill or knowledge)
Long-Term
(Sustained)

9. Vision Statement
This is a brief statement about your hope for the future. What do you want for the families and community that you serve? A vision statement does not necessarily need to be measurable. Your program is not necessarily responsible for single- handedly achieving it. Rather, your program should contribute to its achievement.
Vision statements can be very diverse. Some examples are:
Strong, Safe and Healthy Families
All Children Ready for School by 6
Families Free From Violence

10. Covenant for Children Youth and Families: Vision Statements
Early Childhood Success: All young children will be safe, healthy and ready to learn as they enter and engage in school.
Positive Youth Development: All school age children and youth will live in a permanent home, be in school, on track (involved in age appropriate activities) and connected to their community.
Strong Families/Strong Communities: All families and communities will have capacity and resources to successfully nurture, support and inspire their children on their journey from birth to adulthood.

11. What is a target population?
This is a description of the population you serve. As specifically as possible, identify the people who will receive your services. Examples include:
“any parent living in our county with a child under 3”, or
“any non-custodial parents residing in our town”,
“All parents of children with disabilities living in the County” and
“parents referred by the local DCP&P agency due to risk of out-of-home placements.”

12. Services and Assumptions
This is the portion of your logic model where you briefly describe the services your participants will receive. What are the activities provided that are directly linked to the outcomes that you wish to achieve?
For example, if you are following a particular curriculum, such as Parents as Teachers, you would name the curriculum and indicate how many sessions will be offered over what time periods. Other examples could be “Parent education groups using the Families Together curriculum”, “Bi-monthly home visits by Maternal Child Health Nurses”, etc.

13. Assumptions
The services you offer should be based on what is known to be effective.
What assumptions are you making that suggest your services will bring about the desired outcomes, with the population you serve?

14. What are resources?
Resources are what your program needs to provide your services.
Do you need to provide food for your parent education group?
Do you need to buy a curriculum?
Is there any specialized training that your staff will need?
Will you need to provide child care, transportation, meeting space?
Think of this section as a modified budget justification.

15. What are outcomes?
If you are successful in providing your services, what changes will your participants experience?
Generally, outcomes describe who, will do, what as a result of your services.

16. Progression of Outcomes
Outcomes can be:
short-term, usually changes in attitude, beliefs and knowledge;
intermediate, which can be developing and practicing new skills;
or long term, including permanent changes at an individual level or changes that create an impact on larger social structures.

17. Examples of Outcomes
(Short-term, or learning outcome): Participants demonstrate knowledge of how to create a child-safe home.
(Intermediate or practice outcome):
Participants make their home child-safe.
(Long term, permanent change):
Participants maintain a child-safe home.

18. What are Indicators?
Indicators answer the question: What would I see or hear that would tell me that an outcome was being achieved?
Indicators are concrete, specific descriptions of what will you will measure.
An indicator can include the number or percentage of participants projected to achieve it.
For example, “85% of the participants access …”

19. What are some examples of indicators?
For example, let us say that one of your outcomes is “Participants make their home child-safe”. What would you see or hear that would tell you that a participant has a child safe home? Here are a few possible indicators:
Participants access and install supplies needed to have a child-safe home.
Participants remove household hazards that jeopardize child safety.

20. Measurement Tools Ask yourself:
How will I “test” or capture information that tells me that the outcomes have been achieved?
These mechanisms for gathering data are called measurement tools-usually surveys, questionnaires or checklists.

21. Measurement Tools
Programs need to choose their measurement tools carefully.
They should choose tools that genuinely measure the outcomes you wish to achieve to ensure accurate results.
Review tools carefully before making decisions on which you will use.
Find out what tools programs similar to yours have used. Ask others about their experiences using it.

22. Developing your evaluation plan:
It is important to determine how and when you will collect the data necessary to measure outcomes.
Who will be responsible for doing the measurement or data collection,
how will it be collected (face-to face interview, mail-in surveys, web-based questionnaires, etc.) and when the data collection will take place.
It is also critical that staff be identified to manage the data. Usually this means entering the data into a spreadsheet or database.
You will also need to plan time for staff and participants to come together and reflect on lessons learned from the evaluation.

23. Evaluation Implementation Plan
Activity
Steps
Responsible Parties
Target
Date (s)
Build
Logic
Model
Write program vision.
Describe population served, including the need to be addressed.
Describe the services & resources needed; What assumptions are you making that lead you to believe your service strategies will “work”?
Write program outcomes and indicators.
Determine how you can measure these outcomes.

24. Evaluation Implementation Plan cont’d
Activity
Steps
Responsible Parties
Target
Date (s)
Select
and/or
Develop Tool
Select or construct appropriate measurement tools
Develop or obtain a method for data entry/management.
Administer and/or review tools and administration procedures with participant focus group.
Revise program developed tools as needed and/or select different standardized tools according to findings.

25. Evaluation Implementation Plan cont’d
Activity
Steps
Responsible Parties
Target Date (s)
Administer
Select time, place & participants for evaluation.
Train staff on how to administer the evaluation tool.
Make copies of all tool to be used.
Administer evaluation tool.

26. Evaluation Implementation Plan cont’d
Activity
Steps
Responsible Parties
Target
Date (s)
Analyze
and
Report
Enter data into database/spreadsheet and make necessary tabulations.
Analyze data and meet with staff team to review results.
Make program decisions based on findings.
Report on results of the evaluation.

27. Q & A
FRIENDSNRC.ORG
Ready, Set, Go…Fill in your logic model