1. Evaluation 101 – Evaluation for the Small Community Based Organization
Leticia Peguero
Ayorkor Gaba
The Robert Wood Johnson Foundation

2. “If you don’t know where you’re going, you’ll end up somewhere else….
To know how well you’re doing…you must have some place you’re trying to get to….
“If you don’t know where you’re going, you’ll end up somewhere else….

3. ASSUMPTIONS
VALUES
BELIEFS

4. Pathway Mapping How does it work?
We all hold beliefs and theories that guide our thinking about problems and the way we respond to them.
Clarity about the problems and about how to work with these problems is critical because these ideas and beliefs guide our work…

5. Pathway Mapping How does it work?
Define key problems/issues challenging our communities and clients
What are our UNDERLYING ASSUMPTIONS?
Describe WHAT we are hoping to change (our desired outcome) in this community through our intervention
Describe HOW (strategies, actions, interventions) we hope to accomplish these outcomes

6. Pathway Mapping How does it work?
Examine assumptions about what works….
While assumptions can be based on experiences from the field, sometimes they are built on intuition. Challenging these assumptions allows us to look more closely at their theory and reflect on the choices we are making.

7. Pathway Mapping How does it work?
Think about the assumptions we are making concerning WHY an intervention might work...
Discuss differences in assumptions among stakeholders in your organizations
CHALLENGE THOSE ASSUMPTIONS!!
What and whose experiences are they based on?

8. Think about the assumptions that we are making concerning WHY a particular intervention might work
We don’t want our theory to fall apart so…lets identify our assumptions…

9. Strategies
Actions
Outcomes

10. Closely examine the links between our strategies, actions and outcomes and ensure they are logical and achievable
Identify and address gaps in our strategies – paths that might not get us where we want to go

11. Positive changes that you hope to achieve in the long run
Context:
Identify the major conditions that serve as the backdrop to your work…what are the opportunities and challenges surrounding your work
Assumptions
Goals/Strategy of the Program
The overall approach to conducting your work. If you cannot implement all of the goals…prioritize them
Actions/Program Activities
Activities are the individual services your organizations uses to implement strategies. Identify the services you want to provide
Short Term Outcomes
Identify the short term changes in your target population after they have participated in your program/activity
Long Term Outcomes
Positive changes that you hope to achieve in the long run

12. Theory of Change…..

13. Theory of Change… What is it ? It helps to clarify
A theory of change explains your underlying understanding of the issue you are addressing
It helps to clarify
WHY you are doing
WHAT your doing

14. Developing a theory of change…
Convene stakeholders for a planning session
Designate a facilitator
Address each element of the theory of change
Share with stakeholders and revise based on feedback

15. Using a theory of change This should be a living document
Should be kept and revised and compared with goals and outcomes of the program/s
Can be used as a reference tool
Should be transparent

16. As a planning and communication tool it can:
Using a Theory of Change
As a planning and communication tool it can:
Can provide opportunities to correct faulty underlying assumptions about the program’s design
Can help inform decisions about changes to the program/s as it evolves
Involve stakeholders in planning and helping them get clarity and consensus around program activities and outcomes

17. Using a Theory of Change
As an evaluation tool:
It can lay the foundation for the evaluation by:
Helping program staff identify outcomes that should be measured
Provide a map to inform your understanding of program outcomes

18. OUTCOMES

19. Outcomes ???
Outcomes are:
Changes in:
behavior
skills
knowledge
attitudes
condition
status
Outcomes should be:
Related to the business of the project
REALISTIC and ATTAINABLE
RELEVANT to the project
Within the program’s sphere of influence

20. are what a program is accountable for…
OUTCOMES….
are what a program is accountable for…

21. The group, condition, process targeted for change:
Grantee Outcomes…
The group, condition, process targeted for change:
Maya Language speakers in South Florida
Refugee DV offenders,
Restaurant workers in NYC
And what we hope to change about it
Improve communication and coordination of mental health services for the Mayan community
Increase community awareness about Domestic Violence in the refugee community
Develop restaurants workers knowledge about health resources

22. Are logical and reasonable if there
Outcomes
Are logical and reasonable if there
Is a logical connection between our activities and outcome
Activity: English Language Learners in Lincoln Nebraska will be given instruction on English language terms for anatomy, symptoms, interacting with health care providers
Potential Outcome that we can measure….

23. Outcomes…
Example:
Center for New Community will Improve services by health care providers to the Spanish speaking population of the twelve target communities
Possible measurable outcome:

24. Are logical and reasonable if it is…
Outcomes…
Are logical and reasonable if it is…
Reasonable to believe that the outcome can be accomplished within the timeframe that a program has.
Based on:
Program previous experience
Context
Resources

25. Access HealthColumbus
What do we mean by Outcomes?
Access HealthColumbus
Target Group:
Latino Immigrants
Hope to Change:
Overall Health
What are you striving for – Short Term?
Identify strategies to improve health
Develop models
Select models

26. What do we mean by Outcomes?
Long Term:
Create best practice in the field for working with this population
Replicate program in other counties in Ohio, In the Midwest…in the USA
Publish your findings

27. What is meant by improving strategies to improve health?
Questions to think about?
What is meant by improving strategies to improve health?
How will you know (measure) that it is the strategy that has improved health outcomes?
What are the health outcomes that you are looking at?
Baseline measures – Will you collect baseline information?

28. Georgia Department of Human Resources
What do we mean by Outcomes?
Georgia Department of Human Resources
Target Group:
Refugees in need of services in Georgia
Hope to Change:
Community Awareness and understanding of Domestic Violence
Give community greater access to services
What are you striving for – Short Term?
Support services to 105 partners and their families
Intervention for 80 refugee men
Increase awareness

29. What do we mean by Outcomes?
Long Term:
Create best practice for working with domestic violent offenders (or those thought to be at risk)
Create best practice for working with Refugee men of this community
Build a movement of men involved in preventing domestic violence
Create a network of local, regional and/or national organizations addressing this issue

30. What are the goals of the intervention?
Questions to think about…
What are the goals of the intervention?
How is the program defining community awareness?
How is the program measuring community awareness?

31. Who is our client/customer/ What does our client value?
The 5 Questions….
What is our mission?
Who is our client/customer/
patient?
What does our client value?
What is our plan?
What are our results?

32. Final Thoughts… There are no correct number of outcomes
Outcomes can go beyond the project’s formal conclusion
PLEASE be realistic about when to expect outcomes
Short, Intermediate, Longer term

33. Context (Contextual Analysis)
Assumptions
Goals/Strategy of the Program
Actions/Program Activities
Short Term Outcomes
Long Term Outcomes

34. Good Sources on Evaluation
Kellogg Foundation -
“Level Best – How Small and Grassroots Nonprofit can Tackle Evaluation and Talk Results” By Marcia Felsten and Marianne Philbin, 2007
The Drucker Foundation Self-Assessment Tool Process Guide and Workbook,
www. pfdf.org

35. Leticia Peguero, lpeguero@rwjf.org
Ayorkor Gaba,