1. Logic Models
How to Integrate Data Collection
into your Everyday Work

2. Agenda Introduction to Logic Models Developing a Basic Logic Model
Exercise One
Using a Logic Model to Plan for Evaluation
Exercise Two

3. Introduction to
Logic Models

4. Logic Models: An Overview
A systematic and visual way to present and share your understanding of the relationships among the resources you have to operate your program, the activities you plan, and the changes or results you hope to achieve.
Resources
Needed to accomplish activities (“Inputs”)
Activities
Process, tools, events, technology, and actions
Outputs
Results of activities (e.g., counts, types, levels of services delivered)
Outcomes
Short-term and long-term specific and observable changes (S.M.A.R.T)
Impact
Organizational, community, or system level changes
Measurement tool (indicator) required for all activities, outputs, outcomes, impacts

5. Resources Definition: inputs
Factors that enable resources:
Funding
Existing organizations
Potential collaborating partners
Existing organizational or interpersonal networks
Staff & volunteers
Time
Facilities
Equipment
Supplies
Factors that limit resources:
Negative attitudes
Lack of resources
Policies
Laws
Regulations
Geography

6. Activities Definition: Processes or actions to accomplish objectives
Products
Promoting materials
Providing educational curricula
Services
Educating and training
Counseling
Health screening
Infrastructure
Enhancing infrastructure
Building relationships, collaborating
Building capacity (for desired results)

7. Outputs Definition: direct products of activities
Number of classes taught, meetings held, or materials produced and distributed
Program participation rates and demography
Hours of each type of service provided

8. Outcomes Changes in: Attitudes Behaviors Knowledge Skills Status
Definition: observable and measurable changes in a specific population, community, or category (S.M.A.R.T)
Changes in:
Attitudes
Behaviors
Knowledge
Skills
Status
Level of functioning

9. Impacts Specific Measurable Attainable Realistic Timely
Definition: organizational, community, and/or system level changes expected to result from program activities. May include improved conditions, increased capacity, and/or changes in policy. Outcomes and impacts should be S.M.A.R.T.
Specific
Measurable
Attainable
Realistic
Timely

10. Logic Model Example Planning a family vacation Resources Activities
Holiday flight schedules
Family schedules
Frequent flyer holiday options
Holiday weather
Activities
Create family schedule
Get holiday flight info
Get tickets
Arrange ground transport
Outputs
Tickets for all family members
Frequent flyer miles used
Money saved
Outcomes
Family members enjoy vacation
Impact
Continued good family relations

11. Developing a Basic Logic Model for Your Program:
The Why

12. Why use a Logic Model? It is a useful planning tool
It helps you identify and collect the data needed to monitor and improve programming
It presents program information and progress towards goals in ways that inform, advocate for a particular program approach, and teach program stakeholders.
Quote from WKKF Handbook: “You can’t do ‘good’ evaluation if you have a poorly planned program.”

13. Criteria for Program Success Benefits of Program Logic Models
Benefits of a Logic Model
Program Elements
Criteria for Program Success
Benefits of Program Logic Models
Planning and Design
Program goals and objectives, and important side effects are well defined ahead of time.
Finds “gaps” in the theory or logic of a program and work to resolve them.
Program goals and objectives are both plausible and possible.
Builds a shared understanding of what the program is all about and how the parts work together.
Program Implementation and Management
Relevant, credible, and useful performance data can be obtained.
Focuses attention of management on the most important connections between action and results.
Evaluation, Communication, and Marketing
The intended users of the evaluation results have agreed on how they will use the information.
Provides a way to involve and engage stakeholders in the design, processes, and use of evaluation.

14. Exercise One
In groups, discuss your organization’s existing logic models and how they are utilized in evaluation.

15. Planning Evaluations with Logic Models:
The How

16. Types of Evaluations Formative Summative
Examines program service delivery or technology, the quality of its implementation, and assessment of organizational context, inputs
Focuses on program activities, outputs, and short-term outcomes in order to make mid-course corrections
Provides suggestions to strengthen program implementation
Summative
Examines effects (results) or outcomes of program
Focuses on program’s short and long-term outcomes and impact
Describes quality and effectiveness of program on participants and community

17. Program Implementation
First Determine What Will Be Evaluated
Program:
Context
Program Implementation (Process)
Outcomes (Results)
Program Implementation
Program Results
Resources
Activities
Outputs
Outcomes
Impact
Inputs
Processes or actions
Direct products of activities
Measurable changes for specific population, community
Organizational, community, system-level impacts
Measurement tool (indicator) required

18. Context Evaluation Questions
Intent of Evaluation: assess how a program functions within the economic, social, and political landscape of its community to determine factors that might influence ability to perform the work as planned
Examples
Can we secure a donated facility?
How many medical volunteers can we recruit?
What is it about the free clinic that supports its ability to reduce the numbers of patients seeking care in Memorial Hospital’s ER?

19. Implementation (Process) Evaluation Questions
Intent of Evaluation: assess the extent to which activities were executed as planned
Examples
How many patients were seen in the clinic each night, month, year?
Which organizations most frequently referred patients to the clinic?
What supplies were donated to the clinic?

20. Outcome Evaluation Questions
Intent of Evaluation: determine the extent to which progress is being made toward the desired changes (results) in individuals, organizations, communities or systems
Examples
What was the cost/visit in the free clinic? How does this compare to the cost/visit in Memorial’s ER?
How satisfied were clinic patients with the care they received?
Did the reading program improve the reading scores of the youth who participated in the program?

21. Evaluation Questions and Planning
Focus Area: From your program logic model, list the components of the most important aspects of your program.
Audience: Identify the key audiences for each focus area. Who has an interest in your program?
Questions: For each focus area and audience, list the questions they may have about your program.
Information Use: For each audience and question you have identified, identify the ways you will use the evaluation information.
Indicators: Describe what information could be collected that would indicate the status of your program and its participants for each question.
Technical Assistance: Indicate the extent to which your organization has the evaluation and data management expertise to collect and analyze the data that relates to this indicator.

22. Focusing Your Evaluation Questions
Evaluation Focus Area
Audience
Question
Information Use
Determine what will be evaluated
For each focus area identified, list the audiences that are most likely to be the most interested in that area
For each focus area and audience identified, list the questions they might have about your program
For each audience and questions identified, list the ways and extent to which you plan to use the evaluation information

23. Audience Questions and Evaluation Use
Typical Questions
Evaluation Use
Program Management and Staff
Are we reaching our target population?
Are our participants satisfied with our program?
Is the program being run efficiently?
How can we improve our program?
Programming decisions, day-to-day operations
Participants
Did the program help me and people like me?
What would improve the program next time?
Decisions about continuing participation
Community Members
Is the program suited to our community needs?
What is the program really accomplishing?
Decisions about participation and support
Public Officials
Who is the program serving?
What difference has the program made?
Is the program reaching its target population?
What do participants think about the program?
Is the program worth the cost?
Decisions about commitment and support.
Knowledge about the utility and feasibility of the program approach.
Funders
Is what was promised being achieved?
Is the program working?
Accountability and improvement of future grantmaking efforts

24. Establishing Indicators
Evaluation Focus Area
Indicators (examples)
How to Evaluate
Resources
Logs or reports of financial/staffing status.
Compare actual resources acquired against anticipated resources.
Activities
Description of planned activities. Logs or reports of actual activities. Descriptions of participants.
Compare actual activities provided and/or types of participants reached against what was proposed.
Outputs
Logs or reports of actual activities. Actual products delivered.
Compare the quantity of what was actually delivered against what was expected.
Outcomes & Impacts
Participant attitudes, knowledge, skills, intentions, and/or behaviors intended to result from your activities
Compare the measures before and after the program.

25. Exercise Two
Discuss the sample logic model and its similarity to the logic model developed for your organization.
Discuss any challenges you faced – or may possibly face – in developing a logic model like the sample.