Focusing Your Evaluation Activities Chapter Four cont…

Review  Needs assessment  Formative evaluation  Implementation evaluation  Process evaluation  Outcome evaluation  Cost analysis

Outcome Evaluation  Is the program effective?  What are effects of participation?  What are negative effects?  Do clients benefit or improve?  Are clients any better than they would have been?

Outcome Evaluation cont…  Intended outcome is positive change  Or maintaining functional ability  Or prevention programs  Consider total outcomes both pos and neg  Outcome eval in a sequential manner

Timing  Most programs need time to refine  Staffing and program design will change  Revision is a normal part of a new program  Outcome eval is done when stability is present  Outcome eval after one year of operation

Credibility  Outcome eval in a self report format  Memory loss about the program  Reporting bias on forms  Personal testimonials may be good for reports  They are not seen as highly credible  Assessment should be prior and after exercise in order to assess program impact

Credibility cont…  Administer multiple measures of client outcome  Compare effectiveness of different programs  Run two programs and measure each  Compare results at conclusion

Sequential Approach  Look at individual programs first  Look at short term first then long term  Use new clients for assessment as old ones may have already changed  Tracking is key to assessment  Determine “WHY” program is not working  Make modifications to the program

Cost Analysis  Program efficiency  What are costs of overhead  What is the cost per case  What is cost to produce outcomes  Capital expenses are equipment at beginning  Operating expenses are salaries and custodial

Cost Analysis cont…  Advertisement cost analysis  Cost per case is total cost/membership #  Cost-outcome is direct benefits to exercisers  Cost required to produce outcomes?

Program Goals and Objectives  What does your program want to accomplish  Goals are larger in scope  Objectives are individual items to achieve goals  Table 4.2 page 55  Your rationale or mission statement is a program goal  Implementation objectives are proposed events

Goals and Objectives cont…  Service objectives are what is happening to members  Output objectives are what you want to accomplish  Outcome objectives are the most important  Outcomes are inputs vs results  Do people benefit from the program?  Positive results are outcomes

Performance Indicators  Results are the bottom line  Client satisfaction is not an outcome  What will you accomplish in the future?  Targets for performance  Weight loss  Fitness improvement  Return to work

Performance Indicators cont…  Future targets are based on past performance  Other programs can serve as benchmarks  Tracking is the best way to set targets  Don’t set too low or too high  Success is relative to objectives

Program Logic Model  A diagram showing objectives and performance indicators and linkages  Figure 4.2 page 58

Lab 2  Visit a fitness center:  Objectives  Outcomes  Cost