1. Evaluation Basics for Youth Programs Ben Silliman, Ph.D. Dept. of 4-H Youth Development

2. Reasons to Love Program Evaluation Personal: feedback on personal effectiveness Practical: information for improving delivery Professional: process and outcomes data to show accountability, impact

3. Understanding Evaluation Name the best thing(s) you’ve seen/experienced in a youth-at-risk program Name the worst thing(s) you’ve seen/experienced in a youth-at-risk program

4. Dimensions of Evaluation Good things come from… A clear vision of what is happening A clear vision of what is happening A positive climate for youth A positive climate for youth Programs that make a difference Programs that make a difference Organizations that run effective programs Organizations that run effective programs

5. Dimensions of Evaluation Good things come from research-based… Environmental scanning Environmental scanning Program evaluation Program evaluation Outcome evaluation Outcome evaluation Organizational/ Organizational/ professional evaluation professional evaluation

6. Environmental Scanning What do people need? What resources do people have?

7. Needs and Asset Mapping Demographic Data Community Economic and Social Surveys Focus Groups Targeted Needs Assessments Community Coalition consensus

8. Program Evaluation How good is the program?

9. Positive Climates/Best Practices Safe Environments Safe Environments Structure Appropriate for Structure Appropriate for Supportive Relationships Supportive Relationships Social Inclusion Social Inclusion Standards of Conduct Standards of Conduct Skill Learning Focused Skill Learning Focused Contribution-Oriented Contribution-Oriented Connections to Family, Community Connections to Family, Community

10. Program Climate Physical and Psychological Safety oSafe, healthy facilities oConflict resolution oWellness habits

11. Program Climate Appropriate Structure oClear consistent boundaries oAge-based opportunities oEffective monitoring

12. Program Climate Supportive Relationships oWarmth, affirmation oCommunication oPositive guidance

13. Program Climate Opportunities to Belong oAcceptance, inclusion oCultural diversity/ tolerance oTeamwork

14. Program Climate Positive Social Norms oHigh expectations for ethics, behavior oHigh achievement orientation oService orientation

15. Program Climate Efficacy and Mattering oGrowth in confidence and autonomy oAbility to make a difference oMeaningful changes effected with individuals, organizations, communities

16. Program Climate Opportunities for Skill Building oMastery of practical skills in many domains oAbility to relate and navigate varied cultures oImprovements in vocation- related skills

17. Program Climate Integration with Family, School, and Community oParent involvement and family activities at home oIncreased academic performance and school leadership oYouth-adult partnership

18. Outcome Evaluation What’s happening with the kids?

19. Logic Model: Building a Game Plan The Big Picture: Winning Legacy Giant Steps: A Habit of Playing Well Small Steps: Mastering the Fundamentals The Right Stuff: Good Preparation Setting the Table: Good Resources Setting the Table: Good Resources

20. The Logic of Evaluation InputsInvestments of resources of resourcesMoneyStaffVolunteersEquipmentSupplies Effects of constraints constraintsLawsRegulations Funder requirements Outputs Products of programs Classes taught Materials distributed Hours of service Participants served

21. The Logic of Evaluation Impacts Long-term outcomes Behavioral change Behavioral change Social-economic change Social-economic change Cultural change Cultural changeOutcomes Short-term outcomes Knowledge KnowledgeAttitude Skills Skills Aspirations Aspirations

22. Evaluating Outcomes in Small Steps Knowledge Knowledge Attitudes Attitudes Skills Skills Aspirations Aspirations

23. Outcome Domains Physical  Health  Safety  Wellness  Risk Reduction

24. Outcome Domains Intellectual/Cognitive  Critical thinking/ problem solving problem solving  Academic achievement  Project skills/mastery  Decision-making  Creative thinking

25. Outcome Domains Emotional  Coping  Stress management  Optimism  Self-efficacy  Character

26. Organizational/Professional Evaluation How (well) do leaders make things happen?

27. Organizational Processes Serving People Designing and Producing Better Goods and Services Building an Efficient and Effective Workforce

28. Six Sigma Processes Define the project goals and customer (internal and external) requirements Measure and determine customer needs and specifications; benchmark competitors and industry Analyze the process options to meet the customer needs Design (detailed) the process to meet the customer needs Verify the design performance and ability to meet customer needs

29. Professional Evaluation Professional Standards Novice-to-Expert Rubrics and Certification for General & Specialized Competencies Individual and Team goals, performance standards, and professional development Continuous quality improvement

30. Human Subjects Satisfaction, description is program feedback Individual performance change is consent-required feedback

31. Replicating and Sustaining Programs  Stage 1: Program running as planned, having intended effect (Process Evaluation/Formative Evaluation/Quality Control)  Stage 2: Change in knowledge, attitude, skill, or aspiration (Outcome Evaluation/Summative Evaluation/Achievements & Impact)