Evaluation Basics Principles of Evaluation Keeping in mind the basic principles for program and evaluation success, leaders of youth programs can begin learning and using evaluation practices…

Evaluation Basics Practices of Evaluation Practices of evaluation include tools and procedures that help leaders… implement evaluation processes accomplish evaluation purposes recognize evaluation principles Practices are the “nuts and bolts” of doing evaluation

Evaluation Basics Practices of Evaluation Sample practices for each process step include… Planning: SMART Objectives Focusing: Evaluability/Standards Checklist Design: Timetable Methods: Survey Template, Observation rubric Data Collection: Excel Spreadsheet 1 Analysis and Interpretation: Excel Spreadsheet 2 Communicating Results: Reporting Template

Evaluation Basics Practices of Evaluation Planning: The SMART acronym describes steps to turn a general goal into a detailed roadmap for change. SMART objectives are… Specific about the type, amount, timing of change Measurable in setting criteria for observing change Achievable, recognizing program limits, possibilities Relevant to larger goals and purposes Time-bound in the program scope and sequence

Evaluation Basics Practices of Evaluation Focusing: An Evaluability Checklist helps leaders decide whether and how to evaluate their program. Two key evaluation processes are… Formative, assessing the operation of the program Summative, determining the results of the program

Evaluation Basics Practices of Evaluation Focusing: Formative Evaluability Indicators describe elements to check to see if the program is on track. Key elements of program operation include… Participants, assessing the operation of the program Staff determining the results of the program Resources, materials and equipment identified, if not procured Evaluation, plan and procedures in place

Evaluation Basics Practices of Evaluation Focusing: Evaluation Standards help program leaders determine evaluability, or when the program is ready to evaluate. Evaluation Standards include… Utility, usefulness of the evaluation to stakeholders Feasibility, potential for effectiveness and efficiency Propriety, potential for fair and ethical assessment Accuracy, dependability of findings for a setting

Evaluation Basics Practices of Evaluation Designing: A Timeline enables program leaders to plan and manage each stage of evaluation. Key elements of program operation include… Time 1Time 2Time 3Time 4 Task 1 Task 2 Task 3

Evaluation Basics Practices of Evaluation Selecting Methods: Adaptable data collection tools that, enable program leaders to gather fixed-choice and open-ended data from program participants and/or observers include… Survey, which asks questions about program objectives and experiences; and Observation Rubric, which provides a framework for judging conditions or performance

Evaluation Basics Practices of Evaluation Data Collection: Simple spreadsheets provide excellent ways to organize and store data as follows: Quantitative (numerical) data typically documents participant data by row, each question category by column; and Qualitative (verbal) comments can be entered one- per-row, usually without respondent characteristics

Evaluation Basics Practices of Evaluation Analysis and Interpretation: Most spreadsheets can be used to make simple counts or comparisons, or to organize responses by category: Quantitative (numerical) data can be added or averaged by column, with items totaled by frequency (count) or percentage (ratio); and Qualitative (verbal) comments can be grouped by theme, set either before or after data collection

Evaluation Basics Practices of Evaluation Communicating Results: Although different stakeholders or persons interested in a program, appreciate different descriptions, all persons benefit from understanding: Summary Information: program title, purpose, audience, timing and setting; evaluation findings simply stated, followed by conclusions, implications, and credits for ideas and funding.

Evaluation Basics Performance in Evaluation Glossary Evaluation as PYD Evaluation Program Development Taking Stock Implementing Evaluating (Valued) Outcomes (Valued) Processes Organizational Mission Assessed Needs YD Outcomes: Caring, Competence, Confidence, Contributing, Character Eval. Process Steps Planning Focusing Designing Selecting Methods Data Collecting Analyzing & Interp. Communic. Results

Evaluation Basics Performance in Evaluation Glossary Evaluation Purposes Show accountability Grow capacity Know how to improve programs Bestow relationships

Evaluation Basics Performance in Evaluation Glossary Evaluation Principles Develop programs with strength and focus Intensity Frequency Duration Evidence-based Research-based Program fidelity Readiness for change Stages of change Develop programs for long-term impact Programs activities Develop & evaluate in stages (maturation) Start-up/pilot Functioning/Established Mature Formative evaluation Summative evaluation

Evaluation Basics Performance in Evaluation Glossary Evaluation Practices Planning/SMART obj Focusing/Evaluability Design/Timetable Methods/Survey & Observation Rubric Fixed choice Open-ended Data Collection/Spreadsheet Quantitative Qualitative Analysis & Interpretation/Spreads heet Communicating Results/Template: program title, purpose, audience, timing and setting; evaluation findings simply stated, followed by conclusions, implications, and credits for ideas and funding.