Presentation on theme: "Tips and Tools for Measuring Program Impacts: From Delivering Programs to Reporting Impacts Office of Educational Innovation and Evaluation 2323 Anderson."— Presentation transcript:
Tips and Tools for Measuring Program Impacts: From Delivering Programs to Reporting Impacts Office of Educational Innovation and Evaluation 2323 Anderson Avenue, Suite 220 Kansas State University Manhattan, Kansas 66502 Phone: 785-532-5930 Fax: 785-532-7185
Tips Find and review evaluation instruments Utility and feasibility Purposes of evaluation: Accountability, Improvement, Knowledge Development, Oversight and Compliance Who wants to know what about your program? Multiple, complementary sources of data
Tools Rubric: Reviewing Evaluation Instruments LowMediumHighNotes UTILITY Value to PFT Consider: How well will the evaluation instrument gather data that aligns with the PFT action plan? Alignment with Program Curriculum Consider: How well aligned is the evaluation instrument with the curriculum that was taught?
Part II: Collecting and Managing Data to Measure Impacts
Tips Plan early – determine when evaluation will take place in the programming cycle Review and manage data as you receive it Analysis quantitative and qualitative by hand, using Excel Who wants to know what about your program?
Tools Data Analysis Plan Worksheet Question or Item Number What did I hope to learn from this question? What analysis technique would assist in that learning? How could I present the data to convey what I learned? E.g., #2 If participants felt that they increased their knowledge of best management practices for a healthy yard. Count and percentage of respondents who reported Agree or Strongly Agree Bar or pie graph that compares Agree and Strongly Agree responses to Disagree, Strongly Disagree and Unsure responses
Tips KSRE Impact Reports Who wants to know what about your program? Illustrate your data Be clear and concise, avoid jargon Use the active voice
Tips Choose Graphics to Effectively Convey Data Region Education (in Years of Schooling) 89-111213-1516 Northeast25.3225.2918.2618.2916.34 North Central32.0929.0418.7824.3219.02 South38.8131.0219.3315.6616.79 West25.3721.0920.2923.9717.52 Example #1 – Table NOT formatted effectively Table 1. Chewing Gum Rates in the United States by Geographic Region and Level of Education* *Education level based on number of years of schooling. Adapted from a report of the U.S. Department of Chewing Gum. Education Level Region NortheastWestSouth North Central Average 825 393230 9-112521312927 12182019 13-151824162421 16 18171917 Average2022242523 Example #2 – Table IS formatted effectively Table 2. Chewing Gum Rates in the United States by Geographic Region and Level of Education*
Tools What Does the Data Mean? What are the key findings? Write a paragraph to summarize the results. What do you find interesting about these results? What would your audience find interesting? E.g., After participating in the program, many participants reported feelings of increased financial abilities. In particular, a majority of participants reported increased skills to track spending (64%), to create a spending plan (71%), and to plan for unexpected expenses (77%). One respondent commented, The skills I learned from this program will help me stretch my money to the next payday.
Contact Information Office of Educational Innovation and Evaluation Kansas State University 2323 Anderson Avenue, Suite 220 Manhattan, KS 66502 Phone: 785-532-5930 www.ksu.edu/oeie Sarah Bradford (785) 532-5677 firstname.lastname@example.org Amy Hilgendorf (785) 532-5538 email@example.com