1. Healthy Futures Performance Measures Session

2. Session Overview Combination of presentation and interactive components Time at the end of the session for Q&A Use notecards on table to write questions

3. Objectives Strengthen understanding of the three elements integral to a strong performance measurement plan: –Data Collection –Data Management –Data Reporting Experience planning for the data collection process

4. Agenda

5. How does my program demonstrate impact? What evidence do I need to collect to show I achieved my selected outcome?

6. 6 Amount of service provided (i.e. people served). Reflect the changes or benefits that occur. How does my program demonstrate impact?

7. Work Plan Refresher The Service Activity Description should explain what the Senior Corps volunteers are doing in a way that shows how they will achieve an output and outcome. It should include: who the beneficiaries are; what the volunteers will be doing with the beneficiaries; how often volunteers will provide the service; for how long, and; where the service will take place.

8. Work Plan Refresher 50 Senior volunteers will provide respite services to 150 primary family caregivers of older, homebound adults. They will serve a minimum of 8 hours per week for at least 12 weeks. They will serve each client a minimum of 2 hours per week for 12 weeks. Service activities may include companionship/friendly visiting and reminiscing, light meal preparation, letter writing/correspondence, going for short walks, and reading stories together.

9. Work Plan Refresher The Instrument Description should: give the name of the instrument; briefly describe who will collect the data; from whom the data will be collected, and; when it will be collected.

10. Work Plan Refresher Station supervisors will conduct the Respite Care Survey with caregivers every 6 months at the middle and end of the program year.

11. Five Data Quality Elements A Standard Process to Consider Data Validity Completeness Consistency Accuracy Verifiability

12. Validity: Data Relevance Ask yourself: –Is the data relevant? –Are your reported items consistent with the approved goals of the current grant? –Are you measuring what you intended to measure? Example H14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support. Data Instrument: Survey Questions in Survey: Did you have a long wait for a volunteer? Are you happy with the service?

13. Data Completeness Ask yourself: –Does the reported data contain enough information to represent performance measure activities? –Did the reported data come from all stations and/or a random sampling of volunteers/ service recipients? Example H14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support. Data Instrument: Survey Survey Distribution: Jane provides the survey to 7 of 12 stations because she is running behind on her schedule

14. Consistency: Standard Protocols and Procedures Ask yourself: –Do you clearly document data collection methods? –Do you use the same data collection method for all sites? –Do you train data collectors? –Have you taken steps to eliminate bias? Example H14 Outcome: Number of caregivers of homebound OR older adults and individuals with disabilities who reported having increased social ties/perceived social support. Data Instrument: Survey Survey Distribution: Jane provides the same survey to all stations at the same time

15. Accuracy: Data Free From Error Ask yourself: –Do you have a plan to collect and review data? –Do you have documentation on your data collection plan? –Does the data vary significantly in unexpected ways? Example Data Collection Approach: Collect data from all 12 stations Data Received: 150 surveys Data Check: Jane’s colleague reviews the data to check her calculations, as per her organization's Data Collection Handbook

16. Verifiability: Data Collected According to Plan Ask yourself: –Did the grantee double check that the data collected followed the plans and procedures? –Did the grantee use controls throughout data collection? –Does the grantee regularly review and update its data quality control procedures? Example Situation: Jane submits her PPR and leaves for vacation; her colleagues cannot validate the data she input into it.

17. Doing Data Collection: Identifying Your Source List pros and cons of possible data sources Identify a preferred data source Consider back-ups

18. Doing Data Collection: Considering Methods Survey Interviews Observation Standardized tests Tracking sheets Focus groups Diaries or journals Secondary data

19. Method Requirements Sample questions are provided for the required survey

20. Doing Data Collection: Creating a Schedule Identifies who will collect data, using which instrument, and when –Include stakeholders in planning –Include time for collecting, analyzing, and reporting data Share with team to keep everyone informed

21. Doing Data Collection: Selecting a Final Measurement Instrument For CNCS priorities and performance measures, look for instruments by program –Go to http://www.nationalservice.gov/resources/performance- measurement/senior-corps http://www.nationalservice.gov/resources/performance- measurement/senior-corps –Programs and projects can also look anywhere they like to find instruments: Use Internet search engines Leverage experience and known best practices in your field Talk to others within you professional network to find out what they are using

22. Doing Data Collection: Selecting a Final Measurement Instrument You may have a pre-existing tool or want to modify a resource found on the Knowledge Network Consider: How else might you use this data? What may other stakeholders want? How will this affect your data collection and instrument design? Keep in mind the 5 data quality elements when developing or modifying a measurement instrument.

23. Learning Activity

24. Data Management Now what? Consider: –Potential usage –Access –Storage and maintenance

25. Data Management Data collection procedures and guidelines Source documents for data (e.g. sign-in sheets, activity logs, attendance logs, client tracking database, tracking forms, surveys, etc.) Formalize data sharing agreements in MOUs

26. Data Reporting

28. Q&A

29. Thank you Stay current by visiting the Knowledge Network! http://www.nationalservice.gov/resources/performanc e-measurement/senior-corps http://www.nationalservice.gov/resources/performanc e-measurement/senior-corps