1. Introduction to the User’s Guide for Evaluating Learning Outcomes from Citizen Science Tina Phillips, Cornell Lab of Ornithology Marion Ferguson, Cornell Lab of Ornithology

2. DEVISE Assessed the state of evaluation in citizen science Determined common goals, objectives, and indicators across projects Inventoried existing instruments; developed/modified new and existing evaluation tools Provide professional development opportunities Build community of practice for evaluations of citizen science projects DEVISE instruments can be found at: CitizenScience.org/Evaluation DEVISE instruments can be found at: CitizenScience.org/Evaluation

3. USER’S GUIDE FOR EVALUATING LEARNING OUTCOMES FROM CITIZEN SCIENCE Professional Development, as part of DEVISE Multidisciplinary Practical overview of evaluation techniques, tips, and best-practices Templates and worksheets to help with planning and implementation

4. FRAMEWORK FOR EVALUATING CITIZEN SCIENCE LEARNING OUTCOMES

5. Who is it for? The guide is designed to be used by citizen science practitioners who want to evaluate outcomes from their citizen science projects Since it has been released, we have seen it downloaded by people in a wide variety of roles throughout the field of citizen science

6. * n = number of downloads as of 11/4/14

7. Who uses it? - Zoos and Aquariums - Monitoring Programs - Science Museums - Colleges and Universities - Consulting Firms - Conservation Organizations - Educational Organizations - Public Schools - Government Agencies - Wildlife Societies - Citizen Science Projects - Environmental Service Groups - Non-profit Organizations - Individuals

8. How is it used? Start-to-Finish: plan, implement, and share evaluation Supplementary: utilize sections and resources to augment or build up an evaluation plan already in development Fine-Tune: use templates and resources provided to ensure that an existing evaluation plan is properly constructed and executed

10. PLAN: INVENTORY Gives guidance on documenting information relevant to the project: Background and context Goals Targeted outcomes Develop logic model

11. PLAN: DEFINE DEFINE SCOPE OF WORK Determine what will be evaluated Identify key questions Determine indicators for each outcome Lay out timeline, budget, and limitations Users together with stakeholders will:

12. PLAN: DESIGN Guides users in designing a data collection strategy that identifies procedures that are: Feasible Cost-effective Viable Determine best method for project needs with overview of strengths/weaknesses of evaluation designs

14. IMPLEMENT: DEVELOP Collect credible data that will increase the evaluation’s accuracy and utility Choose the right instrument Use this section with Appendices F and G

15. IMPLEMENT: FIELD TEST Field test instruments and refine as needed based on test results -Suitability -Utility -Clarity Create a data management plan

16. IMPLEMENT: ADMINISTER Recruit participants for evaluation: -Determine best medium for collecting data -Communicate purpose and risks/benefits to participants -IRB Administer final instruments and collect data

18. SHARE: ANALYZE Includes a sample codebook, gives an overview of the standard practices for analyzing each of the following types of data: Quantitative Qualitative Mixed Methods

19. SHARE: REPORT Overview of evaluation report components: Evaluation Report Executive Summary Program Description Evaluation Methodology Findings and Results Discussions and Interpretations RecommendationsReferencesAppendices

20. SHARE: DISSEMINATE & INFORM Includes strategies for: Sharing findings with stakeholders Publishing Additional ways to disseminate work Using evaluation to inform program

21. IMPLEMENT: DEVELOP, TEST, ADMINISTER Appendices and Resources Nearly half of the guide is made up of useful appendices and resources, including templates, worksheets, and sample evaluation planning

26. The User’s Guide for Evaluating Learning Outcomes from Citizen Science is available for download here: CitizenScience.org/Evaluation Interest in Science and Nature Nature Relatedness Scale Self-Efficacy for Science Self-Efficacy for Environmental Action Motivation for Science Motivation for Environmental Action Evaluation instruments currently available: