1. + STARS Evaluation Assistant Webinar 1 September 19, 2014 Evaluation Projects

2. + Part 1: What Evaluation Assistants Do

3. + What is Evaluation?

4. + How do we tell our story?

5. + EA Role Team Lead for Local Evaluation: 1. Activity Reports (1 per activity) Story Teller of Your STARS Computing Corps 2. Narrative Reports (April 15) Applied Researcher 3. Evaluation Project and poster at Celebration 2015 Submit Activity Reports Online ongoing Participate in meetings and webinars monthly EA Toolkit Support Resources What you doHow you do it Demonstration

6. + Recap of EA Deliverables Activity Reports Consent Forms & Registration Fall Synopsis of Activity Reports by Thanksgiving EA Project Milestones: Logic Model (Oct 3) Timeline (Oct 24) Activity Reports Narrative Report (April 15) Evaluation Poster (May 1) FALL 2014Spring 2015

7. + Part 2:Understanding Evaluation Evaluation Projects

8. + Evaluation is…. Evaluation is the systematic assessment of the worth or merit of some object (Trochim) Applied social science research A way of viewing the project, its participants, it effect Accounting for CONTEXT and MEANING Determining the VALUE of something Measurements are the currency

9. + Understanding Evaluation 1.Systematic ally define what data will be collected, how, when & by whom 2.Process the data to determine what it means within specified contexts 3.Tell constituents about findings in appealing & appropriate ways 4.Disseminat e broader findings to share within disciplines

10. + Evaluative Thinking Reflective practice of systematically using data to inform decisions Ongoing process Approach to making decisions, what actions to take, to become/remain effective at the organizational (or individual) goals It is LEARNING and APPLYING learning

11. + Evaluation and Academic Research EvaluationAcademic Research Applies research methods to answer questions about a program’s efficacy & value Considers context (stakeholders, climate, etc.) Makes a judgment call on WORTH, and defines value structure (i.e. value to whom?) Applies research methods to answer a research question Knowledge gained in scientific discipline Does not apply value structure; values are seen as bias

12. + In summary, evaluation… Uses research methods Applies methods to social, political and economic realities Helps to interpret academic research within larger socio- economic/geo-political realities Answers the “so what?” of observed outcomes Research = Knowledge Discovery Evaluation = Extracting Meaning from Knowledge

13. ContextInputsOutputsOutcomes/ImpactGoals Computing enrollment & graduation is in a state of decline nationally K12 students lack appropriate background to pursue computing Computing pipeline lacks diversity NSF Funding for 50 Alliance members K12 Partnerships STARS Leadership Corp Faculty Associates Community & Industry Partners K16 Outreach Celebration Demonstration Projects Social media Online Repository & Toolkit Dissemination Evaluation K16: Develop, test, deploy academic interventions; Pedagogical reform Increased post- secondary enrollment & graduation Increased scholarship & research Increased dissemination of best practices & national adoption of Alliance Community of Practice Increased BPC partnerships & advocacy K16 Recruitment Graduate School Bridging Post-secondary Student Retention Scholarship Advancement Alliance Sustainability Dissemination Scaling Formative assessment: SLC and K16 Outreach, Partnerships, Tools, Celebration Community of Practice & National Resource: Lessons Learned, Best Practices, Social Network, Resources & Evaluation Repository/Toolkit, Research, Summative Evaluation STARS Alliance Logic Model

14. ContextInputs Outcomes/Impact Goals CHALLENGES PROBLEM TO SOLVE ACTIVITIES RESOURCES Short & Long Term Accomplishments E.G. Tools/relationships built, developed Numbers served Attitudes Actions What do you hope to accomplish and how will you know? Logic Model Framework for Computing Corps Due Oct. 3

15. Overview of EA Projects Evaluation project that the EA manages Can do alone or manage a team of Corps students at your school A distinctive measurement of the impact of a Corp project at your school Example- measure impact of high school outreach program on high school student interest in computing Example- assess mentoring program at your school

16. + EA Projects Are NOT a description of what the Corp is doing or has done Are a summation of data collected, observed, documented about the impact of what the Corp has done at a school

17. Components of EA Projects Determine which Corp program to assess at your school Align project to the program goals E.g. if your Corp conducts high school outreach, what are the goals? Determine which goal(s) to measure and what data to collect Complete a project description and timeline (see www.starsalliance.org EA Toolkit for samples)www.starsalliance.org Assess the program and report findings via Celebration Poster Session

18. Get Support Corps Project Teams Share ideas and tools within your STARS Computing Corp Ask prior EAs at your school for input, ideas, suggestions Visit the EA Toolkit Sample IRBs, sample outreach surveys and more Discuss with EA community Online or via monthly meetings

19. + Q&A Evaluation Projects