1. Phillips Brooks House Association (PBHA)’s Summer Urban Program Overview: PBHA is a student-run non-profit with a dual mission of student development and community impact. PBHA youth programs build positive relationships with youth, families, and community partners to ensure that Boston and Cambridge youth ages 5-19 can build the social, emotional, and academic skills needed to define, access, and achieve their own success in and beyond our programs. Site info: 12 public school buildings in Boston and Cambridge Program Type: Mix of academic and enrichment Students Served: 920 youth, 5-18 year olds Staff: all sites combined have 130 FT seasonal college students, 10 FT year-round professionals, 80 FT seasonal high school students

2. Why Evaluation 423 Staff Applications 910 Youth 129,647 hours of programming …. But wait, there’s more! 5 Long-term Outcomes Academic, Support System, Future, Community Engagement, Self-Worth Staff Management Training, Coaching, Data Collection Processes

3. Objectives and Agenda Building an Inquiry Approach Evidence-Based Improvement Process – 1. SAYO – 2. Writing Rubric – 3. APT Observation Data Individual Program Data Debrief and next steps

7. Ex 1

8. Ex 2

9. (n=6) 1 = Strongly Disagree; 3 = Neutral; 5 = Strongly Agree Ex 3

10. 1 = Strongly Disagree; 3 = Neutral; 5 = Strongly Agree (n=6) (n=40) Ex 4

11. Validity Reliability: measurement tool design that is consistent each time it’s used Sampling: the questions relate well to the construct and representative group answered the questions Scaling: scores using scaled answers lead to more or less significant differences between two or more groups of results

14. What do you notice? (evidence) What do you wonder? (focus questions) What more data do you need? Identify a problem we control Plan highest leverage strategy Focus on Evidence and inquiry into data Program matic decisions and approach based on data

15. Case Study Benchmark average for each index = 3.0 (Mostly True) 2014

16. APT Averages by Domain (out of 4) n=42 (no Middle Up)

17. Data Transparency/Sharing Google Presentations Infoactive: – https://infoactive.co/plays/32998 https://infoactive.co/plays/32998 Tableau Public