1. A801 Section 8 October 31, 2012 Policy and Program Evaluation

2.  Situating Ourselves in the Course (1 minutes)  Logical Framework Problem Tree Analysis (10 mins)  Discussion of Weiss (10 mins)  Design an Evaluation (1hour)  Housekeeping & Questions (5 minutes) Agenda

3. 1 Introduction to Comparative and International Education 2 The Process of Policy Analysis 3 Education Policy Options Course Overview

4. 1 Introduction to Comparative and International Education 2 The Process of Policy Analysis 3 Education Policy Options Course Overview –Week 7 – Assessing the Performance of an Education System –Week 8 – Policy and Program Evaluation –Week 9 – Curriculum, Standards, and Assessment

5. “Your problem definition should not include an implicit solution introduced by semantic carelessness. Projected solutions must be evaluated empirically and not legitimated merely by definition. Therefore, keep the problem definition stripped down to a mere description, and leave open where you will look for solutions.” (Bardach, 2009, pg. 7) Defining the Solution into the “Problem”

6. “New schools are being built too slowly.” vs. “There are too many schoolchildren relative to the currently available classroom space.” Example

7. 1.List all the problems that come to mind. Problems need to be carefully identified: they should be existing problems, not possible, imagined or future ones. The problem is an existing negative situation, it is not the absence of a solution. 2.Identify a core problem (this may involve considerable trial and error before settling on one). 3.Determine which problems are “Causes” and which are “Effects.” 4.Arrange in hierarchy both Causes and Effects, i.e., how do the causes relate to each other - which leads to the other, etc. LogFrame Problem Analysis Tree

9. Summary Weiss - Evaluation

10. Design a Program Evaluation  Imagine that NFTE wants to conduct an alumni study in 2012-13.  Their goal with this study is to re-connect with our alumni in order to tell the story of the deep, long-term impact of their youth entrepreneurship programs.  They hope that this survey will help inform the field on the impact of youth entrepreneurship programs.

11.  NFTE has approximately 200,000 alumni in the U.S. They instituted formal systems for gathering alumni information only four years ago (2008).  Prior to that, they kept track of many of our program graduates on an ad hoc basis, largely via personal communications with founder, Steve Mariotti, and other NFTE staff members. NFTE Facts

12.  NFTE has basic data – name, address, NFTE program information, email in some cases – for only about 10,000 alumni, plus piecemeal data on another 3,000 or so.  They also have a set of alumni connected to NFTE through Facebook and LinkedIn, some of which are likely duplicative of the 13,000.  Their system contains lots of incomplete data, many duplicates to be cleaned, and much data that is likely obsolete, in the numerous cases where alumni have not updated their information since their initial registration. Alumni Database

13. Challenges of Tracking Alumni  It is very difficult to find alumni who completed NFTE prior to 2008 when NFTE set up the database.  Moreover our alumni population is relatively young and therefore naturally transient. They are unlikely to prioritize updating their contact information with NFTE each time they relocate.  Finally, NFTE has not devoted significant staffing resources to alumni tracking.  THINK OF WAYS IN WHICH NFTE COULD POTENTIALLY USE SOCIAL MEDIA TO RECONNECT WITH ALUMNI.

14. Design an Evaluation In teams design an evaluation that will answer the following questions:  Where are NFTE graduates now? What have they accomplished? How does their business ownership, employment, and education compare to that of their broad peer group? Keep in mind that not only does NFTE want to answer these questions, but they also want to inform the field’s general understanding of the impact of entrepreneurship education programs that target youths.

15. Design an Evaluation  Familiarize yourself with NFTE (vision, mission statement, where they operate).  www.nfte.com www.nfte.com  Design a survey tool that will help NFTE measure impact.  Please create a shared google doc for this exercise.  Invite veb682@mail.harvard.eduveb682@mail.harvard.edu  Think of existing datasets that NFTE could compare the results of their study to. They will use these comparisons to build a picture of our impact, into which we would integrate compelling individual stories.  Develop a timeline for NFTE. The data must be collected and analyzed before April 2013.

16. Activity timeline  1:20 – 2:05pm: Research + Survey Design + Strategy Design  2:05 – 2:20pm: Group debrief

17. Housekeeping Paper return date