1.  American Evaluation Association San Antonio, Texas - November 2010 Office of Strategic and Performance Planning, Bureau of Resource Management

2. 2  The Department operates more than 270 embassies, consulates and posts worldwide  The Department has 39 functional, management and regional bureaus  The State Department and USAID share a Joint Strategic Plan and seven joint strategic goals, from which they develop their strategic priorities

3.  Diplomacy and development function in an integrated fashion  State and USAID Joint Goal Framework  Integrated nature of work of State, USAID, and Defense o Diplomatic, economic, military, political, legal, and cultural o Embodies Secretary Clinton’s Smart Power approach 3

4.  Reconstruction and Stabilization  Global Challenge of HIV/AIDS and Investment in President’s Emergency Plan for AIDS Relief  Energy Security and Combating Threat of Climate Change  Public Diplomacy  Consular Affairs – a backbone of State Department since the beginning  Counterterrorism and Border Security  Advancing Rule of Law and Human Rights  Trafficking in Women and Girls  Overseas Building Operations 4

5.  Congress  Office of Management and Budget (OMB)  Government Accountability Office (GAO)  Office of the Inspector General (OIG)  Monitoring and Evaluation units within bureaus 5

6.  Office of Strategic and Performance Planning  Office of State Programs, Operations, and Budget  Office of the Director of U.S. Foreign Assistance  Functional/Management Bureaus and Regional Bureaus 6

7.  To create a “culture for evaluation” where evaluation information is used to learn how our programs are performing (real world results), to inform policy and resource decisions, and to improve program management  To develop a comprehensive agency evaluation plan and strategy to promote and sustain evaluation as a management tool  To increase the quantity and quality of evaluations/assessments conducted by bureaus and posts 7

8.  One size does not fit all in program evaluation, particularly given the different programs and projects managed worldwide by State and jointly by State and USAID  Diplomacy’s impact is often difficult to measure  Availability of resources (funding, staff) has proven a challenge 8

9.  Program evaluation is essential both for prospective planning of programs & for retrospective assessment of effectiveness  Office of Management and Budget (OMB) emphasizes the use of evaluation information to support major budgetary decisions about program reform and/or expansion  Having a comprehensive evaluation capability allows agencies to share information about effective practices across government programs 9

10. 10  The Office of Management and Budget (OMB) asked the State Department to prepare an inventory of planned, ongoing and recently conducted evaluations  Information was requested for all evaluations conducted by outside contractors funded at any amount, and for evaluations conducted internally if the cost was $500,000 or more  In Fiscal Year 2010, functional bureaus reported a total of 283 evaluations, with approximately $85 million dollars obligated or expected to be obligated  Regional bureaus reported 31 active evaluations

11. 11 Plan & Set Goals Collect Data & Analyze Results Use Data for Decision- Making Communicate Results Bureau Strategic and Resource Plans Customer Service Surveys; Randomized Control Trials; Case Studies Inform Bureau & Mission strategic priorities, improve program management, inform resource allocation decisions Bureau Websites; SMART Database

12. 12  $70 million in grants annually to organizations to address human rights issues in the areas of civil society, media, women’s rights, etc.  Grantees required to provide quarterly reports and comprehensive M&E plan  DRL commissions independent evaluations to examine cumulative results of DRL programs within a particular geographic/thematic area, e.g., Europe Media Programs  DRL uses reports, semi-annual reviews, and evaluation information to adjust programs to ensure they meet intended objectives and to target future programmatic efforts

13. 13  Used random sampling to survey supervisors and non- supervisors (e.g., team leaders) on (a.) level of awareness of multi-staged HR delivery model and (b.) degree of satisfaction with each tier  Survey conducted Summer 2010  Approximately 25,000 responses from non- supervisors and 4600 responses from supervisors  Multi-stage HR Delivery Model: Self-service; HR Service Center (Charleston, SC); HR Service Providers; Corporate HR  Analysis ongoing; HR interested to see variances in awareness and satisfaction overseas and in DC

14. 14  Provide M&E training to all new domestic and overseas officers to identify and raise to bureau’s attention relevant evaluation questions for the officer’s specific portfolios  M&E officers work directly with program officers to build evaluation into regional-specific (e.g., resettlement of Iraqi refugees) and project specific RFPs  Combination of domestic and overseas evaluations provides flexibility in learning what works; approach useful for program management and higher-level decision-making

15. 15 Sample of External Evaluation: Burundi Refugee Integration Project Research Question:How did the combination of assistance in the health sector and reintegration assistance impact the safe return and reintegration of refugees? Time Period:October 2002 to September 2007 Results:Confirmation that reintegration programs need to be continued for several years beyond actual return of refugees Use of Evaluation Results:Policy advocacy with the UN High Commissioner for Refugees and other governments; to inform bureau deliberations on funding allocations for populations affected in other parts of the world.

16.  New Evaluation Policy o Building a cadre of experts to provide technical assistance in areas such as M&E systems design, project management, survey design, SOW prep, methods, etc. o Implementation of bureau evaluation plans tying evaluation to bureau- level strategic planning goal setting, and performance indicators o Development of Indefinite Quality Contract o Reconstituted Evaluation Working Group  Participation in OMB evaluation initiatives, including seeking funding for high-priority evaluations o Annual documentation of ongoing and planned evaluations 16

17.  Launch of Community of Practice Website  Annual Conference on Program Evaluation  Program Evaluation Discussion Series  Diplopedia site for Program evaluation  Creation of guidelines, templates and tools to facilitate the planning and management of evaluations 17

18.  Consult closely with evaluators and program managers in defining goals and objectives for development/implementation of evaluation policy, evaluation guidelines, etc.  Respect the “organization evaluation structure”  Engage with senior, experienced officials early and regularly  Promote, facilitate the training and development of staff to effectively plan and manage evaluations  Provide concrete “how to” examples using evaluation information to inform policy decisions, changes in program design or shifts in program implementation 18