1. U.S. Federally-funded Contract Evaluation: Communicating to Enhance Relationships, Responsibilities, and Relevance Sponsored by the Government Evaluation TIG 2012 American Evaluation Association Conference Panel Session 108, Marquette Section VI October 24, 2012

2. Collective Disclaimer 2 The opinions expressed in this session are those of the individual participants, and do not reflect an official position of the organizations for which we work.

3. Purpose of the Panel The vast majority of United States Federal government evaluations are conducted by contractors. The issue of communication between contractors and their federal clients is rarely explored. The AEA annual conference provides an opportunity for dialogue between evaluation contractors and federal staff. This session is intended to begin a dialogue and exploration. 3

4. Question Set 1: Communication 1.What are some of the more effective strategies for communication between contractors and government clients? 2.How can communication be strengthened? 3.Does more frequent communication help, hinder, or just add to the cost? 4

5. Question Set 2: Handling Obstacles 1.What are some useful mechanisms for handling obstacles that invariably come up in conducting contracted evaluations? 2.What practices should be written into contracts as a requirement, in order to make those best practices more systemic? 3.To what extent do legal and regulatory processes limit the quality of contracted evaluations and inhibit communication? 5

6. Context for Question Set 3 “It is the policy of the Federal Government that executive agencies shall not engage in noncompetitive contracts except in those circumstances where their use can be fully justified and where appropriate safeguards have been put in place to protect the taxpayer. In addition, there shall be a preference for fixed-price type contracts. Cost- reimbursement contracts shall be used only when circumstances do not allow the agency to define its requirements sufficiently to allow for a fixed-price type contract.” Source: President’s March 4, 2009 Memorandum on Government Contracting, http://www.whitehouse.gov/the_press_office/Memorandum-for-the-Heads-of- Executive-Departments-and-Agencies-Subject-Government 6

7. Question Set 3: Contracting 1.What is the right balance between highly specific requests for proposals that dictate methods to be used and less specific requests for proposals that put a lot of the responsibility for study design and methods selection on the contractor? 2.Would additional dialogue on the purpose, scope, and methods of an evaluation lead to more effective and/or efficient evaluations? 3.Do federal agency contracting officers request/require particular contracting vehicles? 4.What effect has the President’s memorandum had on federal contracting for evaluation? 7

8. Question Set 4: Evaluation Use How important is it for the government to specify the potential uses of the evaluation and the stakeholders who might be interested in the design, conduct, and evaluation results? 8

9. Discussion Comments from Kathy Newcomer. Questions and comments from the audience. For a copy of these slides, please contact David Bernstein by email (see next slide). Conclusions and acknowledgements. 9

10. Contact Information 10 Chair: David J. Bernstein, Westat, davidbernstein@westat.comdavidbernstein@westat.com Discussant:Kathryn Newcomer, The George Washington University, newcomer@gwu.edunewcomer@gwu.edu U.S. Federal Government Panelists: Katherine Dawes, U.S. Environmental Protection Agency, Dawes.Katherine@epamail.epa.govDawes.Katherine@epamail.epa.gov Panel audience members Contractor Panelists: Herb Baum, Data Recognition Corporation, HBaum@DataRecognitionCorp.comHBaum@DataRecognitionCorp.com Jennifer Hamilton, Westat, JenniferHamilton@westat.comJenniferHamilton@westat.com William Scarbrough, Pacific Institute for Research and Evaluation, bscarbrough@PIRE.orgbscarbrough@PIRE.org