1. Program Performance Accountability & Measurable Results Kevin Keaney, Chief Pesticide Worker Safety Programs U. S. EPA 2005

2. OPP Outline H Performance Measurement: Context, Value, Types H Accountability Drivers & Problems H GPRA & PART H Elements of a Good Measures Program

3. Where Accountability Fits Strategically Program Success is built from critical components. CLEAR GOALS Public Health & Ecological Protection Outcomes GOOD PLANNING Strategic Objectives EFFECTIVE IMPLEMENTATION Multiple Levels ADEQUATE RESOURCES Acquisition & Allocation MEANINGFUL ACCOUNTABILITY Measures & Tracking

4. OPP Value of Performance/Results Data MULTIPLE USES H Guiding Program Management: Decisions; Priorities; Resources H Performance Accountability H Tracking program success and progress H Communication H General Public H Interested Stakeholders H Decision Makers (Congress, Admin.) Meaningful Accountability = Good Management = Good Government

5. OPP 5 Types of Performance Measures Performance measures vary depending on level of organization & need. Strategic Goals Broader/Long-Term Specific/Short-Term Mid-Range Objectives Annual Commitments Individual Performance Stds

6. OPP Accountability Drivers / Mechanisms Statutes EPA 5 Year Strategic Plan 3 Year NPM Commitments Budget Formulation Annual Work Plans Budget Allocation Tracking Systems/Reporting Audits Performance Standards

7. OPP Problems w/ Current Systems H Components: …are complex, …are created at different times, …by different people, …for different purposes, …little thought about connections or alignment among the parts. H Inevitably creates: …conflicts, …confusion, …unnecessary burdens, & …mixed messages. H Doesn’t work: …no accountability …no consequences.

8. OPP Accountability Overload FIFRA! GPRA! REGIONAL INITIATIVES! OECA! LAWSUITS! STRATEGIC PLAN! OMB! PART! BUDGET! SPECIAL PROJECTS! MEDIA COVERAGE! FLEXIBILITY! FOCUS! POLITICAL PRESSURE!

9. OPP Accountability Drivers/Mechanisms Statutes EPA 5 Year Strategic Plan 3 Year NPM Commitments Budget Formulation Annual Work Plans Budget Allocation Tracking Systems/Reporting Audits Performance Standards

10. OPP Accountability Drivers/Mechanisms Statutes EPA 5 Year Strategic Plan 3 Year NPM Commitments Budget Formulation Annual Work Plans Budget Allocation Tracking Systems/Reporting Audits Performance Standards

11. OPP GPRA GPRA = Government Performance and Results Act of 1993 H GPRA is a statute aimed at increasing accountability by requiring measurement of program results & linking those results to resource allocation. H Provides a statutory framework for strategic planning and reporting that requires federal agencies to: H Establish accountability measures to assess the outcomes of each program activity, and H Prepare performance plans that establish annual performance goals.

12. OPP GPRA Auditing H Program Assessment Rating Tool (PART) H Office of Management & Budget assumed the role of evaluating progress against GPRA. H OMB is committed to audit all federally funded programs against GPRA requirements on a 5 year cycle. Size of individual programs receiving detailed audit varies. H This is the third year. Several Pesticide programs have been audited.

13. OPP GPRA Audits OMB Fire is invented

14. OPP GPRA Auditing H Program Assessment Rating Tool (PART) H A matrix of questions with a numeric value & requiring succinct answers. H Principle focus has been on the existence of Outcome Measures satisfying OMB. H OMB has not been bound by pre-existing measures (EPA’s Strategic Plan; NPM Guidance). H Moving target of changing requirements. H Short turn-around times. H Average initial scores of programs have been very low (40 - 60 out of 100) and many classified as “Results Not Demonstrated”. H “re-PARTing” is the norm. H Results are published each year with the federal budget. H Programs not demonstrating results for successive years are subject to budget cuts or elimination.

15. OPP PART Budget Consequences EPA OMB

16. OPP Criteria for Good Measures VALUE HIGH Outcomes Clear Wide use LOW Beans Complex Narrow use Meaningful Understandable Useful BURDEN LIGHT Few Simple Low User-friendly HEAVY Many Hard High Technical # of Measures Difficulty of Collection Data System Costs Expertise Required

17. Performance Measures Quadrant Analysis High Value Low Value Light Burden Heavy Burden OUTCOME NIRVANA OUTPUT HELL 1. High Value/Light Burden 2. High Value/Heavy Burden 4. Low Value/Heavy Burden3. Low Value/Light Burden

18. Measures Comparison Matrix VALUEBURDEN MeaningfulUnderstandableUseful Possible Measures Criteria

19. OPP Process for Measures Improvement H Specific Problem Formulation H Baseline data collection: Current measures; current data systems, etc. H Scope of project? Strategic measures only or a universal results ethic (measures for all projects). H Participants? OPP; OECA; Regions; States/Tribes; industry; NGOs; etc. H Outputs? Measures for Strategic Plan/NPM/Annual Workplans/PART; Public Reports; Change risk assessment/management practices; Change or consolidate databases. H Timeline/Deadlines? 6 months (PART); 2 years (next 5 Year Strategic Plan).

20. Process Structure Senior Level Steering Committee Set Direction; Manage Process; Resolve conflicts / issues Program Specific Workgroups Gather Data; Coordinate; Develop Measures + Data Systems Regis.WPC&TESA Regis. Review SAI Coordinate w/ Partners & Stakeholders RegionsUSDA OW,ORD, OECA Industry States, Tribes NGOs Databases & Information Systems Evaluate existing sources; Opportunities for cost savings

21. OPP Take Away Message H All government funded activities will be held to higher standards of performance accountability to justify budgets. H Cooperative work is required to establish meaningful accountability measures and performance targets. H Well documented progress toward quantifiable public health and environmental protection goals will help ensure continued program operation and funding.

22. OPP Program Performance Accountability & Measurable Results QUESTIONS ?