1 Module 4: Designing Performance Indicators for Environmental Compliance and Enforcement Programs

2 Basic Concepts

3 Indicators…  Measure and manage environmental compliance and enforcement program performance  Ensure accountability to legislative bodies, budget authorities, constituent groups, and the public  Evaluate and improve effectiveness in achieving program mission

4 Types of Indicators  Input: personnel, funds, and other resources that contribute to an activity (example: number of inspectors)  Output: quantitative or qualitative measures of activities or work products (example: number of inspections)  Intermediate Outcome: changes in knowledge, behavior, or conditions that result from program activities (example: pounds of illegal discharge reduced)  Final Outcome: the change in environmental quality attributable to program activities (example: improvement in water quality)

5 Audiences & Users  National/state policy makers and legislators  Regulatory agencies  Industry sectors  Stakeholder and interest groups  General public  International bodies

6 Benefits of Indicators  Targeting of program operations  Ability to set goals and adjust strategies  Decision-making for resource allocation  Ability to identify problems and correct performance  Ability to motivate employees  Communications with the public

7 Best Practices

8 Identifying Indicators  Define Scope of Indicators  Conduct Meetings with External Stakeholders and Internal Staff  Apply the “Logic Model”  Develop Guiding Principles  Select Criteria for Evaluating Potential Indicators  Develop Common Definitions for Key Terms  Inventory Existing Data Sources  Select an Appropriate Combination of Indicators

9 Criteria for Evaluating Indicators  Relevant: to goals, objectives, and priorities  Transparent: promote understanding of program  Credible: based on data that is complete and accurate  Functional: easy to use  Feasible: value to program outweighs cost  Comprehensive: cover important operational aspects

10 Developing Indicators  Select final indicators using criteria  Establish internal work teams to:  Define measures in more detail  Review existing data for relevancy  Develop new information collection and reporting processes  Establish schedule for testing and implementing indicators  Conduct pilot projects to test indicators and correct problems  Resolve technical and methodological issues  Execute implementation in phases

11 Using Indicators  Monitor Implementation with Regular Reports  Use Indicators to Analyze Performance of Organizational Units  Use Indicators to Review Effectiveness of Specific Programs  Use Indicators to Report to External Audiences  When Analyzing Data from Indicators, Understand the Data Limitations  Assess and Adapt Indicators

12 Applying Indicators to Water Governance

13 Examples of Indicators of Effective Water Governance  Number of discharge permits revoked  Total value of fines levied against violators  Number of facilities ordered to cease operation until compliant with water regulations  Number of permits granted or updated  Number of inspections/patrols conducted per month  Amount that discharges from facilities have been reduced  Exceedances addressed or returned to compliance

14 A “Logic Model” of Performance Measurement Indicators InputsresourcesOutputsactivities Intermediate Outcome behavior change Final Outcome environmental Impact Personnel Funds for salaries, contracts, IT, etc. Inspections conducted Enforcement actions taken Fines assessed Improved facility management practices Reduced illegal discharges Greater understanding of how to comply Improved ambient water quality Improved human and ecological health

15 Exercise: Applying the “Logic Model” InputsresourcesOutputsactivities Intermediate Outcome behavior change FinalOutcome environmental impact

16Conclusion