Presentation is loading. Please wait.

Presentation is loading. Please wait.

Agency Capabilities and Performance in Applying BCA Richard B. Belzer Regulatory Checkbook Mt. Vernon, VA Another Modest Proposal.

Similar presentations


Presentation on theme: "Agency Capabilities and Performance in Applying BCA Richard B. Belzer Regulatory Checkbook Mt. Vernon, VA Another Modest Proposal."— Presentation transcript:

1 Agency Capabilities and Performance in Applying BCA Richard B. Belzer Regulatory Checkbook Mt. Vernon, VA Another Modest Proposal for Reform

2 Society for Benefit Cost Analysis History of Federal Practice  Formalized since 1981 Agency responsibility to perform BCA OMB responsibility to perform peer review  Controversies and conflicts are hard-wired With Congress and among agencies Staffing, organization, and reprogramming Methodological issues and research priorities

3 Society for Benefit Cost Analysis ‘CSI: Executive Branch’  Bureaucratic biology BCA: A bacterial infection  Agencies have developed antibodies Program control over / co-option of agency economists Methodological corruption  Benefits can be found if you look hard enough  Costs can be ignored if you are determined enough OMB: Dissociative Identity Disorder ( ICD-9-CM Dx )  Performs external, independent peer review of RIAs… …but advance the President’s agenda…  …without being held publicly accountable for the difference where these functions conflict Administrative Procedure Act: Placebo therapy

4 Society for Benefit Cost Analysis Evidence of ICD-9-CM Dx Caveats  Agency (not OMB) estimates  From a draft report Invalid Assumptions  All ‘major rules’ are included  Agencies adhered to OMB Circular A-4  Estimates of costs, risks, and benefits are unbiased  Uncertainty is unimportant

5 Society for Benefit Cost Analysis Agency Use of Science in BCA: April March 2000  73 rules, 36 about risk  Embedded bias 7 (‘judgment’) 2 (assumptions) 6 (upper bounds) 1 (sensitivity analysis)  Uncertainty ignored 6 (point estimates) 2 (ranges) 1 (distribution)  Motive for performing BCA Statutory reqmt: 100% EO 12866: 14%

6 Society for Benefit Cost Analysis Examples: Major Rules from  ‘Hazardous Waste Corrective Action’ C: ~$1 trillion; B: not estimated  ‘Partially-Cooked Beef Patties’ C: < $10 million; B: not estimated  ‘Seafood HACCP’: impossible benefits Reduced risks from seafood (‘consumers underestimate true risk’) Increased consumer confidence in seafood safety (‘consumers overestimate true risk’)  ‘Meat & Poultry HACCP’: imaginary benefits Assume regulation reduces risk 90%

7 Society for Benefit Cost Analysis Examples: Major Rules from  ‘Hazardous Waste Corrective Action’ C: ~$1 trillion; B: not estimated; R: ~0  ‘Partially-Cooked Beef Patties’ C: < $10 million; B: not estimated; R: ~ 30 deaths/yr  ‘Seafood HACCP’: impossible benefits Reduced risks from seafood (‘consumers underestimate true risk’) Increased consumer confidence in seafood safety (‘consumers overestimate true risk’)  ‘Meat & Poultry HACCP’: imaginary benefits Assume regulation reduces risk 90%

8 Society for Benefit Cost Analysis Examples: Major Rules from  5 Major Rules PTO ‘Limits on Claims’ PTO ‘Limits on Continuations’ PTO ‘IDS’ PTO ‘Appeals’ DHS ‘No Match’  OMB made aware of costs but deemed none of them ‘major’

9 Society for Benefit Cost Analysis $1 billion! Examples: Major Rules from  5 Major Rules PTO ‘Limits on Claims’ PTO ‘Limits on Continuations’ PTO ‘IDS’ PTO ‘Appeals’ DHS ‘No Match’  OMB made aware of costs but deemed none of them ‘major’

10 Society for Benefit Cost Analysis HACCP Implementation Period BCAs Can Be Validated, But They Aren’t  Often easier than ex ante analysis  Very limited federal agency interest  Limited external WTP for ‘academic’ work

11 Society for Benefit Cost Analysis Dx 2001: Centralized Review Doesn’t Work  Limited and progressively smaller review staff  Transaction-driven, and hence reactive  Review occurs after decisions have been made  Asymmetrical rules of engagement  Limited enforcement tools  Enforcement has limited utility  Presidential initiatives are exempt  Congressional sensitivity is unbounded  Statutory/judicial deadlines trump OMB review  OMB has limited political and public support

12 Society for Benefit Cost Analysis Rx 2001: Failed Remedies  Persuasion  Co-option  RIA Guidance  Return to sender  Suspension of review

13 Society for Benefit Cost Analysis Rx 2001: First Modest Proposal Failed Remedies  Persuasion  Co-option  RIA Guidance  Return to sender  Suspension of review Possible Ideas  Regulatory budgeting  ‘Prompt’ letters  Publish peer reviews  BCA blueprints  Integrate BCA & PRA

14 Society for Benefit Cost Analysis Rx 2001: Implemented Failed Remedies  Persuasion  Co-option  RIA Guidance/Cir. A-4  Return to sender  Suspension of review Possible Ideas  Regulatory budgeting  ‘Prompt’ letters  Publish peer reviews  BCA blueprints  Integrate BCA & PRA  Information quality  Peer review policy

15 Society for Benefit Cost Analysis How Have These Remedies Fared?  Guidance, ‘prompts’, and returns

16 Society for Benefit Cost Analysis OMB Review: Lion or Pussycat?  Lion Reagan I Bush 41 Bush 43 (2001 only)  Pussycat Reagan II Clinton I & II Bush 43 ( )

17 Society for Benefit Cost Analysis How Have These Remedies Fared?  Guidance, ‘prompts’, and returns  Peer review policy (2005)

18 Society for Benefit Cost Analysis How Have These Remedies Fared?  Guidance, ‘prompts’, and returns  Peer review policy (2005) Covers ‘influential’ scientific, technical, & statistical information used in BCA’ Good idea, badly implemented  Too much agency discretion  Agencies control charge and reviewers  Reviewers cannot veto Agency BCAs reviewed by OMB are exempt!

19 Society for Benefit Cost Analysis How Have These Remedies Fared?  Guidance, ‘prompts’, and returns  Peer review policy (2005)  Information quality

20 Society for Benefit Cost Analysis Information Quality Law and Guidelines  ‘Information Quality Act’ (2001) 44 U.S.C note  OMB government-wide guidance (2002) Reduce error: Pre-dissemination review Correct error: Administrative procedures  Agency responses All complied except DHS Almost all were not serious

21 Society for Benefit Cost Analysis Information Quality Terms and Definitions  Utility  Integrity  Objectivity Substance  “accurate, reliable, and unbiased” Presentation  “accurate, clear, complete, and unbiased manner”  “within a proper context” Procedure (transparency, reproducibility)

22 Society for Benefit Cost Analysis Types of Information Quality Error Substance  Transcription  Units  Excess precision  Misapplication  Misuse  Bias (not policy-neutral) Presentation  Caveats omitted  Sources omitted  Context omitted  Data/results censored  ‘Best’ models only  Bias (not policy-neutral) Procedural  Non-transparency  Irreproducibility

23 Society for Benefit Cost Analysis Types of Bias  Incidental Not material to decision-making  Unavoidable No model is perfect, data are limited and expensive  Purposeful ‘2 + 2 = 5 for large values of 2’ Best justification given: ‘avoids false negatives’ Consequence: many false positives Conflates analytic and decision-making functions Lets analysts make policy decisions Fundamentally incompatible with information quality and BCA

24 Society for Benefit Cost Analysis An Example of Freely Advertised Purposeful Bias “EPA’s policy is that risk assessments should not knowingly underestimate or grossly overestimate risks. This policy position prompts risk assessments to take a more ‘protective’ stance given the underlying uncertainty with the risk estimates generated.” U.S. Environmental Protection Agency Office of the Science Advisor An Examination of EPA Risk Assessment Principles and Practices. Available:

25 Society for Benefit Cost Analysis Why Biased Risk Assessment Is So Toxic to BCA  Even if every other methodological problem in BCA is solved… BCA results are invalid and highly misleading  Baseline risks are overstated  Benefits of regulation are overstated CE-analysis yields unreliable rankings  Rank order would be correct if bias was constant  Bias is not constant  Tolerating bias incentivizes bias  Some interests are made better off if risk assessment and BCA are discredited

26 Society for Benefit Cost Analysis Flexible Principles: Activists & BCA Complaint Regulation Sophisticated Sabotage (2004) UndercountedBenefits OvercountedCosts Too hard/unethical to quantifyBenefits Double-countingCosts DiscountingRate too high Intergenerational benefitsOverwhelming Existence valueMassive Biases in data & analysisMust be ‘protective’

27 Society for Benefit Cost Analysis Flexible Principles: Activists & BCA Complaint Regulation Sophisticated Sabotage (2004) Roads, Bridges, Dams, etc. Damming the West (1973) UndercountedBenefitsCosts OvercountedCostsBenefits Too hard/unethical to quantifyBenefitsCosts Double-countingCostsBenefits DiscountingRate too highRate too low Intergenerational benefitsOverwhelmingOverwrought Existence valueMassiveNegligible Biases in data & analysisMust be ‘protective’Inexcusable always

28 Society for Benefit Cost Analysis Regulatory Checkbook Study of 5-Year Agency Performance  Procedural compliance Timely? Informative? Interpretively correct? Forgiving of limited sophistication?  Substantive compliance Accurate: Is response true? Honest: Are legitimate complaints admitted? Responsive: Is the remedy reasonable?

29 Society for Benefit Cost Analysis Agency Performance on the Least Demanding Procedural Index Requests for Correction of Information Quality Errors, : Selected Agency Response Times (Days) Agency N Cases Avg (days) SD (days) Min (days) Max (days) ‘Timely’ (days) % ‘Timely’ HHS % EPA %

30 Society for Benefit Cost Analysis Why Is Agency Performance So Bad?  Institutional Externally imposed Some errors are useful Acknowledging error increases litigation and compromises enforcement  Legal WH decision to issue guidance, not promulgate a rule Limited enforcement of guidance by OMB No enforcement of guidance by courts No congressional oversight

31 Society for Benefit Cost Analysis What Can the New Society for Benefit Cost Analysis Do?  Publish a journal Some theory Lots of applications  Publish an external, authoritative manual OMB guidance documents (1990, 2000, 2003) need improvements that OMB staff cannot make An external authority is better  Peer review federal agency BCAs Use information quality levers (insist on objectivity) Convene genuinely independent panels M*A*S*H approach (‘meatball economics’)


Download ppt "Agency Capabilities and Performance in Applying BCA Richard B. Belzer Regulatory Checkbook Mt. Vernon, VA Another Modest Proposal."

Similar presentations


Ads by Google