1. Reference Guide on the Ex-Post Evaluation of CAs’ Enforcement Decisions Ania Thiemann Head of Global Relations Competition Division, OECD ICN AEWG workshop Gaborone – 11 March 2016

2. Ex-post evaluation is an examination of a CA’s enforcement decision that: 1.is performed to determine what has been its impact on the affected market, relative to alternative scenario(s) 2.is done some time after the decision 3.is based on the use of ex-post data Why a Guide on Ex-post Evaluation? Ex-post evaluations can significantly improve the CAs’ decision- making process if conducted regularly The Reference Guide is not a “cookbook” on how to perform evaluation It discusses the issues to be taken into account It explains what methodologies can be used It provides references to the relevant theoretical papers It is rich in examples derived from existing studies What is Ex-Post Evaluation 2

3. Why Ex-Post Evaluation (I) Analysis of the competition concerns Decision to intervene / not to intervene Implementation Ex-post evaluation 3

4. The benefits of ex-post evaluation Improving decision-making by learning from past experiences Increasing accountability and advocating the role of CAs The benefits of regular ex-post evaluations Why Ex-Post Evaluation (II) 4

5. Select the decision to assess Choose the evaluation team Identify the counterfactual Select the methodology Determine the variables to study Collect data and information Perform the analysis Verify the robustness of the results Incorporate the lessons learnt How to perform an Ex-Post Evaluation 5

6. The example of New Zealand: Test key assumptions and hypothesis Aim: test whether key market predictions were correctly predicted Target decisions : mergers cleared because Low barriers to entry Enough competition after merger Divestiture Buyers’ countervailing power Data: publicly available information and interviews Conclusions on the validity of the main expectations Approach easier to achieve in terms of time, data availability and resources Csorgo, L. & Chitale, H. (2015). Targeted Ex Post Evaluations in a Data Poor World. New Zealand Commerce Commission. http://www.comcom.govt.nz/the-commission/media-centre/speeches/targeted-ex-post-evaluations-in- a-data-poor-world/http://www.comcom.govt.nz/the-commission/media-centre/speeches/targeted-ex-post-evaluations-in- a-data-poor-world/ Focus on methodology 6

7. Example: gasoline studies in the US Concern that increased market concentration resulting from petroleum mergers that occurred in the late 1990s were responsible for increased gasoline prices Bureau of Economics staff conducted six studies of eight consummated mergers in the petroleum industry Studies did not find evidence of a significant increase in retail price following the studied mergers Conclude that causes other than mergers were responsible for large increases in gasoline prices Source: Daniel Hosken (FTC), presentation at the OECD Workshop on Ex-Post Evaluation, 22 April 2015. Available at http://www.oecd.org/daf/competition/workshop-expost-evaluation-competition-enforcement-decisions.htm http://www.oecd.org/daf/competition/workshop-expost-evaluation-competition-enforcement-decisions.htm Focus on lessons learnt 7

8. Annex A A discussion of a specific subset of ex-post assessments of enforcement decisions: the studies of the effectiveness of remedies Annex B An outline of the methodologies that are currently available for assessing the effects of enforcement decisions Annex C An extensive list of references, classified according to (i) the nature of the decision assessed, (ii) the methodology employed, (iii) the variables studies and (iv) the sector examined The Annexes to the Reference Guide 8

9. The OECD advocates the systematic ex-post evaluation of policy interventions as a means to improve their quality Ex-post evaluations help to determine if a policy intervention reached its objectives and if not, then why not. This provides useful lessons for the better design of future interventions Conclusions 9

10. THANK YOU For questions: silvia.carrieri@oecd.orgsilvia.carrieri@oecd.org More on OECD competition work at http://www.oecd.org/daf/competition/ http://www.oecd.org/daf/competition/