1. Evaluation and Impact Estimation at the OFT and its applicability to CREW Natalie Timan Deputy Director, Office of Fair Trading 13 March 2013 1

2. Introduction ● Contents:  Evaluation objectives: why do we measure impact?  Types of evaluation within the OFT  Essential features of an Impact Assessment  Methodology  Two worked examples  Application to CREW  Ex-post evaluation 2

3. Evaluation objectives ● The aim of our evaluation work is two-fold:  External accountability: to evaluate whether the OFT delivers its objectives and does so cost effectively and to demonstrate this externally;  Internal management and prioritisation: to inform future OFT work regarding how we prioritise, conduct and follow-up our work to ensure we maximise our impact. This is not an internal evaluation of our procedures or efficiency, but an assessment of the accuracy of our predictions about markets and the effectiveness of our interventions. 3

4. Types of evaluation ● Impact estimation: ex-ante estimation of anticipated impact usually undertaken shortly after the OFT’s intervention.  Direct financial savings for consumers only – no wider benefits. ● Ex-Post evaluations: ex-post estimation of actual impact usually undertaken some years after the OFT’s intervention. ● Application to CREW: ex-ante estimation may be the more useful type of evaluation given:  Cost effectiveness;  Data limitations;  Time constraints. 4

5. Welfare standard ● Both types of evaluation look at benefits to consumers as the OFT has a Consumer Welfare standard.  Typically estimate effect on prices and output as a proxy for consumer welfare ● OFT does not estimate potential impact of its decisions on business as part of positive impact estimation. ● Impact estimation looks at quantifiable direct financial benefits to consumers:  Decrease in price;  Monetized improvements in quality, range or service;  Monetized time savings;  Benefits that consumers gain from making better informed choices about what goods to purchase. 5

6. Essential features of OFT’s Impact Estimation I* ● Assessed on regular (annual) basis during next year. ● Relatively undemanding in cost and time, utilising data from the original interventions and/or simple default assumptions. ● Estimates are performed using ex-ante data. ● Assumed that no intervention can have a negative impact. ● Estimates are deliberately ‘conservative’. * These features were presented by Professor Steve Davies to the OECD on 25 February 2013 6

7. Essential features of OFT’s Impact Estimation II ● Estimates in terms of static consumer benefits. ● Annual moving averages employed. ● ‘Point’ estimates, rather than a range of plausible values. ● Covers mergers, cartels, abuse, market studies and consumer enforcement. Does not cover advocacy of competition to other government departments. ● Deterrence impact not included in estimate. ● Possible beneficial effect of competition policy of productivity and innovation (and growth) not estimated. 7

8. Methodology: assumptions ● To estimate the impact of each individual case intervention, information is required on:  The size of affected turnover;  The price increase removed or avoided;  The length of time the increased price would have prevailed absent the intervention. ● An adjustment could be made for the deadweight loss (i.e. Surplus lost by consumers who are deterred from making any consumption). ● Ideally use any case-specific estimates from the case team; if not default assumptions required: 8

9. Methodology: default assumptions MergersCartelsAbuse Affected turnover Turnover of affected goods Price effect GUPPI (or 3-5%) plus deadweight loss estimate 10-15% Duration (yrs)266 9

10. Two recent examples: An Abuse of dominance case Summary of total consumer detriment two year period Product A £m Firm 150 Firm 220 Firm 330 Firm 440 Total harm over 2 year period140 Total harm over 2nd year alone93 Assumed duration (years) into future absent OFT intervention ie we think it would have continued for further 2 years 2 Total consumer detriment averted in following 2 years (deflated and discounted to prices at decision date) 291.96 10 ● The figures represent the extra turnover derived from the implementation of various initiatives over a two year period, borne directly by the UK consumer. ● This is calculated by multiplying the volumes sold by the amount of increment, bearing in mind that the price increases were maintained and carried over into the following years. So, for example, if the price of one product was increased by £5 in year one and £5 in year two, then the total benefit would be £15.

11. Two recent examples: A cartel case Turnover and Profit (£m)Firm 1Firm 2Firm 3Firm 4Firm 5Firm 6Total £m Net Fees£5£2£3£10£20£10£50 Net fees converted from year of cartel to prices at year of decision£5.1£2.0£3.1£10.0£20.4£10.2£51 Saving in net fees from OFT intervention15%£0.8£0.3£0.5£1.5£3.1£1.5£8 Savings in net fees from OFT intervention over 3 years3£2.2£0.9£1.3£4.4£8.9£4.4£22 11 Attribution of impact is justified by: Leniency programme was used Evidence that cartel met after application for leniency Assume 15% reduction in net fees Assume 3 years duration. Duration is scaled down from rule of thumb of 6 years on account of: Instability of cartel Some participants were not adhering to rules of cartel Total impact: £22m

12. Impact estimation 2009-10 Benefits2010-11 Benefits2011-12 Benefits Competition enforcement £84m£83m£151m Merger control£125m£90m£8m Market studies and reviews £107m£117m£207m Consumer protection enforcement £42m£36m£35m Total estimated benefits £359m£326m£402m Benefit/Cost ratio778 12 ● The figures presented above are 3-year rolling averages.

13. Application to CREW I ● Rules of thumb  Affected turnover  Discount rate  Cartel overcharges in developing countries  Price rise in abuse cases  Alternatives to merger simulation  Duration 13

14. Application to CREW II ● Deterrence ● Producer surplus: barriers to entry ● Competition, innovation and growth  Output = f [natural resources, capital, labour] technology, management  Prioritise interventions that lift restrictions on e.g.: Land (natural resource); Finance (capital) Markets associated with labour participation and higher education (labour) Innovation intensive sectors e.g. Pharmaceuticals Tackling public restrictions on competition -> managers not sheltered from impact of competition 14

15. Ex-Post Evaluation ● What is ex-post evaluation?  An ex-post assessment of the actual impact of our interventions in enforcement cases.  Primary purpose: measure the net benefits to consumers of any intervention. Secondary purpose: examine wider benefits (usually unquantifiable), such as the effect on consumer confidence and deterrence.  Typically compares the quantified net benefit to the cost incurred by the OFT in the initial intervention. ● How do we evaluate?  Can involve: (i) desk-based research on prices and other market data, (ii) surveys of consumers, businesses and other stakeholders.  Usually conduct two to three evaluations a year. Can be conducted immediately after a case is completed, but typically there is a delay, potentially up to several years, whilst recommendations are implemented. 15

16. Methodology: Ex-post evaluation ●Formal techniques:  Simulations and structural models  Event studies  Difference in Difference techniques ●Reality:  Identify changes in market characteristics (price, quality, choice etc)  Identify changes in consumer behaviours (shopping habits, awareness and understanding, product selection)  Link those changes to actions or recommendations – are the changes attributable to us?  If possible monetise the changes that are attributable to authority 16