1. Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review

2. What I will Talk About Some concepts Macro productivity measurement: the Atkinson Review (and Gershon) Micro productivity measurement: some NHS examples Challenges for the future

3. Productivity: What do we mean? Getting more for the same, or the same for less Technical efficiency allocative efficiency value for money effectiveness economy cuts Increase in output divided by increase in volume of input Improving services while money is tight

4. Measuring Productivity: Who By, What for? Do you get what you pay for? Accountability to the tax payer Just try harder Performance management Cant we get another one in? - Local drivers to expand/ improve The XX industry is inefficient by international standards Evidence of use of resources Best of breed have 3 times more throughput than bottom quartile Benchmarking, competitive pressures

5. Some key questions Can Government measures of public service productivity be valid? Is measuring productivity the same as measuring performance? Improving productivity matters – is measurement a side alley? Does measurement of efficiency create distortions?

6. The Atkinson Review How to use National Accounts to measure public service productivity and How to do it better

7. How National Accounts Measure Public Services Traditionally, output = input From 1997, System for National Accounts changed: measure outputs via activities UK early implementer – health (cost weighted activity index), schools (pupil days and quality adjustment), social services, social security admin ONS began publishing productivity articles

8. ONS Output Measure, 2003

9. Better Measures for the National Accounts Queries on basis of public service measures National Statistician set up Review Dec 2003, chaired Sir Tony Atkinson Measurement of Government Output and Productivity for the National Accounts

10. Atkinson Report Jan 2005 9 principles output should be measured in a way that is adjusted for quality, taking account of the attributable incremental contribution of the service to the outcome Specific recommendations for improvement to measures used in NA Encouraged ONS productivity articles UK Centre for Measurement of Government Activity

11. ONS Health Productivity Oct 2004

12. DH Press Release Oct 04 John Reid (Secretary of State for Health) says it is absurd to measure NHS output without taking account of quality

13. Quality as part of NHS Output How many domains of quality? –Health gain –Patient experience What can we measure? How can we link quality measures to the NHS output index? How should we weight different aspects of quality? How valid is a partial story?

14. Accounting for Quality Change Average over last 5 years: Value of health 1.5% Value weight for statins 0.81% York/NIESR adjustment0.17% Patient experience* 0.07% Blood pressure control *0.05% Heart attack survival 0.01% Total **2.68% Quality adjusted output growth 6.29%

15. ONS Health Productivity 2006

16. Education and Other Areas DfES action following Atkinson – new use of pupil attainment tests and GCSE grades DfES article, ONS education productivity article Controversial issues – grade drift, real earnings effect Also productivity articles for adult social care, social security administration

18. Value and Validity of Atkinson-based Productivity Measures Designed to compare total outputs with total inputs Focus on attributable impact on outcomes and quality change Data incomplete; biased towards areas of attention / improvement; analysis by Depts Major developmental issues on techniques ONS consultation document Sept 2006

19. Performance = Productivity? A perfect Atkinson measure captures all activities, all contributions to outcomes Is this the same as measuring organisation performance? If not – choices about key priorities, value judgements Should those values inform the aggregate productivity measure?

20. Another macro measure: Gershon Gershon includes savings from better procurement, Atkinson doesnt Specific changes in use of inputs valued as £m Better use of productive time – evidence from outputs and outcomes?

21. Gershon and Atkinson Gershon efficiency improvements are managed, not just measured Strong delivery support for Gershon, changes evident Measured to deliver target? – NAO scrutiny Will measured productivity gains match? – HMT, OGC and UKCeMGA should be able to explain

22. Micro Measures: NHS NHS Institute for Innovation and Improvement, with DH Developing and issuing key metrics for health authorities, PCTs, Trusts, FTs Systematic focus on key areas for improvement – volume, costs, variation Clinical engagement, benchmarking

23. First Metrics: 22 Sept 2006 Pdfs and web based tool Finance, workforce, procurement Clinical productivity: –Potential bed days saved through reduced length of stay –Day case rate –Reduce pre-operative bed days

24. Reduce wasted bed days 390,000 bed days £100m

25. Benchmarking Evidence based assessment of performance Addressing variations and helping outliers Providing a tool kit Opportunity to test performance against peers Stimulus to learn how to improve practice Recognition of success and challenge to do better

26. XXX NHS Trust

27. Value and Power of Micro Productivity Indicators Practical engagement in what to change Service based, clinical focus Small steps on long journey Risks that change happens, savings dont Check for unintended perverse incentives Tipping points for cutting overheads

28. Impact of Micro Changes on Macro Measure NHS III says: 1 – Reduce Avoidable Emergency Admissions 2 – Reduce Unnecessary Outpatient Appointments, Follows-up, and DNAs 3 – Avoid Unnecessary Procedures 4 – Improve Day Case Performance 5 – Reduce Wasted Bed Days 6 – Accurate Clinical Coding 7 – Reduce Variation in Length of Stay 8 – Improve Staff Productivity 9 – Actively Manage Staff Costs

29. If NHS follows NHS III advice.. Lower growth in some hospital activities Shifts in unit costs Shifts in NHS spending to primary care, community, public health (harder to count) Probably, slower growth in output Measurable change in quality, outcomes? Productivity measure?

30. Measures, not Targets Macro and micro productivity measures both need careful interpretation Micro are more useful for managers, front line, actionability Macro should help with do you get what you pay for – but may mislead Use of any efficiency measures changes behaviours – intended, maybe also unintended

31. Lessons of History For the first decade of its existence (1982-92) the HCHS efficiency index received little adverse comment either from the NHS or policy analysts. However, once it was converted from a retrospective analytical exercise into a management tool it attracted a barrage of criticism….attaching rewards to counts of particular activities led to reporting drift, reclassification and at worst statistical dishonesty Clive Smee Speaking Truth to Power 2005

32. Dilemma: Measuring with Right Impact Productivity is Govt business: accountability Many drivers in public service to improve productivity Accountability needs metrics Metrics are almost always partial, may mislead or have unintended incentives Choice of metrics affects what is delivered Moral: use mix of macro and micro indicators, with intelligence & transparency