1. Governance by Targets & Terror: Synecdoche, Gaming & Audit
Gwyn Bevan (LSE) & Christopher Hood (All Souls College, Oxford)
Westminster Economic Forum
20 April 2005

2. Governance by Targets & Terror
New approach to health care governance in 2000s
Decisive breakthrough in governance – or repeat of history of Soviet Union?
Conclusion

3. Governance by Targets & Terror
New approach to health care governance in 2000s
targets & indicators linked to negative feedback
Voltaire: ‘ici on tue de temps en temps un amiral pour encourager les autres’
Decisive breakthrough in governance – or repeat of history of Soviet Union?
Conclusion

4. Three phases of governance of public health care systemUK
Concordat clinical autonomy & resource constraints
no real command or control from the centre
1980s: various attempts to empower managers
1990s: attempt to control public health-care professionals through quasi-markets
separating providers & purchasers
England only
from new ‘concordat’ of higher spending accompanied by P.I.s & targets monitored from centre by multiple & overlapping units (& ‘terror’?)

5. NHS spend as % of GDP % GDP
Sources: Office of Health Economics, HM Treasury, & official projection of 9.4 per cent by 2008

6. Institutional arrangements
Prime Minister’s Delivery Unit
22 targets
Treasury
130 PSA targets– c. 10 for health
Audit Commission & NAO
Audit finance & vfm
Dept of Health
50 targets by trust
type
Healthcare Commission
Quality regulator develops & publishes ratings
Health care ‘trusts’ (c.700)
money
reporting
dialogue

7. Some underlying assumptions
Synecdoche: a part can meaningfully stand for the whole
What is measured is a good indicator of performance (e.g. ‘threshold effects’ at the top of the quality range either will not occur or do not matter)
Gaming effects (the ‘knights-to-knaves’ problem) are either small or unimportant

8. Defining priorities Residual domain β
Domain α: government’s priorities

9. Measuring priorities Domain αn: no measures Residual domain β
Domain αg: good measures M[αg]: no false positives or negatives
Domain αi: imperfect measures M[αi]: large numbers false positives & negatives

10. The synecdoche assumption: the part can represent the whole
Residual domain β+: omitted because unimportant or cannot be measured
Domain α-: government’s priorities for which good & imperfect measures exist

11. ‘Threshold Effects’ either don’t matter or won’t Happen
Success
Frequency
Failure?
Waiting time
In months
Target

12. Knights either will not turn into Knaves or Knavery can be Controlled
‘Saints’
may not share mainstream goals
public service ethos so high that they voluntarily disclose shortcomings to central authorities
‘Honest triers’
broadly share mainstream goals
do not voluntarily draw attention to their failures
but do not attempt to spin or fiddle data in their favour
‘Reactive gamers’
broadly share mainstream goals,
but aim to spin or fiddle data if they have a motive or opportunity to do so
‘Rational maniacs’
do not share mainstream goals
aim to manipulate data to conceal their operations

13. Target & PI systems against 4 types of actors
‘Saints’
‘Honest triers’
‘Reactive gamers’
‘Rational maniacs’
Expected effect of targets
NO CHANGE

14. What is measured is a good indicator of performance?
Works for saints
Problems for honest triers
Vulnerable to Reactive gamers
Fails for rational maniacs
Gresham’s law: saints & honest triers  Reactive gamers?

15. What is measured is a good indicator of performance? Agent satisfied
signals M[α-]
All is well?
Domain α-  but domain β+ ?
Domain α- ?
Failure on M[α-] concealed by problems of
definition & measurement?
gaming?
Domain α-
Domain β+
Signals M[α-]:

16. Governance by Targets & Terror
New approach to health care governance in 2000s
Decisive breakthrough in governance – or repeat of history of Soviet Union?
Some evidence from the English NHS
Conclusion

17. Development of star ratings in England
2001
2002
from
2003
Acute (156)
Specialist (20) 
Ambulances (31)
Mental Health (88)
PCTs (304)

18. Evidence of impact of key targets
Ambulances
75% category A calls < 8 minutes
Hospitals
total time in A&E < 4 hours
waiting times for elective inpatient admission
Reported successes
Problems of measurement & gaming

19. Reported success: 75% category A calls<8 minutes
Before
After

20. Problems of measurement & gaming: third ‘corrected’ response times
‘Corrections’’ only 2% to 6%
75% < 8 minutes
75% < 8 minutes
Source:

21. Impact: A&E total time < 4 hours
& a 20% increase in numbers in A&E
Source: Improving Emergency Care in England

22. Problems of measurement & gaming in A&E
Government figures: mid-2004, target met by 96% of patients
Healthcare Commission survey (55,000 patients): 77% of patients stayed < four hours in A&E
Problems of gaming
Queues of ambulances outside
Moving staff & cancelling operations over period of measurement

23. Impact of key targets on hospital waiting times for elective inpatient admission
Target waiting time (months)
for elective inpatient admission

24. Impact: hospital waiting times elective admission
Numbers waiting elective admissions (‘000s)
Star ratings
published
Source: Chief Executive’s Report to the NHS – Statistical Supplement (2004)

25. Impact of key target: hospital waiting times elective admission
Numbers waiting elective admissions (‘000s)
Star ratings
published
Source: Chief Executive’s Report to the NHS – Statistical Supplement (2004)

26. Impact of key target: hospital waiting times elective admission
% patients waiting for
hospital admission > 12 months
Source: National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38

27. Impact of key target: hospital waiting times elective admission
% patients waiting for
hospital admission > 12 months
Source: National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38

28. Impact of key target: hospital waiting times elective admission
% patients waiting for
hospital admission > 12 months
Source: National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38

29. Impact of key target: hospital waiting times elective admission
% patients waiting for
hospital admission > 12 months
Source: National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38

30. Natural experiment in UK countries
England (from 2001): annual performance (star) rating
Zero to 
‘naming & shaming’
Wales, Scotland, Northern Ireland
no ranking
no incentives
Spend per capita on
health care
(UK = 100)
Source:

31. Waiting times: problems of measurement & gaming
Audit Commission: reporting errors at least one PI in 19 trusts
Problems of gaming
NAO: 9 NHS trusts inappropriately adjusted their waiting lists
Audit Commission: 3 cases of deliberate misreporting of waiting list information
Sources:

32. ‘Synecdoche’ problems over quality of care
2 failures that resulted in major public inquiries
Bristol case of paediatric cardiac surgery
Shipman
All difficult to detect from outside
All could plausibly have sent ‘satisfactory’ M[α-] signals under star ratings regime
And M[α-] ‘failures’ might nevertheless provide excellent quality of care

33. Governance by Targets & Terror
New approach to health care governance in 2000s
Decisive breakthrough in governance – or repeat of history of Soviet Union?
Conclusion
How far did English NHS satisfy assumptions?
Repeating Soviet history: 1939? 1969? 1989?
Policy implications?

34. How far did English NHS satisfy assumptions?
Synecdoche: a part can meaningfully stand for the whole?
What is measured is a good indicator of performance?
Gaming effects (the ‘knights-to-knaves’ problem) are either small or unimportant?

35. Repeating Soviet history: 1939? 1969? 1989?
1939: waiting times?
1969: ?
1989: balanced scorecard?

36. Policy implications: targets retained, limiting gaming?
Limits on transparency
Designing & resourcing systems of validation
Sources of information other than data reported by those being monitored