1. Developments & Issues in the Production of the Summary Hospital-level Mortality Indicator (SHMI) Health and Social Care Information Centre (HSCIC)

2. Overview Overview of SHMI How SHMI was developed and is governed How issues are captured and handled How changes to the methodology are reflected and communicated

3. What is SHMI Single hospital-level mortality indicator to be used across the NHS in England reports mortality at trust level across the NHS in England using standard and transparent methodology Excludes specialist, community and mental health hospitals Uses 2 linked administrative datasets Hospital Episode Statistics (HES) Office for National Statistics (ONS) death registrations Produced and published quarterly by the HSCIC First publication as an experimental Official Statistic in October 2011

4. Details of SHMI Covers all deaths reported of patients admitted to acute, non-specialist trusts who either die while in hospital or within 30 days of discharge Indicates whether a trust’s mortality ratio is as expected, higher than expected or lower than expected Not suitable as a direct measure of quality of care on its own, it is a trigger for further investigation Recommended to be used as part of a range of more detailed indicators Contextual indicators published alongside to add some context to the interpretation of the SHMI Other contextual indicator only available to trusts via previewer

5. Why we are producing SHMI National review of hospital summary mortality ratios (HSMR) Review commissioned because of concerns about different indicators in use, the lack of consistency and lack of clarity about the way some were being calculated Review looked at both technical & audience/use issues Following the recommendations from this review, the Department of Health committed to implementing the SHMI as the single hospital-level mortality indicator which could be adopted across the NHS

6. How SHMI was developed and is governed A steering group to define the high-level requirements A detailed independent statistical modeling and analysis exercise carried out by ScHARR, University of Sheffield A technical group to agree on the specifications HSCIC commissioned to lead the continued development and improvement of the SHMI, working with a range of stakeholders as well as publishing on a quarterly basis Annual review through the Indicator Assurance Pipeline Process

7. Details of SHMI Observed Deaths the number of patients who die following treatment at the trust Expected Deaths the number of patients who would be expected to die on the basis of average figures for England, given the characteristics of the patients treated there Risk adjusted for age, admission method, sex, co- morbidity and diagnosis group calculated from a full logistic regression model in SAS using MLE

8. Using Statistical Process Control Using statistical modelling (logistic regression and other generalised linear models), we develop a national baseline model which provides us with an risk estimate of patient outcome Providers who do not conform to the national baseline (with associated control limits) are indicating special cause variation This variation has not been explained by the baseline model, many possibilities for reasons why, but this warrants a follow-up

9. SPC Chart – Funnel Plot Upper Control Limits Lower Control Limits England Baseline

10. Issues Capturing and Handling Issues Log Methodology Changes Publication Feedback captured from users Issues added to Issues Log Review and updates at every publication Updates to issues announced through Methodology Changes Changes reflected on publication Documentation Outputs …

11. Issue Description 2 distinct views from technical working group 2 bandings published for October 2011 publication Confusing & undermine the value of the SHMI Methodology Changes 1 banding published from January 2012 publication although both control limits still published Issue 1: Over-dispersion

12. Issue Description 95 models converged out of 140 models Complete/quasi-complete separation and ridging errors in dataset Over and under estimating of events Methodology Changes Use 3 years data for model build Inclusion of year index for case-mix adjustment Applied SAS in-built logistic regression options Ongoing issue to be reviewed further Issue 2: Model convergence

13. Issue Description A small number of acute non-specialist trusts have hospices within their organisation Current guidance on coding for palliative care difficult to follow and subject to a variety of interpretations Methodology Changes Contextual indicator Ongoing issue to be reviewed further Issue 3: Palliative Care

14. Future Developments Use of co-morbidity(s) and/or prior-morbidity(s)Additional contextual indicators Weekend Mortality Social Deprivation … Use of more/all episodes Identification of diagnosis from R-codes Allocation of event to relevant diagnosis group Calculation of co-morbidity(s) or prior-morbidity(s)

15. Summary Overview of SHMI How SHMI was developed and is governed How issues are captured and handled How changes to the methodology are reflected and communicated Further details available on http://www.ic.nhs.uk/services/summary-hospital-level- mortality-indicator-shmi

16. Clinical Indicators Health and Social Care Information Centre TEL: 0845 300 6016 email: enquiries@ic.nhs.ukenquiries@ic.nhs.uk Questions & Answers