New Data Validation Report (WOVEN) – Ian Bullard, NHS Information Centre

“We can only be sure to improve what we can actually measure” Lord Darzi, High Quality Care for All, June 2008 “...our greatest mistake would be to forget that data is used for serious decisions in the very real world, and bad information causes suffering and death.” pg 46 Bad Science, Ben Goldacre “Collect once, use many times”

ESR Data Quality/Validation – the past 3 levels ESR Operational –Organisation driven – responsible for their own data quality –Run at organisation discretion ESR Data Warehouse –Dashboard reports (not validation) –Cover key areas –For interest at organisation level and DW users –Run at organisation or DW user discretion IC –Data used to feed official publications and management information (NHS England only, Stats Wales publish data for NHS Wales) –Demonstrate increased confidence in ESR data –Provide evidence of increasing quality over time All 3 developed in partnership (ESR, IC, NHS) but occasionally conflicts / confusion had arisen

The WOVEN data quality cycle Monthly summary and detailed report for each Organisation WOVEN 2 was introduced in August 2011 replacing the McKesson ESR data quality reports  Monthly DQ report pushed out to users keeping them informed of local DQ issues  Timely data – direct from local ESR  More comprehensive validations with potential to add, remove or alter  Option to exclude valid anomalies  All Wales summary  Wales and England NHS Organisations for greater benchmarking  Ranked relative to peers (out of 422 Orgs)  Ensure improved data quality for all ESR based collections eg Census  Direct feedback to Organisations still need to run compliance reports to ensure that CRB checks are in place etc. Identify poor performers – Organisations to work in partnership with WfIS Data Standards

The WOVEN data quality cycle Identify systemic issues that may be addressed by changes in ESR, data standards or guidance: –NWD 2.4 in December 2011 Informatics Staff Area of Work (AoW) improvements; Add N8L Nursery Nurse in Neonatal Nursing; Remove Assistant Practitioner in Pharmacy –Healthcare Scientists improvements to follow in 2012 –Guidance documents continually being reviewed Currently fundamentally overhauling the Job Role / Area of Work Guidance on KBase & IC website in future Occ Code / JR / AoW ‘Decision Tree’ WOVEN Guidance

The WOVEN data quality cycle Flat report (without date limits eg for equality data items) sent to ESR central team to maintain some historic DQ reporting. These were sent to NHS Wales organisations in October 2011 Option to exclude valid anomalies – standard process ESR Management Reporting SIG managing the validations now the system is live –All validation amendments (add, remove, alter) via local MR SIG –Minor cosmetic changes by request to IC – validation text

All Wales Summary WOVEN Report

NHS Wales WOVEN Ranking – Oct 2011 Total of 422 organisations in Wales and England NHS Wales organisations have been included in the WOVEN2 data quality testing process since August 2011, WOVEN has been operating in NHS England for about 2 years NHS Wales was part of the pilot process for WOVEN2 NW, Yorkshire and Humber have dedicated (funded) Data Quality projects Org CodeOrganisation NameFinal ScoreRanking RT4Welsh Ambulance Services NHS Trust A1Betsi Cadwaladr University LHB A7Powys Teaching LHB RQFVelindre NHS Trust RYTPublic Health Wales NHS Trust A3Abertawe Bro Morgannwg University LHB A2Hywel Dda LHB A4Cardiff & Vale University LHB A6Aneurin Bevan LHB A5Cwm Taf LHB

What Queries are impacting on our WOVEN scores? New Starters data collection – particularly collecting Equality Data for new starters (50% blank records across NHS Wales) and Recruitment Source (62% blank records across NHS Wales, this varies from 7% to 75% blank records) Assignment Category – Permanent, Fixed Term etc (0.5% blank across NHS Wales, this varies from 0 errors in one organisation to over 270 errors in another organisation) Destination on Leaving & if NHS the name of NHS Organisation - (30% blank across NHS Wales, varies from 0% to 100% blank records)

Why ESR data quality matters AMQs and FoIs highlight the ‘margin of error’ required in ESR and Census data and can lead to reputational damage for organisations and individuals due to poor quality data. –PQ (NHS England) last year asked for the number of school nurses in a PCT. –Census data reported zero, which the PCT had signed off as part of their Census submission –upon receiving this PQ the PCT indicated to DH it had made some coding errors and wanted to correct its Census figures. –IC policy (Official Statistics) is that unless the impact is significant at national level figures are not changed, post publication, hence the need for organisations to continually monitor their data quality. ESR data is used for workforce planning to evidence where targets have been met, e.g. commitment to increase the number of health visitors and introduction of family nurse partnerships in NHS England. AQF Workforce Targets for NHS Wales eg 10% move to Community Using ESR data helps to replace burdensome central collections and continuous monitoring can occur.

Why ESR data quality matters Census data –Organisations need to follow the WfIS Data Standards and ESR guidance eg Locum Medical staff, correctly coding hosted staff and those staff who work in a different organisation to their employer and Management Costs –Continue to monitor workforce data to support Census In NHS England ESR data is now feeding a number of QIPP benchmarking tools and therefore Trusts are being compared on their efficiency and productivity based on the ESR data they currently hold. –Trusts may be embarrassed when poor quality data indicates they are initially shown to be an outlier within these benchmarking tools –Highlights need for better linking of workforce and activity standards. In NHS Wales Workforce & OD Measures are being developed

Next Steps for the IC: Improve standards, guidance and ensure the DQ process for Organisations continue to be refined Reduce occasions for data collection direct from organisations. Improved Medical and Dental guidance –GMC numbers are key –Hosted staff and workplace org fields in ESR need to be correct –Locums and Honoraries – WIRG taking forward Job Role / Area of Work guidance –Provide examples of how data standards can link together –Developing a decision tree to aid Trusts navigate through the data standards TCS and other consequences of changes to the NHS England –Numerous reporting streams, potential for increased data confusion and reporting of ‘NHS’ workforce outside of the core NHS What do you want from us?

Next Steps for you: Monitor and act on your monthly WOVEN DQ reports –Increase DQ checking and correcting on an ongoing basis –Increase DQ checking and correcting in response to IC / NLIAH queries –Increase DQ checking for things that seem wrong, act on specific IC / NLIAH feedback – eg old Occ Codes etc. Work with your department leads to ensure ESR is accurately capturing the workforce Continue to develop local Data Quality activities supporting local and WfIS Data Standards Check the Benchmarking Group you have been allocated to – contact the iView team if you don’t agree Contact the IC workforce team for any help and feedback on guidance issues or to suggest improvements to existing data standards: for WOVEN, for data standards, for

Conclusion Improving data quality relies upon collaborative working – you cannot achieve it alone! Good data quality in administrative systems means we can stop burdensome collections on the service – saving time and money which can be better applied elsewhere. Acceptance of ESR data as authorative by users at all levels. Ultimately quality data will save money for the NHS by improving decisions (taken locally and nationally)