1. Risk assessing clinical audit findings
Anne Jones, Head of Clinical Audit and Effectiveness
Kingston Hospital NHS Foundation Trust

2. Session objectives
To share our process for introducing a risk assessment approach to clinical audit results, focusing on:
Why take a risk assessment approach?
The road to risk assessment
Undertaking the risk assessment – an example
Escalation process
Training the risk assessors
Benefits to patient care
Practical example

3. Why take a risk assessment approach?
At a trust-wide or corporate level Clinical governance describes the structures, processes and culture needed to ensure quality of the care and continuous improvement Quality governance used by Monitor to refer to how a Board should lead on quality and identify and manage risks to quality.1
CA and RM have long been associated with governance structures, whether clinical or quality.
One of the key tasks of clinical governance or quality governance is to manage risk and one of the key tools for highlighting and assessing risk is clinical audit
1 Bullivant J, Burgess R. Corbett-Nolan A. and Godfrey K. (2012) Good Governance Handbook. HQIP/Good Governance Institute

4. Why take a risk assessment approach?
At the front line To support frontline clinicians to assess the significance of clinical audit results and ensure improvement in patient care

5. Relationship between clinical audit and risk
Risk driving clinical audit 
High risk is one of the main drivers of clinical audit topic choice 2
‘Address specific local risks’ 3
At Kingston, risk registers reviewed for topics for annual clinical audit programme and risks arising throughout the year are added to the clinical audit programme
2 Burgess R. (2011) New Principles of Best Practice in Clinical Audit , HQIP
3 Bullivant J, and Corbett-Nolan A.(2010) Clinical audit: A Simple guide for NHS Boards and partners, HQIP
To: Anne Jones

6. Relationship between clinical audit and risk
Clinical audit driving risk management ?
Quantify the implications for clinical and organisational practice including potential for harm
Clinical audit can …. Identify major risk, resource and service development implications in an NHS trust ‘3
At Kingston, process for linking clinical audit to risk and risk registers had not been formally established
We wanted a system which allowed us to see the wood for the trees.
We wanted to have a better way of prioritising topics where the audit results did not meet the standard so that we could channel in resources accordingly. Risk assessment basically allowed us to compare apples and pears.
3 Bullivant J, and Corbett-Nolan A.(2010) Clinical audit: A Simple guide for NHS Boards and partners, HQIP

7. Other drivers HQIP’s Clinical audit: A simple guide for
NHS Boards and partners3
“Ensure that clinical audit results having potential significance, due to the identification of a risk, are brought to the attention of the clinical governance committee …..”
Grant Thornton’s paper ‘Clinical audit: a brave new world’? (2011)4
“Results are not fed into organisational risk assessment. We see many examples of where individuals or groups learn from clinical audit and adjust their practice accordingly. However, we see virtually no examples where the results of clinical audit feed back into the board’s risk assessment and business planning.
3 Bullivant J, and Corbett-Nolan A.(2010) Clinical audit: A Simple guide for NHS Boards and partners, HQIP
4 Dossett P. (2011) Clinical audit: a brave new world? Grant Thornton
HQIP:
Need to work out what ‘potential significance due to the identification of risk’ is
NED often have a finance/business background and would understand risk management terminology.
Grant Thornton:
Working with trust auditors led me to re-examine how we classified and presented audit results to NEDs

8. How does this fit with the current national picture?
Francis Report5 - Recommendation 5: The Board should review audit processes and outcomes on a regular basis. Keogh Report 20136: Ambition 2 The boards and leadership of provider and commissioning organisations will …have rapid access to accurate, insightful and easy to use data about quality at service line level. A failing of one of the hospitals was noted as “The governance systems are not providing the expected level of assurance to the Board, and the escalation to the Board of risks and clinical issues is inconsistent” Berwick Report The most valuable information of all is information on risks and on things that have gone wrong p10 5 Francis Report 6The Keogh Report (2013) Review into the quality of care and treatment provided by 14 hospital trusts in England 7The Berwick report (2013) Improving the Safety of Patients in England

9. What were we hoping to achieve?
To provide clinicians, managers and clinical audit staff with a system to categorise and quantify the level of concern/risk
Escalate appropriate concerns upwards
Target resources
To ensure a systematic approach where results are below standard
To drive robust action planning and quality improvement
To focus on areas where quality improvement is most required
To feed back into Board’s risk assessment and business planning

10. Kingston Hospital’s road to risk assessment
System
Timeframe
Detail
General reporting against standards
Prior to 2011
Clinicians set standard to achieve prior to audit – met/not met
RAG rating of clinical audit results
2011
Clinicians set RAG standards to achieve prior to audit starting, producing RAG audit results.
Review
Clinicians found it difficult to set amber and red objectively.
Unclear whether always flagging up the major issues or right issues.
Risk assessment of clinical audit results
2013
Clinicians set ‘green’ standard to achieve. If not achieved, risk assessment undertaken to more objectively categorise amber/red.
Particularly difficult when you have lots of audit criteria
Not sure whether flagging up major issues/right issues because of arbitrary setting of amber and red. This was important for everyone, and particularly for the trust board.
Wanted to make sure that where potential harm = red

11. 5 steps in risk assessment1.
List the hazard – something with potential to cause harm 2.
Decide who or what is affected by the hazard 3.
Evaluate risks arising from the hazard and decide whether existing controls are adequate - risk is effect of the hazard if it occurs 4.
Record the risk assessment 5.
Review the risk assessment as necessary
Ask them what experience they have?
Who/What is affected: patients, staff, the service itself, reputation, finance, environment, visitors.
Domains = Quality, compliance (eg NHSLA), reputational, strategic, financial

12. Kingston’s risk assessment form 8
8 Kingston Hospital NHS Foundation Trust (2013) Risk Identification, Assessment and Risk Register Procedure

13. Grading the consequences for audit of clinical quality1
Negligible 2
Minor 3
Moderate 4
Major 5
Catastrophic
Peripheral element of treatment/care suboptimal
Overall treatment/care suboptimal
Treatment/care of reduced effectiveness
Unacceptable quality of treatment/
care
Totally unacceptable quality of treatment / care
Minimal intervention / treatment
Minor intervention / treatment
Moderate injury requiring intervention
Major injury leading to long term incapacity
Incident leading to death
Increase in length of stay by 1-3 days
Increase in length of stay by 4-15 days
Increase in length of stay by >15 days
Likelihood scoring 1
Rare 2
Unlikely 3
Possible 4
Likely 5
Almost certain
Probably never happen
Not expected to happen but may
Might happen occasionally
Will probably happen but not persistent
Will undoubtedly happen, possibly frequently
Our Risk Identification procedure lists a number of different ‘domains’ that may have risk implications. These include patient safety, quality of care, staffing, adverse publicity, financial. The one here is an example of the domain for clinical quality, as it usually assessed in a clinical audit.
8 Kingston Hospital NHS Foundation Trust (2013) Risk Identification, Assessment and Risk Register Procedure

14. The process
At start of each clinical audit project, a target for acceptable care or treatment is agreed by the clinical team – ‘green’ RAG rating. Linked to policy, guideline or set standards. The audit is carried out and the results compared to the target(s) set. Result > target = Green Result < target risk assessment to categorise level of risk Risk Rating < 8 = Amber rated Audit Risk Rating > 8 = Red rated Audit
Still set ‘green’ but amber and red decided through risk assessment
Cut off points for amber and red decided in conjunction with Risk Management Department and Clinical Audit Group

15. Hazards (potential to cause harm):
EXAMPLE: Antibiotic prescribing
Audit criteria:
Target
Result
Indication given
95%
96%
Stop/review date recorded
70%
Duration in accordance with guideline
79%
Antibiotic prescribed according to guideline or Micro
89%
Hazards (potential to cause harm):
Review/stop dates not put on antibiotic prescription charts and patients left on antibiotics too long. Incorrect antibiotics prescribed.
Risks (effect on the patient/hospital):
Antibiotic acquired infection, eg C. difficile, increased resistance to anti-biotics, side effects from treatment, longer recovery period, increased cost
Consequence 3 = moderate – C. diff infection likely to be in hospital by extra 4+ days (moderate)
Likelihood 3 = possible – might happen or recur occasionally
Risk rating
Consequence 3 (moderate) x Likelihood 3 (possible) = 9 (RAG rated ‘Red’)

16. Safe and effective prescribing of antibiotics
Trust-wide  
Safe and effective prescribing of antibiotics 
Medicines policy
M-F Pharmacist WR
Micro/Pharmacist twice weekly WR
Stop date section on drug chart
Training for junior docs
Consultant supervision on ward rounds
Weekly publication of data
Daily Micro/Pharm WRs
Hazard: Incorrect AB
Duration of AB too long
Risks: AB acquired infection; Treatment side effects; Longer recovery; AB resistance; Cost

17. Summary of results and RAG rating
Clinical Audit report
Summary of results and RAG rating
Audit criterion
Result
Standard set %
Risk rating C x L
RAG rating
Indication for AB given
96%
95%
Stop date on chart
70%
3 x 3 = 9
Red
Correct duration
79%
Correct antibiotic
89%
Action Plan
Recommendation
Action
Barriers
Responsibility
Timescale
Train junior doctors
Add to induction programme
None
F1/F2 tutor
Ensure drug charts are fully completed
Supervision on ward rounds
Consultants
Weekly
Ensure staff know results
Publication via KPI scorecard
Audit Facilitator
1st of each month
Extend input from Micro/Pharmacy
Increase Antibiotic Pharmacist to FTE
Budget restrictions
Pharmacy Manager

18. Escalation of ‘red’ audit results
Clinical audit results
Reported to Clinical Audit Group and Clinical Division
Risk Management Committee
Reported to Quality Assurance Committee*
Escalation of ‘red’ audit results
Red
◄ Challenge ►
CAG/CD – clinical staff. Escalated to Risk Leads in specialties and divisions.
RMC – Senior management
QAC – NEDs
QAC oversight of what RMC is doing
Risk Register
* Sub Committee of the Trust Board

19. An iterative process …. Original Revised
Risk assessment for all criteria that did not meet standard
Risk assessment of findings as a whole
Undertaking a ‘quick’ risk assessment and proceeding to full risk assessment if risk >8
The full risk assessment was not lengthy and this two-step process made it more complicated

20. Training
Trust-specific training delivered to Clinical Audit Facilitators by Risk Manager
Training delivered to our Clinical Audit Leads by Risk Manager and Head of Clinical Audit
Audit project leads trained by Clinical Audit Facilitators at the time of undertaking a risk assessment
All managers and senior nursing staff trained in risk assessments

21. Your turn ……..

22. Falls Audit example Nursing / Trustwide   Describe the topic area 
Existing controls:
Falls Policy
Audits
Staff training
Analysis of individual falls
Identify the hazards
? ? ?
? ? ?
List all the risks

23. Falls Audit example Nursing / Trustwide   Falls care plans 
Hazard: Failure to provide care plans for all patients at risk of falls
Existing controls:
Falls Policy
Audits
Staff training
Analysis of individual falls
Risks: Physical harm from fall; increase in LOS, increase in falls per1000 bed days

24. Conclusions
Greater clarity regarding the level of risk to patient care where clinical audit results do not meet standards – good for patients, clinicians and the Trust Board
Greater objectivity than previous system
Provides ‘at a glance’ results and allows clinicians, senior staff and Board members to concentrate on the things that matter
Change and quality improvement to patient care is enhanced by ensuring action plans are robust and staff are held to account for completion, via risk register
Work in progress
Building into overall business planning
Challenged re re-audit dates and progress with action plan

25. Thank you for attending this workshop Any questions