Audit, Service Evaluation and Research Midhun Mohan STARSurg Steering Committee Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Learning Objectives Similarities and differences The audit cycle Audit standards Implementing change How to get involved Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Who’s done one before? Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Audit – Definition Clinical audit is a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change. Aspects of the structure, process and outcome of care are selected and systematically evaluated against explicit criteria. Where indicated changes are implemented at an individual, team, or service level and further monitoring is used to confirm improvement in healthcare delivery Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Audit – Definition Clinical audit is essentially all about: checking whether best practice is being followed making improvements if there are shortfalls in the delivery of care How do we decide what best practice is? AUDIT STANDARDS “A standard is an explicit statement describing the quality of care to be achieved, which is definable and measurable” Audit standards are derived from: National Guidelines Royal college guidelines NICE guidelines Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Audit – Standards Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Service Evaluation Designed and conducted solely to define or judge current care Designed to answer: “What standard does this service achieve?” Measures current service without reference to a standard. No allocation to intervention: the doctor and patient have chosen intervention before service evaluation No randomisation Does not require REC review Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Audit Designed and conducted to produce information to inform delivery of best care. Designed to answer: “Does this service reach a predetermined standard?” Measures against a standard. No allocation to intervention: the doctor and patient have chosen intervention before audit. No randomisation. Does not require REC review. Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Research Attempts to create generalisable new knowledge regarding best practice. Tests hypotheses that evaluate or compare interventions. Usually large scale over a long time period. May involve patients receiving a completely new treatment. May involve patients being given different treatments. Uses random representative sampling methods to ensure generalisability of findings. Extensive statistical analysis is usually required. Results need to be generalisable to a wider population. Always requires ethics committee approval. Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Overview Audit - is designed to answer the question "Does this service reach a predetermined standard? Service evaluation - is designed to answer the question "what standard does this service achieve?” Research - is designed and conducted to generate new knowledge and should follow the systems for approval of NHS Research Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

The Audit Cycle Referred to as a cycle or a spiral systematic process Each cycle aspires to a higher level of quality Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Step 1 Identify problem and the local resources available for audit You are an F1 on the respiratory ward One month into your attachment, you notice that patients with COPD who met the criteria for starting non-invasive ventilation had not been given this treatment. – PROBLEM IDENTIFIED You tell your consultant – who tell you to do an audit You discover the hospital has an audit lead and clinical audit office. You contact them and them and they say they are happy to help Go to the office rather than calling – much easier to get things sorted In agreement with your local audit lead, you choose the British Thoracic Society guidelines on non-invasive ventilation in acute respiratory failure and find these on the society’s website. Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Step 2 – Audit Standard Determine what you are trying to measure and define gold standards All patients with an acute exacerbation of COPD with respiratory acidosis (pH <7.35), despite maximum medical treatment on controlled oxygen therapy should receive treatment with non-invasive ventilation After discussion with the respiratory team, you choose a standard of 100% Standard does not have to always be 100% 100% may be unrealistic in some environments Discuss with your team when deciding on percentage Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Step 3 – Measuring level of performance Collect data – paper notes, computer, theatre logs. Ask if unsure! You decide to collect data from the medical notes of the last 50 patients admitted with an acute exacerbation of COPD. You ask the audit office to obtain the notes for you You make an audit performa: What data you are going to collect depends on the audit standard Where are you going to store it? Speak to someone about this Does the patient have a respiratory acidosis (pH <7.35) despite maximum medical treatment on controlled oxygen therapy? – YES/NO Where they treated with non-invasive ventilation – YES/NO If they were not treated with non-invasive ventilation, was a reason given? YES/NO Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Step 4 – Compare performance with standard You analyse the data (SPSS, R, Excel) 40/50 patients had respiratory acidosis despite maximum medical treatment on controlled oxygen 31 of these 40 patients (77.5%) were treated with non-invasive ventilation This percentage did not meet the standard of 100% So you start looking into the reasons why non-invasive ventilation hadn’t been started on 9 people You decide the main reasons were because: Lack of enough machines Lack of medical knowledge from medical staff on how to use it Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Step 5 – Making improvements You present the audit to medical and surgical departments Feeding back crucial! No point in doing it if you don’t present it back and propose improvements Educate all medical staff on the interpretation of ABGs and use of non-invasive ventilation. How? – Propose tutorials and teaching sessions Display posters on when to use non-invasive ventilation in patients COPD Speak to your medical director about developing a business plan to increase the number of non-invasive ventilation machines in the hospital Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Step 6 – Sustaining improvements Repeat audit After a period of time to see if the suggested improvements have made a difference You re-audit after a year You now find 90% of patients are meeting the standard! Great news – there is an improvement! Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Step 3 – Audit cycle summary Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England Identify problem or issue Set audit standard Collect data on practice Compare performance with audit standard Make change & Re-audit

How do I get involved as a medical student Speak to your consultant Ask for projects Very happy to help keen students Ask foundation doctors They all have to do one for their portfolio During intercalation STARSurg ( GlobalSurg ( Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England

Question: Which study type? Assess all operative records for emergency hernia repair in my hospital over one year to check whether the nerve status is documented Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England a.Which study type is this? b.Why? c.Is it likely to require ethical approval?

Question: Which study type? Randomly assign patients undergoing emergency hernia repair in my hospital to receive a mesh implant or no implant and follow up outcomes Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England a.Which study type is this? b.Why? c.Is it likely to require ethical approval?

Thank you Protocol Launch Meeting and Research Skills Course September 16 th 2015, RCS England Research asks “Are we singing the right song?” Audit asks “Are we singing this song right?”