1. www.consultmarklevy.com Audit – how to do it Dr Mark Levy FRCGP Member GINA Executive Clinical Research Fellow, Edinburgh University Editor-in-Chief, Primary Care Respiratory Journal

2. © Mark Levy www.consultmarklevy.com Implementing guidelines into practice – using Audit (Become a reflective practitioner) Dr Mark Levy FRCGP Member GINA Executive Clinical Research Fellow, Edinburgh University Editor-in-Chief, Primary Care Respiratory Journal

3. www.consultmarklevy.com How to do a Medical Audit: Lecture Plan Experiential learning The audit cycle Create a new audit – example ‘Off the Shelf’ Audits Respiratory examples – can apply the principles to other medical disciplines

4. www.sign.ac.uk/guidelines/fulltext/102/index.html

5. G IN A lobal itiative for sthma lobal itiative for sthma

6. www.consultmarklevy.com Putting asthma guidelines into practice – using Audit (Become a reflective practitioner) Reflective practice and experiential learning What is audit? Have a problem?........consider doing an audit? Designing an Audit Doing it

7. www.consultmarklevy.com www.animalswild.com

10. www.consultmarklevy.com

12. Experiencial Learning I hear and I forget Confucious 551 – 479 B.C.E I see and I remember I do and I understand

13. www.consultmarklevy.com Experiencial Learning Personal involvement Self initiated Sense of discovery Makes a difference in behaviour Evaluated by learner Carl Rogers (January 8, 1902 – February 4, 1987) “Everyone has the potential to grow”

14. www.consultmarklevy.com Experiencial Learning :– First Medical Audit Crimean war 1853-1855 Scutari 1854 -unsanitary conditions /high mortality Strict sanitary conditions Records – mortality fell from 40% to 2% Florence Nightingale 1820-1910

15. www.consultmarklevy.com Experiencial Learning ‘Experience for me is the highest authority’ ‘The facts are friendly’ –Learn even if you are wrong –Learn by doing Carl Rogers (January 8, 1902 – February 4, 1987)

16. www.consultmarklevy.com Medical Audit … experiential learning is equivalent to personal change and growth Experience without reflection is not learning Ernest Anthony Codman (1869-1940)

17. www.consultmarklevy.com History of Medical audit i)Medical auditing by scientific methods; illustrated by major female pelvic surgery. Lembcke PA. J Am Med Assoc. 1956 Oct 13;162(7):646-55. ii)A scientific method for medical auditing. Lembcke PA. Hospitals. 1959 Jul 1;33(13):65-6 iii)Evolution of the medical audit. Lembcke PA. JAMA. 1967 Feb 20;199(8):543-50.

18. www.consultmarklevy.com

19. NHS and Audit 1989 White Paper, Working for patients - standardise clinical audit as part of professional healthcare...."the systematic critical analysis of the quality of medical care including the procedures used for diagnosis and treatment, the use of resources and the resulting outcome and quality of life for the patient.“........ 1992/3 GP contract Evidence for appraisal and ? revalidation

20. www.consultmarklevy.com The audit cycle Identify Problem / Learning objective Set standards for achievement Collect and analyse data Reflect on achievement of standards / identify areas for improvement. Make changes Re-audit after appropriate interval Reflect on results / identify learning needs Agree Criteria (data to be collected)

21. www.consultmarklevy.com The audit cycle Identify Problem / Learning objective Set standards for achievement Collect and analyse data Reflect on achievement of standards / identify areas for improvement. Make changes Re-audit after appropriate interval Reflect on results / identify learning needs Agree Criteria (data to be collected)

22. www.consultmarklevy.com Underdiagnosis and undertreatment 2700 children 11% had asthma < 1% previously diagnosed 66% never had bronchodilators 30 % lost > 50 schooldays 1982 Prevalence <4% BMJ 1983;286:1253 - 6

23. Delayed diagnosis of asthma Levy & Bell BMJ 1984 ; 289 : 1115 - 6

26. www.consultmarklevy.com Methods: ‘Real world’ study comparing 1339 patients of all ages and severity of asthma managed by 77 self- selected highly motivated general practices in Scotland with 9617 patients from 319 practices from a national sample

27. www.thepcrj.org Pragmatic real world study of the effect of audit of asthma on clinical outcome: Results More structured reviews More inhaler technique checks Fewer unscheduled consultations Fewer symptoms No significant difference in BTS Step or A&E utitilisation Neville R et al. Prim Care Resp J 2004;13:198-204

28. www.consultmarklevy.com Types of audit – retrospective or prospective? Retrospective: Classically, many people believe that audit should be done retrospectively, ie through analysing past clinical management behaviour. This is one of the acceptable methods for establishing a baseline, and should be done to establish current practice and generate new audit or research questions. This method is also used to repeat an audit (or close the loop or cycle after appropriate intervention following the baseline audit). However, this type of audit cannot always reflect actual practice, mainly because of limitations of collecting retrospective data which often has missing data. Prospective audit is valuable to investigate implementation of new practice following a baseline audit, and also particularly useful for sessional/ freelance doctors and nurss who don’t have a regular place of work.

29. www.consultmarklevy.com Can Sessional, locum or freelance Doctors do audit? They certainly can Retrospective (if they have a regular place of work, with their own computer log on) or Prospective audit which may be more practical

30. www.consultmarklevy.com The audit cycle Identify Problem / Learning objective Set standards for achievement Collect and analyse data Reflect on achievement of standards / identify areas for improvement. Make changes Re-audit after appropriate interval Reflect on results / identify learning needs Agree Criteria (data to be collected)

31. www.consultmarklevy.com Becoming a reflective practitioner – Audit is the tool Consider a question (PUN/ Den) Decide to audit our work

32. www.consultmarklevy.com Planning an audit (1) Decide on aims/outcomes (SMART Specific, Measurable, Achievable, Realistic, Timeframe) Set standards (what are you going to measure) Selection of patients (eg Age/presentation) How many patients Criteria

33. www.consultmarklevy.com Criteria and standards Criteria = are those aspects of care that you wish to examine – the data you will collect Standards = levels of success that you wish to achieve –Eg – Acute asthma : patients treated with high dose bronchodilator – expect > 80% to also be prescribed oral steroids.

34. www.consultmarklevy.com Planning an audit (2) How are you going to collect the data How long are you going to collect data Who will do the audit? How are you going to analyse the data Plan for meeting to discuss data Agree : - new standards / new audit (timeframe)

35. www.consultmarklevy.com

36. Asthma: Patient AMP, Female, Age 7, consults: She has a cough and has been using a prescribed Ventolin inhaler – you assess her and decide that she has got a cold You notice she hasn’t been put on the asthma register Nor has she been given a self management plan Nor has she been told she has asthma You wonder if indeed she has got asthma..

37. www.consultmarklevy.com Signs,symptoms and findings which increase the probability of asthma in children  wheeze  cough  dyspnoea  chest tightness  Symptoms are frequent and recurrent  Symptoms are worse at night or in the early morning  Symptoms often occur in response to exercise and/or emotion  Symptoms may occur as a response to triggers such as pets, pollens, cold or damp air  Patients may have a history of atopic disease  Patients may have a family history of atopic disease and/or asthma  Widespread wheeze may be heard on auscultation  Variable airflow obstruction (PEF varies by more than 20%) BTS/SIGN /www.sign.ac.uk/

38. www.consultmarklevy.com Aim: To assess quality of my diagnosis of asthma in children; according to The British Asthma Guidelines 1)How do I define ‘diagnosis of asthma’ for the purpose of this audit? 2) What do I mean by ‘Children’? Definition of Diagnosed Asthma : Prescription of asthma medication implies a provisional or confirmed diagnosis of asthma Children = Patients aged 6 - 16

39. www.consultmarklevy.com Aim: To assess quality of my diagnosis of asthma in children, according to The British Asthma Guidelines Standards: i) All diagnosed with asthma have at least one symptom (wheeze, cough, difficulty breathing, chest tightness) ii) At least 70% diagnosed patients had night symptoms iv) Over 60% of diagnosed children should have history of atopy vi) > 70% should have proven variable airflow obstruction ( eg PEF > 20% variation)

40. www.consultmarklevy.com Selection of patients and duration of audit Who: Which patients: When: How Many:

41. www.consultmarklevy.com Selection of patients and duration of audit Who: Myself and members of my peer group Which patients: All patients 6-16 years who I diagnose with asthma or prescribe asthma medication for. When: From 1/11/2010 to 1/2/2011 How Many: Until at least 10 patients are recruited

42. www.consultmarklevy.com Criteria: data needed to collect info: Name Annonymous ID Date presented Variable respiratory symptoms (cough, wheeze, diff breathing, chest tightness ( - yes/No/ Not asked) Night symptoms: ( - yes/No/ Not asked) History of atopy (eczema, hay fever) PEF readings: Previous best, highest, lowest

43. www.consultmarklevy.com Make forms Patient selection form Data collection form Results reporting form

44. www.consultmarklevy.com Patient selection form NameComputer ID Date seen RefDOB/ AgeDiagnosed asthma Prescribed.../.../201_ By me/ previously Inhaler Monteleukast.../.../201_ By me/ previously Inhaler Monteleukast By me/ previously Inhaler Monteleukast

45. www.consultmarklevy.com Data Collection Form

46. www.consultmarklevy.com Data Collection Form

47. www.consultmarklevy.com Report form

48. www.consultmarklevy.com Reflection ? Peer meeting / self reflection What went well What went badly Which Standards did I meet? What do I need to do? Plans to re-audit and complete the cycle

49. www.consultmarklevy.com Report

50. www.animalswild.com

51. www.consultmarklevy.com www.guideline-audit.com

52. www.consultmarklevy.com Guideline-audit.com - unique features Online FREE system for audit Immediate feedback Comparison – individual clinicians' changes – other clinicians world wide Anonymised Only registered users, password protection see their own data for comparison with others Unregistered users see the amalgamated data for everyone, without individual comparison

53. www.consultmarklevy.com www.guideline-audit.com

54. www.consultmarklevy.com Chest Infection Audit Standards Nice Guideline 69 www.nice.org.uk ; Prim Care Resp J 2010;19(1):21-27. DOI: http://dx.doi.org/10.4104/pcrj.2010.00014

55. www.consultmarklevy.com CRB-65 and Mental Test Score CRB-65 score : 0= low risk; 1-2 = increased mortality); 3-4 >9% mortality)

56. www.consultmarklevy.com Patient Data collection Form

57. www.consultmarklevy.com Easy Online Data Entry with drop down response choices

58. www.consultmarklevy.com Results available immediately – with comparison with peers

59. www.consultmarklevy.com Print out a report for appraisal

60. www.consultmarklevy.com

61. Optimal Respiratory Health: Leading the Way Primary Care Respiratory Society UK– www.pcrs-uk.org

62. Levels of Asthma Control Characteristic Controlled (All of the following) Partly controlled (Any present in any week) Uncontrolled Daytime symptoms None (2 or less / week) More than twice / week 3 or more features of partly controlled asthma present in any week Limitations of activities NoneAny Nocturnal symptoms / awakening NoneAny Need for rescue / “reliever” treatment None (2 or less / week) More than twice / week Lung function (PEF or FEV 1 ) Normal < 80% predicted or personal best (if known) on any day ExacerbationNone One or more / year 1 in any week

63. www.animalswild.com

64. www.guideline-audit.com Any queries marklevy@consultmarklevy.com Further information: www.ginasthma.orgwww.ginasthma.org International Asthma Guidelines - GINA http://www.sign.ac.uk/guidelines/fulltext/101/index.htmlhttp://www.sign.ac.uk/guidelines/fulltext/101/index.html UK asthma Guidelines www.gpiag.orgwww.gpiag.org UK special interest group - respiratory www.theipcrg.orgwww.theipcrg.org – International special interest group - respiratory http://www.asthma.org.uk/http://www.asthma.org.uk/ - UK asthma charity – Asthma UK

66. Results: Snapshot Audit > 25% of patients presenting with asthma exacerbations had not been previously diagnosed Only 4 patients (in the two audits) had been prescribed at least 3 preventer inhalers in the 3 months preceding their exacerbation 36/56 (63%) of the exacerbations in the two audits were treated with high dose short acting beta-2-agonists. – However only 12 (33%) of these exacerbations were treated with oral steroids. Follow-up was offered for 55% of these exacerbations ATS Abstract /Poster 2005

67. www.consultmarklevy.com London 2008 www.guideline-audit.com Any queries marklevy@animalswild.com

68. System for auditing management of uncontrolled asthma including acute attacks Patient consults for uncontrolled asthma Participants register Baseline audit

69. www.consultmarklevy.com Checked inhaler technique

70. www.consultmarklevy.com

71. UNSAFE (Unscheduled attendance at general practice surgeries for asthma episodes) Internet based international audit Examines management of unscheduled asthma attendances Provides benchmarking system for clinicians to compare their management of these patients (peer/guidelines) Provides report for inclusion in CME/CPD folder for appraisals www.guideline-audit.com

72. www.consultmarklevy.com Medical audit An educational activity Promotes understanding Resource effective Raises standards Promotes change Ensures reflective practice.....

73. Interrested?? www.guideline-audit.com

74. www.animalswild.com