1. Falling in Love with Audit
Dr Ian Livingstone
Saltaire Medical Centre

2. Aims & Objectives To help us learn how and why to do Clinical Audit.
By the end we should be able to
define clinical audit
define jargon like “criteria” and “standards”
Plan your next audit
be able to make any changes needed as a result of your audit.
Feeling that audit is useful
Aims
To help us learn how and why to do Clinical Audit and leave feeling inspired to do at least 2 audit projects a year.
Objectives
By the end we should be able to
define clinical audit
define jargon like “criteria” and “standards”
explain how to choose a subject for an audit
know how to collect data for your audit
be able to make any changes needed as a result of your audit.

3. Why do clinical audit? Trainer reappraisal NHS reappraisal
Teach registrars
Why do clinical audit?
Trainer reappraisal and it’s what training practices are supposed to do
NHS reappraisal
Teach registrars, summative assessment, nMRCGP, something for their CV
To measure and improve the quality of care and service - clinical governance

4. Why do clinical audit?
To measure and improve the quality of care and service - clinical governance
Most importantly of all the reason why we really bother is the patients

5. Clinical governance
‘a new system in NHS Trusts and primary care to ensure that clinical standards are met and that processes are in place to ensure continuous improvement’
Clinical governance is a system for improving the standard of clinical practice.
It was first described in a government white paper as

6. NICE states that:
“Clinical Audits monitor the use of particular interventions, or the care received by patients, against agreed standards. Any departures from ‘best practices’ can then be examined in order to understand and act upon the causes.”  

7. an improvement in the quality of service
improved care of patients
enhanced professionalism of staff
efficient use of resources
aid to continuing education
aid to administration
accountability to those outside the profession
Clinical audit aims to lead to an improvement in the quality of service providing:-
improved care of patients
enhanced professionalism of staff
efficient use of resources
aid to continuing education
aid to administration
accountability to those outside the profession

8.  The Audit Cycle:
Can you draw the audit cycle?
                      

9. The Audit Cycle: its basic form

10. The Audit Cycle: slightly more detail
Observe Practice
Define Criteria & Standards
Compare Results to Standard
Implement Change
repeat the cycle

11. The Audit Cycle: if u wanna get serious

12. Stages in audit Stage 1 - Deciding what to audit
Stage 2 – a Setting criteria
b Setting standards
Stage 3 - observing practice
Stage 4 - Compare results with standards
Stage 5 - Implementing change
Stage 6 - repeating the cycle

13. Stage 1 - Deciding what to audit
Identifying problems, choosing a topic
Choose something that interests you.
Is the problem common?
Does it have serious consequences?
Can I do something about it?
Choose something that interests you.
Check with others that would be involved with making changes – do they agree with your proposal? This is important – you won’t get changes made if you don’t carry people with you from the start.
There’s no point in auditing something that you think is already being done well – you will find plenty of problems that need to be sorted out first!
Setting priorities
You may come up with a number of possible subjects to audit.
To help yourself prioritise, ask yourself:
Is the problem common?
Does it have serious consequences?
Can I do something about it?
Important clinical events“Significant events”Patients' complaintsObservationObservations of staff NICE subjects
Potential for change
Relevant to the practice
     Tips: Try making the audit title a question. For example, if your criterion is "All patients on Viagra should have had a serum rhubarb done in the last year", a good audit title would be: "Have all patients on Viagra had a serum rhubarb in the last year?"
            If your audit results from a problem you've noticed in the practice, document it in your write-up; it suggests that there is a potential for change.
            Think about whether you will be able to make any changes that are needed as a result of the audit. If you know you won't be able to make any changes, there's not much point in doing the audit.

14. Stage 2 - Setting criteria and standards
“An audit criterion is a specific statement of what should be happening.” = THE IDEAL
“All patients with xxxxx should have had a yyyyy in the last zzzzz year/months.”
“An audit criterion is a specific statement of what should be happening.”
→ Examples:
"All eligible women aged should have had a cervical smear in the last 5 years."
“All asthmatics should have had a Peak Flow recorded in the past year.”
“All drugs in our doctors’ bags should be in-date.”
     Tips:
Make sure that there is evidence for your criterion – do a literature search.
Ensure that the criterion is measurable –
·            “asthmatics should have had yearly peak flows” is difficult to measure (how many years will you go back?)
·            “asthmatics should have had a PF recorded in the past year” is more practical
For quickest results, make sure that what you are doing is fairly easy to measure, e.g.’ is Read-coded, though don’t let other data-gathering methods put you off if you you’re really interested in the subject
Don’t try to audit too many criteria at once – one or two will keep you busy enough.
Try filling in the gaps of the following phrase to set your audit criterion:
“All patients with xxxxx should have had a yyyyy in the last zzzzz.”
Relevant to audit subject and justifiable, e.g. current literature
     Tips: Don't make your audit complicated or overambitious! Often the best audits are the simplest; ones with too many/too complex criteria tend to run into trouble. I suggest you limit it to one or two criteria.
            Make sure that your criteria are clearly stated and unambiguous - try using the one on the Setting audit criteria page as a model.
            You must justify your criteria with references. You will need to do a literature search

15. Stage 2 - Setting criteria and standards
“An audit standard is a minimum level of acceptable performance for that criterion.”
The standard should reflect the clinical and medico-legal importance of the criterion or should be something that is realistic to achieve eg if only 70% of your patients are on aspirin post MI, you might want to go for 80% this year, 90% the year after, and 95% the year after that etc
Standards may change with time
eg 90% of patients with an myocardial infarct should be on aspirin
“An audit standard is a minimum level of acceptable performance for that criterion.”
→ Examples:
"At least 80% of eligible women aged should have had a cervical smear in the last 5 years."
“At least 60% of asthmatics should have had a Peak Flow recorded in the past year.”
“100% of drugs in our doctors’ bags should be in-date.”
The standard should reflect the clinical and medico-legal importance of the criterion.
in the example above, 80% of women should have had a cervical smear,
But of those who've had an abnormal smear, 100% should have had action taken.
Standards may change with time:
70% of patients with IHD should have a cholesterol <5mmol/l  - this is pertinent clinically today because the NSF says so
but in the future a higher standard may be expected as a result of clinical governance.
     Tips
Some criteria are so important that they need 100% standard.
However, 100% standards are unusual – patients or circumstances usually conspire against perfection and the standard needs to reflect that.
Your literature search should give you an idea of what standards others have managed to reach.
Your standard needs to follow on directly from your criterion – for example,
“Patients on thyroxine should have had TSH done in the last year; this should have happened in at least 90% of patients”.
Make sure that you can justify your standard. If you can't find any literature evidence to back it up, explain why you chose the percentage that you did.

16. Stage 3 - observing practice
What is happening?
What other ways are there to gather data for audits?
You can collect information from:
computer registers
review of contents of medical records
questionnaires – patients, staff or GPs
data collection sheets
     Tip
Delegate! You don’t have to collect all the data yourself. Most practices have a member of staff who can help you set up the search or organise questionnaires.
Audit will only result in change if you involve all the relevant team-members from the start. This may include the GPs and Nurses who will have to implement any changes, as well as office staff who can help you doing a computer search. Again, you will need to document this.

17. Stage 4 - Compare results with standards
The comparison is the easy bit
What reasons are there for practices not meeting audit standards? – planning change
You might not have achieved your standard because of an organisational problem for instance
Before you start planning changes, you need to work out possible reasons why the practice hasn’t met the standard you’ve set.
What reasons are there for practices not meeting audit standards?
In recent audits at our practice, reasons have included:
Practice reasons:
·         - Results having been put down as free text on computer, rather than coded;
·         - Opportunistic rather than formal recall system in use;
Doctor reasons:
·         - Not all GPs were aware of the practice policy;
·         - Not all partners agreed with the policy;
Patient reasons:
·         - Patients refusing to have tests done;
·         - Patients on holiday when tests due.

18. Stage 5 - Implementing change
Medical audit shows what changes are needed
actually making the changes is the most difficult part of audit!
What are we proud of?
What are we not so proud of?
Introducing change
Medical audit shows what changes are needed:
·         it isn't a method for changing care;
·         actually making the changes is the most difficult part of audit!
Actions to remedy identified deficiencies
Emphasise what has been achieved.
What are we proud of?
What are we not so proud of?
How can we correct any deficiencies?
Changes must be practical!
How are you actually going to make the changes?
Simply saying “We’ve got to do better” won’t result in change
You need to think through in detail
      - what needs to be done
      - who’s going to do it
      - when
      - and how.
Keep a record
Keep brief written record of:                       
The reason for doing the audit;
Criteria and standards;
Results
Plans for change
Action taken
When/whether to repeat the audit.
Just telling people to do things better won't result in change. You need to write up in some detail how the changes will take place.
Example

19. Stage 6 - repeating the cycle
"Closing the loop"
Re-evaluate care to ensure that any remedial action has been effective
"Closing the loop"
Re-evaluate care to ensure that any remedial action has been effective.
Audit is a continuous cycle – if you didn’t meet the standard and you’ve planned changes, you’ll need to repeat the audit to make sure the changes have happened.
When
Keep a record
Audit meetings every 4 months, follow up old ones, plan new ones, inspire those around you

20. Audit may Be seen as a threat.
Be seen as an unpleasant, time-consuming distraction from day-to-day practice.
Antagonise if introduced in an insensitive way.
Give the impression of implied criticism.
Audit may -
Be seen as a threat.
Be seen as an unpleasant, time-consuming distraction from day-to-day practice.
Antagonise if introduced in an insensitive way.
Give the impression of implied criticism.

21. Audit is an easy technique
is effective at improving care.
Not be a menace or a means of discipline.
Audit must -
Assist staff.
Be effective at improving care and assist staff
Not be a menace or a means of discipline.
Have a clear purpose.
     Tip: use the no-blame approach, can change the rules, dates, size
Respect the individual skills of staff - they know their own jobs better than anybody else.
Create an atmosphere in which mistakes can be admitted without blame.
Audit in the right atmosphere can be fun!

22. Audit in the right atmosphere can be fun!
A useful tool to examine the
quality of service provided
to our patients