1. Peer Review in Endocrinology John S. Bevan Peer Review Coordinator Society for Endocrinology

2. Definition (Van Weedt, 2000) Formalized event – explicit pre-determined procedures & questionnaires – standards for reports Information derived from documentation, observation and verbal Standards of good quality and best practice to be used where available Scope – clinical care process and its organisational aspects

3. Experience of other specialities Thoracic medicine Renal medicine Cardiology

4. British Thoracic Society Aims To increase clinical effectiveness & improve patient care To benefit reviewed unit To benefit reviewers Ethos Voluntary, supportive & non-confrontational But also tough and thorough Richard Page & Brian Harrison. Journal RCP (London), 1995, 29: 319-24

5. Endocrinology Peer Review John Wass initiative.... Setting appropriate Standards for Clinical Endocrinology  British Thoracic Society  Clinical Standards Board for Scotland Clinical Committee consultation on draft Ten standards agreed First pilot visits

6. Ten Standards: Clinical Endocrine Care 1. Initial referral & assessment 2. Patient focus 3. Communication 4. Endocrine function testing 5. Interface with biochemistry 6. Endocrine imaging 7. Interface with endocrine pathology 8. Links with other specialities 9. High-cost endocrine therapies 10. Endocrine audit & databases

7. Endocrine Standards – example

8. Assessing the Standards

9. Endocrine Standards - example

10. Supporting documents http://www.endocrinology.org/about/projects/peerreview.html

11. Planning a Peer Review Visit Centre selection  Basic model (TH-DGH)  Tailored model Reviewer recruitment Setting the timelines

12. Planning a Peer Review Visit Centre selection  Basic model (TH-DGH)  Tailored model Reviewer recruitment Setting the timelines

13. Timelines -8-4-2+4+6+8-16 Initial planning: places, people & dates SAQ-out SAQ-back Visit timetables 2-day Peer Review Visit weeks Draft reports Draft reports checked Final reports & feedback questionnaires 0

14. Pre-visit Questionnaire General information about region Outpatient & inpatient workloads Staffing levels Facilities for endocrinology Support services: admin, lab, imaging Speciality links Audit and research Section 12: LOCAL PERCEPTIONS FOR CHANGE What are the changes most wanted by the endocrine unit? How would they like the endocrine unit to be developed? Does this relate to any of the local NHS priorities for change?

15. What happens during a PRV? Pre-visit preparation is vital Visits to people, places and events Recording – using the report form Preliminary feedback Decisions on report writing & deadlines

16. During a visit there’s a lot to fit in!

17. What does the Final Report look like?

18. What happens to the Final Report?

19. PR – extracting the digit!

20. Who’s been visited? Sheffield-Chesterfield Oxford-Reading Hull-York Glasgow THs Manchester-Pennine Stoke-Shrewsbury Nottingham-Derby Bristol-Taunton Cambridge-Ipswich 20 hospitals providing endocrine services  14 teaching hospitals  6 district general hospitals

21. Who’s done the work? James Ahlquist Steve Atkin * John Bevan # John Connell * Julian Davis * Mohgah Elsheikh * Nick Finer # Stephen Gallacher * Ashley Grossman Colin Johnston Tara Kearney * Bill Kelly John Miell John Newell-Price * John Wass * * Volunteered after PRV to their centre # Undertook 2 or more PRVs

22. So, has it been worth all the effort?

23. How was it for the reviewed centres? Was the Review worthwhile for your unit?Yes – 100% Did you feel the Report analysed your unit objectively? Yes – 100% Did the Report identify any unanticipated deficiencies? Yes – 27% Do you plan to share the Report with your managersYes – 100% Would you volunteer to be reviewed again?Yes – 100% If so, after what period?3.4 years (2-5) Having been reviewed, would you volunteer to be a reviewer? Yes – 83% (20-item questionnaire – sent 2 months after visit – 65% RR)

24. Reviewed consultants said...  ‘I believe the PRV was very important – our MD and CEO used the report to lobby PCTs whenever they had a chance’ (DGH)  ‘It got us all thinking about what we do’ (TH)  ‘Excellent report captured all the current issues within our department & the city as a whole’ (TH)  ‘Many thanks to the reviewers for their supportive attitude and for stimulating discussion’ (TH)  ‘We often feel a bit isolated....so it was reassuring to learn we’re actually doing quite well!’ (DGH)

25. How was it for the reviewers? How many weeks before the visit did you receive the Self-Assessment Questionnaire? 2.4 weeks (1-6) Did you find Reviewing a worthwhile experience?Yes – 100% Did you pick up any new ideas during the visit?Yes – 82% How long did it take you prepare the Report6.4 hours (3-16) Would you be a Reviewer again?Yes – 100% If so, after what period?6-12 months (17-item questionnaire – sent 2 months after visit – 61% RR)

26. Reviewers said...  ‘I found it very valuable & it led to more reflection about our own unit’ (TH)  ‘Thanks for allowing me to be a Reviewer – I found it to be an absolutely invaluable & incredibly useful experience’ (DGH)  ‘Excellent development – I benefited from a peer review visit and was happy to reciprocate for the Society’ (TH)  ‘A beneficial, albeit exhausting, exercise for me!’ (TH)  ‘Gave me some good ideas for updating my own protocols and PILs!’ (TH)

27. Reviewers ‘pick up’ good ideas!

28. How well did the endocrine centres do? ‘Essential’ standards (n=30) Teaching hospitals DGHs Percent 84%82%

29. How well did the endocrine centres do? ‘Desirable’ standards (n=18) Teaching hospitalsDGHs Percent 55%45%

30. How well did the endocrine centres do? ‘Desirable’ standards (n=18) Teaching hospitalsDGHs Percent 25%32%

31. Types of Recommendation Total = 128 (71 ‘major’ & 57 ‘minor’) Number

32. Progress on 128 PRV Recommendations... after a mean interval of 3.4 years (range 1-7)...100% update! Number 9 Consultants 5.5 Nurses 2 Secretaries Funding constraints on 9 staffing posts & 6 facility improvements Only 4%

33. Other benefits of peer review How does my centre compare to others in the UK? Workload Waiting times Administration Imaging access

34. Activity comparisons between hospitals “Out-patients seen per Consultant DPA” Teaching hospitals DGHs CCCNCCCN CCCC

35. Some quality indicators Standard 1: Initial referral & assessment

36. Some quality indicators Standard 3: Communication

37. Some quality indicators Standard 6: Endocrine imaging

38. What are the problems? Agreeing the dates Understanding the objectives Selecting the centres (especially DGH) Recording the data Time Finance

39. Where now….. in the decade of Revalidation? Encourage wider UK roll-out  Still voluntary  Who else can review our specialist activities and team-working? New PRV Coordinator, 2010  Dr Petros Perros, Newcastle Endocrine nurse involvement? Workload comparisons? Quality indicators? Bench-marking?

40. Next round of visits... Sheffield-Chesterfield Oxford-Reading Hull-York Glasgow Manchester-Pennine Stoke-Shrewsbury Nottingham-Derby Bristol-Taunton Cambridge-Ipswich Aberdeen-Inverness London-K/G/T London-H/CC Plymouth-Exeter Belfast- Cardiff- Leeds-

41. Grateful thanks to... Volunteers: Centres and Reviewers Society for Endocrinology supporters Clinical Endocrinology Trust