1. Business Case for Quality: Best Practices Surrounding Information Technology Stephen S. Raab, M.D. Department of Pathology, University of Pittsburgh, Pittsburgh, PA April 29, 2005

2. Background n Medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim n Medical errors permeate all levels of patient care and the IOM report has resulted in the greater recognition of error and an increased focus on reporting and reducing errors

3. Background n Scant data on anatomic pathology error frequencies n Institutional error frequencies may be low, limiting hypothesis testing n Limited data on how pathology errors affect patient care n Root cause analysis of anatomic pathology errors rarely performed and results of these analyses are not disseminated

4. Error prevention n Minimize psychological precursors; design buffers; build in redundancy n Design tasks: simplification; constraints; standardization n Systems designed to absorb error n Root cause analysis n Shift from training-focus to performance-focus n Data collection to define problem

5. Initiatives to decrease errors n Creation of audit systems n Use of benchmarking and error tracking systems n Implementation of immediate error reduction systems (e.g., Toyota Production Systems, Six Sigma)

6. Information technology and patient safety n Most systems are focused on diagnostic reporting n Difficulty in performing quality assurance n Few checks to prevent errors n Systems themselves are often a source of error n Few databases to study quality

7. Quality assurance n Cytologic-histologic (CH) correlation performed in all American labs (CLIA’ 88) n No national standards to perform CH correlation n Information systems not structured to perform cytologic correlation –Case pairs may be visualized prior or after sign-out –Most CH correlation performed with manual review

8. Cytologic-histologic correlation n What is the best way to perform correlation and how does one use the data? n Letter sent to 162 American labs requesting information on how they performed correlation (response frequency: 32.1%) n Separated material into forms, logs and tally sheets

9. CAP Checklist n Cytology case number n Sign-out cytology diagnosis n Sign-out cytologist n Original cytotechnologist diagnosis n Sign-out cytotechnologist n Review cytology diagnosis n Review cytologist

10. CAP Checklist n Surgical pathology case number n Sign-out surgical pathology diagnosis n Sign-out surgical pathologist n Review surgical pathology diagnosis n Review surgical pathologist n Significance of discrepancy n Action taken n Reason for correlation

11. Minimum expected and additional variables listed on forms only. Bold line represents minimum expected variables that should be present (n = 15).

12. UPMC technical quality assurance n Record laboratory error: –Accessioning –Gross groom –Histology –Cytology –Transcription

14. National benchmarking n CAP Q-Probes studies – error rates, turn around time, amended report rates n CAP Q-Tracks studies –Cytologic-histologic correlation –Frozen-permanent section review –Small surgical specimen turn around time

15. Second viewing of cases n CAP – Q-Probes study 2004 n 74 labs, 6186 specimens, 415 discrepancies (6.7% rate) n Breakdown of errors –48% with change in same category of diagnosis –21% change in category of diagnosis –18% typographical errors –9% patient or specimen information –4% change in margin status

16. Cytologic-histologic correlation n Year end report for participating labs (56) n In 2003, a total of 19,478 Paps were correlated with 11,336 true positive correlations and 2,433 false positive correlations n Predictive value for positive cytology: 82.3% n Percent positive ASC diagnoses: 64.3% (range 34.1%-89.9%); percent positive AGC diagnoses: 31.4% (range 0%-66.7%)

17. Cytologic-histologic correlation n Best performers and clustering performed n Best performers: –Quarterly reports, track and trend –Summary reports to clinicians –Document correlation in report 5/9 labs –Keep written log of findings 2/9 labs –Review biopsy if discrepancy 3/9 labs

18. AHRQ national errors database n 5 year project to monitor pathology errors by creation of a multi-institutional database n University of Pittsburgh, University of Iowa, Henry Ford Health System, Western Pennsylvania Hospital n Goal is to devise plans to reduce cytology and surgical pathology errors

19. Specific aims 1. Create voluntary, Web-based database and collect errors detected by correlation, secondary review, amended reports, frozen section review 2. Quantitatively analyze error data and generate performance improvement reports 3. Perform root cause analysis; plan and implement interventions to reduce errors 4. Assess success of interventions by quantitative measure; disseminate successful error reduction plans

21. Database challenges n Current absence of standardized and detailed laboratory workload and quality assurance data sets in widely used laboratory information systems n Current lack of efficient and comprehensive electronic de-identification of un-linked institutional laboratory information system and clinical data

22. Database construction n The database is Oracle 9.2.0.4 Enterprise Edition implemented on a Sun Ultra E450 Server running Solaris 2.9. The mid-tier is implemented with Oracle’s Application Server (v9.0.3) on a Compaq DL360 Server running Windows 2000. The application uses the Oracle http server and mod_pl/sql extensions to generate dynamic web pages from the database to the users via a Microsoft Internet Explorer web browser version 6.0 or higher.

23. Logical system design n Schema layer contains the actual data and data relations that are stored entirely as numbers and keys n Meta-data layer, the data is defined in terms of data elements and “data objects” n Procedure layer contains a set of dynamic procedures/functions (in PL/SQL) that externalize the data elements n Presentation layer is a series of forms (for data entry, display, query, etc.) that are populated by data elements from meta-data

24. Future database plans n Database interface to laboratory information systems n Data de-identification n Revision of database architecture n Incorporation of histopathologic image data

25. Variable frequency of errors across sites InstitutionNumber of errors Total number of cases Error percent A19628,3960.69 B27966,1150.42 C796143,8350.55 D17482,7870.21

26. Correlating case error frequency Error frequency SiteGyn correlating %Non-gyn correlating % A9.4911.03 B1.655.86 C4.7211.72 D3.336.14

27. Variability in assessing sampling versus interpretive error Institution ReasonABCDTotal Interpretation21%11%65%4%33% Screening2%8%4%2%5% Sampling61%73%38%92%60% Unknown18%9%03%6%

28. Agreement between original and review reason for discrepancy (interpretation or sampling) Review reason SiteABCD Original reason A0.6150.4120.0240.286 B0.7370.2110.1180,737 C0.4000.5450.6150.40 DUnd

29. Percentage of total error by specimen type Institution OrganABCDTotal Lung52%20%35%17%34% Thyroid0%5%8%2%6% Bladder22%14%16%33%18% Breast2% 5%17%5%

30. Error outcome taxonomy n Significant event: error that affects patient outcome; may be classified by severity (mild, moderate, and severe) n No harm event: error does not affect patient outcome n Near miss event: intervention occurred before harm could take place

31. Differences in grading of discrepancy assignments Institution AssignmentABCDTotal Significant event 0028%14%22% No Harm86%98%32%50%44% Near miss14%040%14%32% Unknown02%021%2%