1. Pathologist Performance Metrics
John Sinard, MD, PhD
Vice Chair and Director of AP
Yale Pathology

2. Notice of Faculty Disclosure
In accordance with ACCME guidelines, any individual in a position to influence and/or control the content of this ASCP CME activity has disclosed all relevant financial relationships within the past 12 months with commercial interests that provide products and/or services related to the content of this CME activity.
The individual below has responded that he/she has no relevant financial relationship(s) with commercial interest(s) to disclose:
John Sinard

3. Disclaimers This is not a discussion of MIPS metrics
This is part of an OPPE program
This is a work in progress. There is still much to do. The destination is still not even clearly defined.
“Dashboard” means different things
Up to the minute status of parameters
Quick view of multiple parameters
Both

4. Pathologist Performance
What we would like to measure:
Diagnostic accuracy
Attention to detail
Responsiveness to clinician clients
Willingness to work
Ability to function as a member of a team

5. Preferable Metrics Measureable
Metric can be measured consistently
Metric is quantitative
Aligned with (or surrogate of) what you would really like to measure
Trendable over time
Interpreted against appropriate benchmarks
Focused on identifying the cause, not just the result

6. Metric: FS/Perm Discrepancy
Valuable in/of itself
Surrogate for diagnostic accuracy overall
Multiple dimensions:
Agreement level
Disagreement reason
Impact
“Centralized” Scoring
Can’t rely on signout pathologist to flag cases
Easy to get hung up on differences in style
Maintain focus on whether or not clinical question was answered

7. FS/Perm Agreement Levels
Answered implied clinical question correctly
Defer
Surgeon had to decide what to do without help from pathology
Margin Assurance
“I didn’t call it positive, but I can’t call it negative”
Disagree
Separately score Reason and Impact

8. FS/Perm Disagree Reason Impact
Interpretive: Dx material present on FS slide(s)
Perms Only: Dx material in FS block but not on FS slides(s)
Sampling: Dx material not in FS block
Impact
None: Correct thing done (or interest only)
Minor: Might have done something different or small additional margin during current surgery
Significant: Patient required additional surgery
Major: Unnecessary substantial surgery/treatment done

9. Metric: Amendment Rates
Not all amendments are created equal
Need a classification system: Reason
Original classification system
Problems with original system
Change in Final Diagnosis Field
Change in Pathologist
Change in Synoptic Summary
Change in Cytotech
Change in Clinical Information
Add/Delete Parts
Change in Gross Description
Accessioned to Wrong Patient
Change in Labeling
Signed out prematurely
Change in Client/Location
Typographical Error
Change in Submitting Physician

10. Cause vs Result Result A Cause 1 Result B Cause 2 Result C Cause 3
Result D

11. New Amendment Classification
Interpretive Error
At time of signout, pathologist did not know the complete and “correct” diagnosis
Includes incorporation of ancillary studies, changes to margin status, etc.
Diagnostic Communication Error
At time of signout, pathologist knew the correct diagnosis, but it was not communicated accurately / fully in report
Includes omitted data, inconsistent data, typographical errors
Administrative
Non-diagnostic issues
Only use if there is NO CHANGE IN THE FINAL DIAGNOSIS or SYNOPTIC SUMMARY
(Other)

12. New Amendment Classification
Interpretive Errors
Oversight/Overlooked (“I didn’t see that”)
Didn’t Consider (“I should have thought of that” or “I never heard of that”)
Considered / Excluded (“I thought about that; decided against it”)
Incorrect Context (“I didn’t know the patient had …”)
Planned (“I signed out the case knowing it would be amended when additional results became available”)
Unknown Reason (doesn’t fit other categories)

13. New Amendment Classification
Diagnostic Communication
Final Diagnosis – Incomplete
Includes amending to report the results of special stains done but not reported
Final Diagnosis – Inconsistent
Typically discrepancy between final dx and synoptic
Use for incorrect staging if all of the elements are correctly reported
Final Diagnosis – Typo – Potentially misleading
E.g., “Stains for HSV and Candida are”
Final Diagnosis – Typo – Not misleading
Use ONLY for single word changes, NOT for missing lines
Wrong case (Case mix-up)
Correct diagnosis entered into incorrect case
Synoptic Summary
Includes missing data, miss-entered data

14. New Amendment Classification
Diagnostic Communication (continued)
Wrong patient – External
Case accessioned to incorrect patient because it came mislabeled
Wrong patient – Internal
Case accessioned to incorrect patient because Pathology chose the wrong patient in CoPath
Labeling (site/source) – External
Usually laterality; specimen came mislabeled
Labeling (site/source) – Internal
Usually laterality; specimen reported incorrectly
FS vs Final Discrepancy not addressed
FS/Permanent discrepancy not specifically addressed in initial report
Additional Material Received/Submitted
Clinician requested looking for more lymph nodes; taken-back blocks show additional information
Other
Try real hard not to use this – give detailed explanation if you do

15. New Amendment Classification
Administrative
Add/Delete Parts
Additional material received, or two specimens merged into one
Clinical Information Incorrect
Incorrect clinical information provided and entered onto report, but did not change interpretation of specimen
Non-diagnostic data – External
Incorrect submitting physician provided
Non-diagnostic data – Internal
Incorrect submitting physician chosen, incorrect client, incorrect outside case number
Amended in Error
Should not have amended the case; no changes made to anything

16. Categorizing Amendments
Amendment information needs to be entered by the Attending Pathologist
Needed a tool to create amendments which:
Was available 24/7
Allowed amending only your own cases
Required entry of needed information
Add Requirement for Amendment note
Needed a tool for centralized review of amendments

17. Additional “Metrics” Case Volume Turnaround Time
No a metric per se, but places other metrics in perspective
Turnaround Time
Average easy to calculate, but less meaningful
Time to xx% of cases is more meaningful to our clinician colleagues
Must compare apples to apples

18. Additional “Metrics”
Use of ancillary testing (special stains, immunos)
Can be a clue to different practice patterns
External Reviews
Agreement level
Percentage of cases within a month of s/o
Case sharing / consensus conference
Surrogate to “teamness”