1. For the CAP Instrumentation and Chemistry Resource Committees
Calibration, Calibration Verification, Analytical Measurement Range, and Clinically Reportable Range
For the CAP Instrumentation and Chemistry Resource Committees 1

2. The Truth, Please What’s your response when
You receive the results of a CAP Linearity Survey?
You come to Checklist Question CHM (one of several relating to Calibration Verification)? 2

3. The Truth, Please What’s your response when
You receive the results of a CAP Linearity Survey?
You come to Checklist Question CHM (one of several relating to Calibration Verification)?
If you’re anything like me, it’s not
“Wow, let me at it. I was really looking forward to taking care of this!”
Who I Am: Gary L. Horowitz, MD, FCAP Director, Clinical Chemistry, Beth Israel Deaconess Medical Center Associate Professor of Pathology, Harvard Medical School Chair, CAP Chemistry Resource Committee 3

4. Terms
Calibration 4

5. Terms that Instill Fear
Calibration
Calibration Verification
Analytical Measurement Range (AMR)
Clinically Reportable Range (CRR) 5

6. Terms that Instill Fear
Calibration
Calibration Verification
Analytical Measurement Range (AMR)
Clinically Reportable Range (CRR)
CAP Linearity Surveys 6

7. Why We’re Here Today confusing, but important, concepts
audioconference format: allows anyone to visit the web as often as needed
objectives:
explain the concepts
reveal how they relate to the accreditation process
show you in detail how the CAP Linearity Surveys enable you to address these issues for your laboratory
ambitious goals, but we have two excellent speakers, experts
in Laboratory Medicine
in CAP Surveys 7

8. Introductions Greg Miller, PhD
Professor of Pathology, Virginia Commonwealth University
Consultant Member, CAP Chemistry Resource Committee 8

9. Introductions Greg Miller, PhD
Professor of Pathology, Virginia Commonwealth University
Consultant Member, CAP Chemistry Resource Committee
Anthony Killeen, MD, PhD, FCAP
Associate Professor of Pathology, University of Minnesota
Member, CAP Instrumentation Resource Committee
Former Member and Chair, CAP Chemistry Resource Committee
Program developed as joint effort of these two CAP Resource Committees 9

10. Objectives
Differentiate between the terms calibration, calibration verification, analytical measurement range (AMR) and clinically reportable range (CRR).
Clarify the accreditation requirements regarding these items.
Explain techniques to perform the appropriate verifications.
Clarify CAP Linearity (LN) Survey reports. 10

11. CLIA 493.2
Reportable range: span of test result values over which the lab can establish or verify the accuracy of the measurement response. 11

12. CAP (two concepts) CLIA 493.2
Analytical measurement range: range of analyte values that a method can directly measure on the specimen without any dilution, or other pretreatment not part of the usual assay process.
Reportable range: span of test result values over which the lab can establish or verify the accuracy of the measurement response. 12

13. CAP (two concepts) CLIA 493.2
Clinically reportable range: range of analyte values that are reported as a quantitative result, allowing for specimen dilution or other pretreatment used to extend the actual AMR.
Reportable range: span of test result values over which the lab can establish or verify the accuracy of the measurement response. 13

14. CAP CLIA
Calibration: set of operations that establish the quantitative relationship between the reagent system / instrument response and the concentration or activity values of an analyte.
Calibration: process of testing and adjusting to establish a correlation between the measurement response and the concentration. 14

15. CAP (two concepts) CLIA 493.2
Calibration verification: assaying materials of known concentration to substantiate the system’s calibration throughout the reportable range. 15

16. CAP (two concepts) CLIA 493.2
Calibration verification: process of confirming that the current calibration settings remain valid for a method.
Calibration verification: assaying materials of known concentration to substantiate the system’s calibration throughout the reportable range. 16

17. CAP (two concepts) CLIA 493.2
Calibration verification: assaying materials of known concentration to substantiate the system’s calibration throughout the reportable range.
Analytical measurement range verification: process of confirming that the assay system will correctly recover the concentration or activity of the analyte over the AMR. 17

18. CLIA 493.1255: Calibration and calibration verification procedures
(a) Perform and document calibration procedures.
Follow manufacturer’s instructions.
Use criteria established by lab during validation of method when put in service.
Whenever calibration verification fails to meet the lab’s specifications. 18

19. CLIA 493.1255: Calibration and calibration verification procedures
(b) Perform and document calibration verification procedures.
Follow manufacturer’s instructions
Use criteria established by lab during validation of the method when put in service
Include minimum, mid-point and maximum value to verify the laboratory’s reportable range (this is AMR)
At least every 6 months or when the following occur:
Change of reagents unless it does not affect the range to report patient results (this is AMR), or QC values (this is CAL)
Major preventive maintenance (this is CAL)
QC trend, shift or out of limits (this is CAL)
Schedule for verifying reportable range (this is AMR) requires more frequent calibration verification 19

20. Calibration (two point)
Analytical Measurement Range extends beyond the high calibrator
= Reportable Results
SIGNAL
CONCENTRATION
CALIBRATOR TARGET VALUES
May be measurement system specific 20

21. Calibration (two point)
Example: typical serum glucose method
Low calibrator = reagent blank (0 mg/dL)
High calibrator = 356 mg/dL
AMR = mg/dL
The AMR is based on the expectation that the signal is linearly related to glucose concentration over the mg/dL range.
Verification of the AMR expectation is required because the calibrators do not define the complete measurement range. 21 21

22. Calibration (multi-point)
Analytical Measurement Range
= Reportable Results
Note that when calibrators span the AMR, the calibration is fully defined over the AMR
SIGNAL
CONCENTRATION
CALIBRATOR TARGET VALUES
May be measurement system specific 22

23. Accuracy (Trueness) Traceability
Manufacturer provides calibration traceability to the highest order reference method or reference material available
Manufacturer’s product calibrator has method (and sometimes reagent lot) specific target values
Cannot mix reagents and calibrators from different manufacturers 23

24. Terminology
Trueness: the mean bias between a routine method and a reference method determined from replicate measurements of multiple patients’ samples
Imprecision is removed (or minimized) by replication
Accuracy: the difference between a routine method and a reference method for a single measurement on a single patient’s sample
Difference includes influence of both bias and imprecision for a single measurement
Refer to the CLSI harmonized terminology database for more detailed definitions at 24

25. Calibration Verification
Verify that measurement system calibration has not changed since the last calibration event
And, where applicable (many routine methods),
Verify that method calibration is in conformance to the manufacturer’s specifications
(i.e. is consistent with the manufacturer’s calibration traceability)
Special case (uncommon) when a lab can verify calibration traceability to an accuracy based reference system 25

26. Commutable Reference/ PT/ QC Material
Has equivalent mathematical relationships among the results of different measurement procedures for a reference material and for native clinical samples (International Standards Organization, Clinical and Laboratory Standards Institute derived).
Produces a numeric result from a routine method that is the same as would be obtained for a native clinical sample that has the same quantity of an analyte (measurand) present. 26

27. Non-commutable materials may have a matrix bias
Bias in a result that is attributable to modification of the usual clinical sample matrix caused by preparation of a reference material, PT material or QC material
CAP has investigated matrix bias in PT materials using fresh frozen off-the-clot samples (that are expected to be commutable with clinical samples)
CAP Home ( > CAP Reference Resources and Publications > Chemistry
CAP has developed accuracy based PT Surveys from this research 27

28. CLSI EP14 to validate commutability10
95% prediction interval 8 6
Test Method 4
Clinical Specimen 2 2 4 6 8 10
Comparison Method (1 RMP if available) 28

29. CLSI EP14 to validate commutability10
95% prediction interval 8 6
Test Method 4
Clinical Specimen
RM Commutable 2 2 4 6 8 10
Comparison Method (1 RMP if available) 29 29

30. CLSI EP14 to validate commutability10
95% prediction interval 8 6
Test Method 4
Clinical Specimen
RM Commutable
RM Not-Commutable 2 2 4 6 8 10
Comparison Method (1 RMP if available) 30 30

31. Effect of non-Commutable PT materials1 5 2 3 d u P o n t D i m e s C h l r ( g / L ) A b - K a S v y p f : N , . c
Patient bias = 0.2%
PT bias = -9.5%
Reference Method L a b M e d . 1 9 9 3 ; 1 1 7 : 3 4 5 - 5 1

32. CAP Accuracy Based Surveys
Glycohemoglobin (GH2) – uses fresh whole blood with National Glycohemoglobin Standardization Program target values
Creatinine Accuracy Calibration Verification and Linearity (LN24) – uses off-the-clot serum with NIST target values
Accuracy Based Lipids (ABL) – uses off-the-clot serum with CDC reference method target values
Additional accuracy based Surveys are in development for analytes with clinical practice guidelines 32

33. Calibration Verification
Run assay materials with measurement system and compare results to measurement system appropriate target values
Result = Target ?
Calibration is OK
Re-calibrate
YES NO 33

34. Calibration Verification Materials
Method product calibrator(s)
Method vendor provided materials for calibration verification
Previously tested clinical specimens
Reference materials with matrix and target values appropriate for the method
PT (or PT validated) materials with matrix and target values appropriate for the method, including materials designed for calibration verification 34

35. Verification of Calibration Trueness (or accuracy for individual samples)
Requires a reference material that is commutable with native clinical samples (not commonly available)
Most PT materials are not commutable due to the cost to prepare commutable materials and the practical need to have many analytes in the same material 35

36. LN Surveys for Calibration Verification
Lab verifies conformance to the manufacturer’s calibration, using:
Commutable reference materials that have been validated to give results equivalent to those for native clinical samples (e.g. CAP LN 24 Creatinine accuracy based Survey with NIST target values)
Non-commutable materials with target values appropriate for a specific method (e.g. most CAP LN Survey materials with peer group evaluation) 36

37. Calibration Verification Concentration Levels
Approximately the same as the calibrators used for method calibration
i.e. to confirm correct calibration
Can also verify the AMR if concentrations, or activities, of the calibrators cover the full measurement range.
LN Survey provides a series of defined concentrations to verify calibration over the AMR 37

38. Frequency of Calibration Verification
Manufacturer’s instructions
Troubleshoot a QC problem
Calibration OR calibration verification at least every 6 months (CLIA), or when:
Change in critical reagents, component, or maintenance
If cal <6 months do not need calibration verification unless part of manufacturer’s calibration process 38

39. Analytical Measurement Range Verification
Non-linear region
Specimen concentration relationship is linear over the AMR
CONCENTRATION
RECOVERED
CONCENTRATION EXPECTED
Based on: (a) admixture ratio (b) assigned value 39

40. Method Calibration and AMR Verification can be a single process
ANALYTICAL MEASUREMENT RANGE
Calibrator must:
1. Have at least 3 values that span the AMR, and
2. Have target values specific for the method (and reagent lot, if necessary).
= reportable values
Analytical signal
No separate verification of AMR is necessary
Method Calibrator 40

41. AMR Verification
1. Specimens (can use patients) with low and high values
2. Mix 1:1 to get midpoint value
3. Mix low:mid and mid:high to get additional values
4. Expected values are:
Sample Proportion Concentration expected
Low 0% 20 mg/dL
0.75 L H 25% 160 mg/dL
0.50 L H 50% 299 mg/dL
0.25 L H 75% 438 mg/dL
High 100% 578 mg/dL
Criteria for acceptability = recovery of concentration in proportion to the amount present (linear relationship) 41

42. AMR Verification
AMR verification can be based on demonstrating linearity either by measuring recoveries or by demonstrating expected proportions 42

43. AMR Verification Measured mg/dL Expected Proportion
200
400
600 50
100
Expected Proportion
Measured mg/dL
Linear relationship verifies recovery of correct concentration relationship over AMR. 43

44. Materials for AMR Verification
Linearity material of appropriate matrix
PT (or PT validated) materials
Previously tested clinical specimens
Previously tested clinical specimens, altered by admixture, dilution, spiking
Reference materials with matrix and target values appropriate for the method
Method product calibrator(s) 44

45. AMR: how close to low and high limits
Goal is to verify acceptable clinical performance
For low limit: use clinical judgment, e.g.
creatinine, 0.2 or 0.3 mg/dL
For high limit: within 10-15% of upper limit of AMR
Challenging to find residual samples for some analytes; need to use clinical judgment, e.g.
blood gas (use commercial materials)
cardiac markers, e.g. Troponin (use LN Survey) 45

46. CAP LN Surveys
Most evaluate performance vs. peer group due to commutability limitations (matrix effect bias)
LN24 offers matrix-free creatinine challenges with reference method values assigned
Verify that calibration conforms to manufacturer’s specifications
Verify that calibration is correct over the analytical measurement range
When accuracy based (e.g., LN24), verify that calibration is traceable to a reference system 46

47. Example of LN2 Report (Calcium)47

48. Example of LN2 Report (Calcium)48

49. Example of LN24 Report (Creatinine)49

50. Example of LN24 Report (Creatinine)50

51. Example of a Non-Linear Result
Creatinine 51

52. Calibration Verification and Linearity Users Guide52

53. Clinically Reportable Range
Range of analyte values that will be reported as a quantitative result, allowing for specimen dilution or other pretreatment to extend the AMR
Policy based on the laboratory director’s medical judgment regarding the clinical use requirements for a laboratory test 53

54. Clinically Reportable Range
Truncation or extension of the AMR based on clinical requirements.
Established once when method is put in service
Does not need to be verified on a recurring schedule
Values outside the CRR are reported as < or > a numeric value.
Checklist question is: Are dilution protocols and diluents (or concentration protocols) specified for all methods for which the CRR exceeds the AMR? 54

55. Clinically Reportable Range
Example: hCG may have a CRR with both low and high limits.
AMR is 3-1,000 mIU/mL (method specification).
CRR is 5-1,000,000 mIU/mL (clinical usefulness).
Results of <3, 3, or 4 are reported as “<5 mIU/mL”.
Results >1,000 are diluted and rerun to obtain quantitative values up to 1,000,000 mIU/mL.
Results >1,000,000 are reported as “>1,000,000 mIU/mL”.
NOTE: CRR limits are the judgment of the lab director and may be different for different labs 55

56. Clinically Reportable Range
Example: AST may have a CRR with only a low limit.
AMR is IU/L (method specification).
CRR is 4 IU/L to any value (clinical usefulness).
Results of <4 are reported as “<4 IU/L”.
Results >900 are diluted and re-assayed.
The upper CRR is not specified because specimens are diluted until a quantitative value is obtained. 56

57. Summary
Calibration or Calibration Verification at least every 6 months.
Calibration verification is not necessary if calibration is performed at least every 6 months. 57

58. Summary
Analytical Measurement Range verification at least every 6 months.
May be required when change a critical reagent lot or other analytical parameter that can influence the AMR.
Not required when a recalibration is performed.
Not required when method calibrators include at least 3 concentrations that span the full AMR.
REQUIRED for initial method validation. 58

59. Summary
Clinically Reportable Range is established once when a method is introduced based on clinical requirements as defined by the lab director.
SOP for dilution or other specimen pre-treatment must be defined.
Not required to verify CRR on a recurring schedule 59

60. Summary
For additional information on the content of the Audioconference
or questions about the CAP Calibration Verification/Linearity
Surveys, please contact the CAP:
By telephone: Customer Contact Center at
, option 1# By
By mail: Surveys Program
325 Waukegan Road
Northfield IL