1. Measurements for 8 Common Analytes in Native Sera Identify Inadequate Standardization among 6 Routine Laboratory Assays H.C.M. Stepman, U. Tiikkainen, D. Stöckl, H.W. Vesper, S.H. Edwards, H. Laitinen, J. Pelanti, L.M. Thienpont, and Participating Laboratories June 2014 www.clinchem.org/content/60/6/855.full © Copyright 2014 by the American Association for Clinical Chemistry

2. © Copyright 2009 by the American Association for Clinical Chemistry Introduction  Proficiency Testing (PT), also known as External Quality Assessment (EQA), has earned a well-deserved position as an element of laboratory quality management  PT with commutable samples and high-quality targets is essential for assessing the accuracy of diagnostic assays and the interchangeability of generated results  The “intrinsic” quality of a manufacturer’s assay may be influenced by the laboratory using it; therefore, assessment under routine conditions is essential See Editorial by Horowitz GL. Assessing Accuracy on the Front Lines: A Pragmatic Approach for Single-Donor Proficiency Testing. Clin Chem 2014

3. © Copyright 2009 by the American Association for Clinical Chemistry Questions  What were the special aspects of the experimental design of the PT survey described?  How was the data assessment done?  Quality indicators?  Targets?  Limits?

4. © Copyright 2009 by the American Association for Clinical Chemistry Materials and Methods – Study Design  Use of 20 fresh-frozen single donation serum samples prepared according to CLSI C37-A  Selection of laboratories using “homogeneous tests” Reagent, calibrator, platform from the same manufacturer  Inclusion of assays installed on random access platforms Abbott Architect, Beckman Coulter AU, Ortho Vitros, Roche Cobas, Siemens Advia, and Thermo Scientific Konelab  Measurement of 8 analytes Creatinine, glucose, phosphate, uric acid, total cholesterol, HDL cholesterol, LDL cholesterol, and triglycerides

5. © Copyright 2009 by the American Association for Clinical Chemistry Materials and Methods – Data Assessment  Quality indicators  Intra-assay and combined imprecision (including sample matrix interference)  Bias and total error  Targets  Peer group  “All-method trimmed mean” (AMTM)  Reference (REF) method values (cholesterol, creatinine, uric acid)  Limits  Reflecting state-of-the-art performance  Related to biological variation

6. © Copyright 2009 by the American Association for Clinical Chemistry Questions  What were the main findings of the study?  What were the limitations?

7. © Copyright 2009 by the American Association for Clinical Chemistry Results – Summary  Excellent peer performance for most assays (except HDL and LDL cholesterol)  Intrinsic quality sufficiently robust for satisfactory performance in a daily laboratory context  Considerable bias for some assays (Table 1/Figures 1 & 2)  Siemens Advia creatinine (-4.2%), Ortho Vitros phosphate (8.9%), Beckman Coulter AU triglycerides (5.4%) and LDL (15%), Thermo Scientific uric acid (6.4%)  More biases at low & high concentration  Inter-laboratory differences >30% (Table 2)

8. © Copyright 2009 by the American Association for Clinical Chemistry Results Table 1. AMTM/REF bias estimates (at low, mid, and high concentration) for each assay (named by manufacturer). The blue, underlined values indicate violation of the limits. CHOL, cholesterol; CREA, creatinine; GLU, glucose; HDL, HDL cholesterol; LDL, LDL cholesterol; PHOS, phosphate; TRIGL, triglycerides; UA, uric acid; NA, not applicable. CHOLCREAGLUHDLLDLPHOSTRIGLUA Bias Limit (%)444.5 4 Abbott3.05.1-0.45.52.70.12.5-4.9 2.53.7-0.21.50.00.1-1.2 2.32.80.2-5.40.90.01.9 Beckman2.7-0.62.10.3NA0.85.8-2.6 3.80.51.9-3.2NA0.55.4-1.3 4.41.11.6-5.6NA0.45.3-0.3 Ortho-0.61.6-3.00.0NA14.6-0.9-2.2 0.01.2-2.8-1.1NA8.9-0.3-2.6 0.40.9-2.4-1.9NA6.70.0-2.8 Roche3.52.6-0.7-0.35.2-1.10.0-4.6 2.52.7-0.63.92.0-0.7-1.6-3.4 1.92.7-0.66.70.4-0.5-2.3-2.4 Siemens0.7-5.51.22.80.01.30.30.4 0.0-4.21.02.2-0.31.10.70.8 -0.4-3.40.51.7-0.41.11.01.2 Thermo Scientific2.2-1.50.8-8.32.1NA-1.95.2 2.6-0.30.7-0.7-0.1NA1.06.4 2.80.50.64.4-1.3NA2.37.3

9. © Copyright 2009 by the American Association for Clinical Chemistry Figure 1. Assay bias (% difference = (mean of peer group result - target value/target value)*100) and total error vs AMTM (glucose, HDL-cholesterol) or REF target values (cholesterol, creatinine). Abbott (red diamond), Beckman (blue square), Ortho (black triangle), Roche (yellow circle), Siemens (red square), and Thermo Scientific (blue diamond). The red-broken bias limits are those listed in Table 1; the blue-broken limits are optimal bias limits from biological variation (online Supplemental Table 6). Results For conversion of the traditional units to SI units used in the online Supplemental Figs., multiply by 0.02586 for cholesterol (mmol/L), 88.40 for creatinine (µmol/L), 0.05551 for glucose (mmol/L),and 0.02586 for HDL cholesterol (mmol/L).

10. © Copyright 2009 by the American Association for Clinical Chemistry Figure 2. Assay bias (% difference) and total error vs AMTM (LDL cholesterol, phosphate, triglycerides) or REF target values (uric acid). Abbott (red diamond), Beckman (blue square), Ortho (black triangle), Roche (yellow circle), Siemens (red square), and Thermo Scientific (blue diamond). The red and blue broken limits are the same as described for Fig. 1. Results For conversion of the traditional units to SI units used in online Supplemental Figs., multiply by 0.02586 for LDL cholesterol, 0.3229 for phosphate (mmol/L), 0.01129 for triglycerides (mmol/L), and 59.48 for uric acid (µmol/L).

11. © Copyright 2009 by the American Association for Clinical Chemistry Results CHOLCREAGLUHDLLDLPHOSTRIGLUA Bias (%) Min-16-9-8-20-17-6-15-7 Max617682513 11 Diff 12226142743192818 Diff 21122101625142116 Diff 391791321141513 Bias Low (%) Min-18-10 -16-22-8-35-7 Max62861442212914 Diff 12438163064296421 Diff 21236122252223818 Diff 3935101939212314 Bias High (%) Min-15-9-6-22-15-6-14-7 Max6106 17109 Diff 121191232 162317 Diff 21117111816121314 Diff 39168181411 13 Table 2. Observed AMTM bias in the participating laboratories. The blue, underlined values refer to maximum absolute laboratory biases >15% and differences between laboratories >30%. CHOL, cholesterol; CREA, creatinine; GLU, glucose; HDL, HDL cholesterol; LDL, LDL cholesterol; PHOS, phosphate; TRIGL, triglycerides; UA, uric acid; NA, not applicable. Diff 1, the difference between the most deviating laboratories, diff 2 and 3 the 2nd and 3rd most deviating laboratories. Note: “bias low and high” stand for the bias at the limits of the concentration range covered by the panel.

12. © Copyright 2009 by the American Association for Clinical Chemistry Limitations of the study  The CLSI C37-A protocol does not in itself prove that the sampels were commutable  The number of laboratories included was, of necessity, relatively small  Concentration ranges covered by the samples were small; no reflection of performance at pathological concentrations  Targets traceable to reference methods not available for all analytes  Selected state-of the-art limits can be debated

13. © Copyright 2009 by the American Association for Clinical Chemistry Conclusions  The results indicated room for improved quality of performance, standardization and interchangeability of results  Notable was the observation that this applies for the simple analytes (and at concentrations) assessed here  The study demonstrated that dedicated PT surveys are a powerful tool to uncover and hopefully solve certain problems  Dedicated PT surveys do not substitute conventional PT but are complementary

14. © Copyright 2009 by the American Association for Clinical Chemistry Final Comment in Editorial “For now, we should celebrate the insights that Stepman et al. have provided. As good as conventional PT may be, we can do better.” “We are indebted to these authors for shedding light on a problem we may have assumed we did not have and, more important, for providing a powerful tool to help us make things better.” Horowitz GL. Assessing Accuracy on the Front Lines: A Pragmatic Approach for Single-Donor Proficiency Testing. Clin Chem 2014.

15. © Copyright 2009 by the American Association for Clinical Chemistry Thank you for participating in this month’s Clinical Chemistry Journal Club. Additional Journal Clubs are available at www.clinchem.org Download the free Clinical Chemistry app on iTunes for additional content! Follow us