1. © Copyright 2009 by the American Association for Clinical Chemistry Glucose Meter Performance Criteria for Tight Glycemic Control Estimated by Simulation Modeling B.S. Karon, J.C. Boyd, and G.G. Klee July 2010 http://www.clinchem.org/cgi/reprint/56/7/1091 © Copyright 2010 by the American Association for Clinical Chemistry Journal Club

2. © Copyright 2009 by the American Association for Clinical Chemistry Introduction  Current Glucose Meter Performance Criteria Developed to determine accuracy needs required to support subcutaneous insulin dosing Most often displayed as error grid  Error Grid Translates Accuracy into Clinical Implications for Subcutaneous Dosing

3. © Copyright 2009 by the American Association for Clinical Chemistry Introduction (cont)  Error Grid Example

4. © Copyright 2009 by the American Association for Clinical Chemistry Introduction (cont)  Advantage of Error Grid Analysis Translates error into clinical impact Visual display of current guidelines (ISO 15197) ± 15 mg/dL at glucose < 75 mg/dL ± 20% at glucose ≥ 75 mg/dL

5. © Copyright 2009 by the American Association for Clinical Chemistry Introduction (cont)  Limitations of Error Grid Analysis Only meaningful for subcutaneous insulin dosing Every meter looks good (majority of results fall into A and B)

6. © Copyright 2009 by the American Association for Clinical Chemistry Question  How do you determine performance criteria necessary for tight glycemic control (TGC)? Scott et al. 1 have reviewed the controversy in this area Outcome studies ideal, but resource intensive Several comparison studies done, conflicting results Fundamental problem, no agreement on how to establish accuracy needs for TGC Error simulation modeling has been used to establish performance criteria for subcutaneous insulin dosing 1 Clin Chem 2009;55:18–20.

7. © Copyright 2009 by the American Association for Clinical Chemistry Methods Start with distribution of observed glucose values during TGC for 2 surgical ICU units during a 6-month period; 86% values ≤ 150 mg/dL, prevent hypoglycemia.

8. © Copyright 2009 by the American Association for Clinical Chemistry Methods (cont) Category Glucose value (mg/dL) Insulin dose (Units/hour) 0< 800 180-900-2 291-1101-3 3111-1302-4 4131-1503-6 5151-1754-10 6176-2006-12 7201-25010-16 8251-30012-20 9301-35014-24 10351-40016-27 11> 40018-30 Glucose value determines insulin dose category. Example of 3 category dosing error is real value in category 0, simulated value in category 3.

9. © Copyright 2009 by the American Association for Clinical Chemistry Methods (cont)  Sample this distribution. For each initial value sampled, simulate 10,000 values with distribution of bias and imprecision: Glucose (simulated) = Glucose initial + [n(0,1) x CV x glucose (initial)] + [Bias x glucose (initial) n(0,1) random number drawn from gaussian distribution centered on zero with SD = 1 Bias (expressed as a fraction of the glucose concentration) is varied from -0.2 to +0.2 in 0.01 increments For each Bias, CV (also expressed as a fraction of glucose concentration) is varied from 0.00 to 0.20 in 0.01 increments

10. © Copyright 2009 by the American Association for Clinical Chemistry Methods (cont)  Calculate percentage simulated values that fall in same insulin dosing category as initial  Calculate percentage 1, 2, or ≥ 3 category dosing errors based on Mayo TGC protocol  Express results as contour plots, showing percentage dosing errors as a function of bias and imprecision

11. © Copyright 2009 by the American Association for Clinical Chemistry Methods (cont)  Superimpose boundaries for 10%, 15%, and 20% total error (TEa) on contour plots  Acceptable performance defined as ≥ 3 category insulin dosing errors occurring < 0.2% of the time Based upon impact of 3 category error on insulin dose given distribution of glucose values and ability to predict percent errors with reasonable confidence interval

12. © Copyright 2009 by the American Association for Clinical Chemistry Results

13. Results (cont)  Up to 60% one category dosing errors when 10% TEa is simulated  Up to 80% one category dosing errors when 15% TEa is simulated  Up to 90% one category dosing errors when 20% TEa is simulated

14. © Copyright 2009 by the American Association for Clinical Chemistry Results

15. Results (cont)  Only 0.2% two category dosing errors when 10% TEa is simulated  Up to 5% two category dosing errors when 15% TEa is simulated  Up to 20% two category dosing errors when 20% TEa is simulated

16. © Copyright 2009 by the American Association for Clinical Chemistry Results Only the 20% TEa condition was associated with any frequency of 3 or more category insulin dosing errors

17. © Copyright 2009 by the American Association for Clinical Chemistry Methods  Second simulation model based on gaussian distribution of simulated results around each of 29,920 initial values Generate 1000 simulated values with distribution of X% error using SAS (Carey, NC) Determine how many simulated values would change insulin dosing category relative to original value

18. © Copyright 2009 by the American Association for Clinical Chemistry Methods (cont) Error condition 10 % error 15 % error 20 % error No change dose71.4 %58.7 %48.8 % 1 category dose28.4 %39.3 %44.8 % 2 category dose0.2 %2.0 %6.1 % ≥ 3 category dose0.0 %0.02 %0.3 %

19. © Copyright 2009 by the American Association for Clinical Chemistry Methods (cont)  Decreasing acceptable error tolerance from 20% to 10% will decrease 2 category errors Additional studies necessary to understand impact of 2 category dosing errors  Only 20% TEa criteria allowed 3 category dosing errors Most dangerous as could lead to hypoglycemia

20. © Copyright 2009 by the American Association for Clinical Chemistry Conclusions  Only 20% TEa condition allowed 3 category or critical errors in either model Imprecision drives 3 category dosing errors  So far models predict that meters that maintain 15% TEa and minimize imprecision may be safe and effective for TGC monitoring

21. © Copyright 2009 by the American Association for Clinical Chemistry Conclusions (cont)  Models do not account for random patient interferences (such as effects of variation due to pO2, hematocrit, or sampling), which would increase the total error of glucose measurements, leading to the need for more stringent performance criteria  Models assume that a single dosing error leads to patient harm More complex models needed to understand cumulative dosing errors