1. APCCB 20041 AVERAGE OF DELTA – A NEW CONCEPT IN QUALITY CONTROL GRD Jones Department of Chemical Pathology, St Vincents Hospital, Sydney, Australia

2. APCCB 20042 Background The Average of Normals (AON) is an accepted QC process for clinical laboratories. AON is the average of a set number of patient results, usually within set limits (eg normal range). The AON rule “fires” when the function exceeds a pre-set limit (eg 2.5 x analytical CV). Delta checks are the comparison of a result with a previous result from the same patient. Delta checks are used to detect blunders or other errors I combine these concepts to produce the Average of Delta (AOD) a new QC tool for clinical laboratories.

3. APCCB 20043 Terminology A Delta Value is a recent patient result minus the preceding result for that patient. An AOD function is the average of a series of delta values. AOD N is an AOD function averaging N delta values. N AOD is the number of samples included in an AOD function. N 90 is the number of samples with valid previous result, required to detect a change in assay bias with 90% probability using an AOD function. CV wi is the within-individual Biological Variation. CV a is the analytical variation expressed as a CV. SD AOD is the SD of the AOD function

4. APCCB 20044 Methods AOD functions were modelled in a spreadsheet application using Microsoft Excel. Variations in CV a and CV wi were modelled using the random number generator with a Normal distribution. Models were based on 100 data sets, each of 110 delta values. Factors adjusted in the model were: –the ratio of Cv a /Cv wi –N AOD –Bias changes in assay performance.

5. APCCB 20045 Modelling Equations Data sets were generated for various values of CV a and CV wi with the variation (CV data set ) in results described as follows CV data set = SQRT(CV a 2 + CV wi 2 ) Second data sets were independently generated using the same values for CV a and CV wi Delta Values were were obtained by subtracting the data points from the second data set from those in the first to produce a series of delta values. Changes in bias were modelled by addition of fixed amounts to the delta values at a fixed point in the data set. AOD functions were set to trigger if a data point fell outside limits defines by +/- 2.5 SD AOD.

6. APCCB 20046 AOD Functions Figure 1 AOD functions for various values of N AOD. CV a = 0.1, CV wi = 0.2 As the value of N increases, the scatter of the AOD function decreases. The decrease in SD with increasing N is equal to dividing by the square root of N. (data not shown) Purple: n=2 SD = 0.22 Red: n=10 SD = 0.10 Blue: n=50 SD = 0.045 AOD value Sample Number

7. APCCB 20047 Effects of Bias on AOD Functions A fixed bias was added after delta value 10 in each data set. The AOD function followed the change in bias with the following features: –With smaller values of N AOD, the response occurred more rapidly, but was smaller relative to the scatter of the AOD function –With higher values of N AOD, the response was slower, but was larger relative to the scatter of the AOD function. Examples are shown in figure 2.

8. APCCB 20048 Figure 2. AOD functions for N AOD of 2, 10 and 50. CV a = 0.1, CV wi = 0.2 A fixed bias of 0.3 was introduced at sample 10. The red lines show the 2.5 x SD AOD. For AOD 2 the value is 0.56; for AOD 10, 0.24; for AOD 50, 0.11. Five example AOD functions are shown in each graph ( ). Blue arrows show N 90. Orange arrows show average first rule firing. Sample Number 36 20 30 14 10 18 AOD 50 AOD 10 AOD 2

9. APCCB 20049 Error Detection with AOD Error detection of bias can be measured as the number of delta values (samples with previous results) required to detect changes in bias with specified certainty. N 90 and average first detection for a range of values for Cv a /Cv wi and N AOD are shown in figure 3. The following conclusions can be drawn: –Earlier error detection occurs with lower values of Cv a /Cv wi –Error detection generally varies with N AOD in a “U-shape” with an optimal range of values depending on CV a /CV wi. Examples of actual values for CV a /CV wi are in the table.

10. APCCB 200410 Figure 3 Number of samples required for average 1st firing ( A ) and N 90 ( B ) for detection of a shift of 2.5 x CV a for various values of CV wi /CV a and N AOD. Earlier error detection with lower CV wi /CV a Optimal error detection with N AOD 5-20 B A N 90 Average 1 st Firing CV wi /CV a

11. APCCB 200411 Table. Examples of Cv wi / CV a. CV wi from Westgard Website (www.westgard.com) CV a from SydPath Laboratory (Olympus AU2700)

12. APCCB 200412 Discussion The limitation of AON is the ratio of group biological variation to analytical CV. AOD may outperform AON if: –CV wi is small compared to between-person biological variation (a low Index of Individuality). –The frequency of samples with previous results is high. –Clinics or weekends affect AON results. Note than AOD should not be affected by change in patient mix as it uses patients as their own control. AON may complement standard QC if it can: –Detect smaller errors than standard QC –Detect errors before standard QC –Allow less frequent use of standard QC

13. APCCB 200413 Conclusions Average of Delta may allow improved error detection without additional QC testing. The process would most suit tests as follows: –A low within-individual biological variation compared to the analytical variation. –A high frequency of repeat testing. Software programs must be written to further evaluate and this tool and allow for use in the routine environment.