Method Comparison A method comparison is done when: A lab is considering performing an assay they have not performed previously or Performing an assay using a different instrument, method or reagent system

Examples include New method not offered be that lab previously Lab is currently performing an assay using a manual method and would like to use an automated method Lab is performing an assay on an older instrument but would like to move that assay to a newer instrument New producer of reagents or in some cases even a new lot of reagents

Method Comparison includes the following Precision Linearity Correlation Accuracy

Precision Study Precision refers to the reproducibility (repeatability) of the method Two Types of Precision are examined Intra run precision Inter run precision (Total Imprecision)

Intra run Precision Usually two levels of controls are used Not the same controls used to check QC One should be in the normal range and the other at or near “Clinical Decision” level Each is run about 20 times all in one run The mean, Std Deviation and CV are calculated

Inter run Precision or Total Imprecision The same controls used for the Intra run Precision study are used They should be run over a course of days or longer, a couple of times on each shift the assay will be normally be performed For example an assay that is done on the day and evening shift, the control would be run twice on each of those shifts for 5 days for a total of twenty results

Statistics for Precision Studies Both methods should be about the same Mean The average of the results Standard Deviation A measure of the amount of scatter about the mean Coefficient of Variation S.D./Mean x 100 = ___% C.V. allows for a better comparison of results expressed in different units or at different levels

Linearity To perform a linearity study the lab must have a substance at various levels Either a substance can be bought, or made, or sometimes a patient sample with a very high value is used Each level is run at least in duplicate and then graphed to look for linearity The highest value and the lowest value are the range that the lab can report Specimens above the highest must be diluted. Specimens below the lowest value are reported as < that value

Correlation A correlation study is done to determine if the result you get using one method will be the same as you get using the other method. To perform a correlation study about 60 specimens are run with both methods Samples are collected that have values that represent the entire assay range The slope, y intercept and r value are determined and the line is drawn on a graph

Correlation Statistics The black line shows “perfect” correlation It has a slope of 1 and a Y intercept of 0 The result using the Test Method will be the same as the result using the Comparative Method

Slope Correlation Statistics Slope The slope of the line should be 1 The black line has a slope of 1 The red line has a slope less than 1 maybe “0.92” showing proportional systematic error

Systematic Error Proportional Systematic Error: Varies with the magnitude of the value, often linearly, and corresponds to a non-unity slope of the correlation plot. Generally reflects a failing of the methodology. Sometimes, may be simply fixed by recalibration and, other times, may be more insidious.

Y intercept Correlation Statistics Y intercept The y intercept of the line should be 0 The blue line has a y intercept > 0 maybe “10” showing constant systematic error

Systematic Error Constant Systematic Error: A constant difference between the measured and true values and corresponds to the “vertical offset” of a correlation plot. Often caused by an interfering substance. Can be practically eliminated by a proper “blank” as long as the interference is constant.

Correlation Graph

Accuracy Accuracy is how close to the real value To perform an accuracy study a substance with a known value is run and the results you get are compared to the expected value

Steps Required to Validate an Assay (in no particular order) Establish Accuracy Establish Precision Establish the Reportable Range (linear range) Determine the Analytical Sensitivity and the Lower Limit of Detection Investigate the Analytical Specificity (search for interferences) Establish the Reference Range (define medically relevant decision points).

Correlation Coefficient -1 to +1

Standard Error Put simply, the standard error of the sample is an estimate of how far the sample mean is likely to be from the population mean, whereas the standard deviation of the sample is the degree to which individuals within the sample differ from the sample mean. If the population standard deviation is finite, the standard error of the sample will tend to zero with increasing sample size, because the estimate of the population mean will improve, while the standard deviation of the sample will tend to the population standard deviation as the sample size increases.