Presentation is loading. Please wait.

Presentation is loading. Please wait.

Chemometrics Method comparison

Similar presentations


Presentation on theme: "Chemometrics Method comparison"— Presentation transcript:

1 Chemometrics Method comparison
Department of Chemical Pathology, University of Pretoria, Dr R Delport 2003

2 Performance characteristics that are taken into account:
Chemometrics Performance characteristics that are taken into account: Precision, Accuracy, Interference, Working Range, and Detection Limit.

3 Chemometrics To carry out a good method validation study, you need to do the following: Define a quality requirement for the test in the form of the amount of error that is allowable, preferably an allowable total error, Select appropriate experiments to reveal the expected types of analytical errors, Collect the necessary experimental data, Perform statistical calculations on the data to estimate the size of analytical errors, Compare the observed errors with the defined allowable error, and Judge the acceptability of the observed method performance.

4 Chemometrics An experimental plan can be formulated by:
Recognizing the types of errors that need to be assessed for this test and method, Identifying the appropriate experiments and the amount of data needed to estimate those types of errors, then Organizing these experiments to perform the quick and easy ones first and the ones taking more time and effort last.

5 Chemometrics Types of errors Imprecision or random errors,
Inaccuracy, bias, or systematic errors, which can be of two types Constant systematic error or Proportional systematic error.

6 Chemometrics The dashed line in the middle of the figure represents ideal method performance where the test method and the comparative method give exactly the same results. The bottom line in the figure shows the effect of a proportional systematic error, where the magnitude of the error increases as the test result gets higher. The top line shows the effect of a constant systematic error, where the whole line is shifted up and all results are high by the same amount. Note that these results will also be subject to the random error of the method, therefore the actual data points would scatter about the line as illustrated in the figure. The range of this scatter above and below the line provides some idea of the amount of random error that is present.

7 Chemometrics Experiments for estimating analytical errors

8 Chemometrics Experiments for estimating analytical errors
Description of different experiments. Replication experiment Provides information about random error Is performed by making measurements on a series of aliquots of the same test samples within a specified period of time, usually within an analytical run, within a day, or over a period of a month. Preliminary experiment involves determining within-run imprecision. Final experiment requires at least 20 working days to provide estimate of the total imprecision, which includes within and between run components.

9 Chemometrics Experiments for estimating analytical errors
Description of different experiments. Interference experiment Provides information about the constant systematic error caused by the lack of specificity of the method. One test sample is prepared by adding the suspected material to a sample containing the analyte. A second aliquot of the original sample is diluted by the same amount with solvent, then both samples are analyzed by the test method and the difference determined.

10 Chemometrics Experiments for estimating analytical errors
Description of different experiments. Recovery experiment Provides information about the proportional systematic error caused by a competitive reaction. Test sample is prepared by adding a standard solution of the analyte being tested to an aliquot of a patient specimen. A baseline sample is prepared by adding an equal amount of the solvent used for the standard solution to a second aliquot of the same patient specimen. The two samples are then analyzed by the test method and the amount recovered is compared to the amount added.

11 Chemometrics Experiments for estimating analytical errors
Description of different experiments. Comparison of methods experiment Is primarily used to estimate the average systematic error observed with real patient samples. Can also reveal the constant or proportional nature of that error. A series of patient specimens are collected and analyzed by both the test method and a comparative analytical method. The results are compared to determine the differences between the methods, which are the analytical errors between the methods.

12 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Get the method working and establish an operating protocol. Set up the instrument, prepare the reagents, calibrate the methods, and obtain results from test samples. Check the standards and be sure the method is properly calibrated, otherwise calibration errors will show up throughout the experimental studies.

13 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Determine the working range. The working range will vary from test to test and must be defined as part of the specifications for the method. Check by analyzing a series of solutions, in duplicate or triplicate, covering the concentrations range of interest. If detection limit is a critical characteristic, it may be assessed at this time or in the next phase of preliminary experiments.

14 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Determine within-run imprecision, recovery, and interference. The replication experiment might include 20 samples of two or three materials whose concentrations closely match the medical decision levels of interest for the tests. Interference experiments should test common problems such as hemolysis, lipemia, and high bilirubin. Recovery experiments assess whether there are any competitive reactions due to the matrix or other materials in the native specimens.

15 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Comparison of methods experiment. Fresh patient specimens, and stored specimens. Minimum of 40 well-chosen patient samples should be tested over a minimum of 5 working days. Distributed one-third in the low to low-normal range, one-third in the normal range, and one-third in the high abnormal range. Method acceptability should be judged on the basis of the sizes of the random, systematic, and total analytical errors.

16 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Comparison of methods experiment. Difference plot Used as initial graph if the two methods are expected to show one-to-one agreement, Displays the difference between the test minus comparative results on the y-axis versus the comparative result on the x-axis, such as shown in the accompanying figure. 

17 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Comparison of methods experiment. Difference plot Used as initial graph if the two methods are expected to show one-to-one agreement, Displays the difference between the test minus comparative results on the y-axis versus the comparative result on the x-axis, such as shown in the accompanying figure.  Differences should scatter around the line of zero differences, half being above and half being below.  Repeat measurements if indicated.

18 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Comparison of methods experiment. Correlation coefficient The correlation coefficient is a statistic that is almost always calculated and reported to describe the results from a comparison of methods study. The new or "test" method values are plotted on y-axis and comparison values on x-axis.

19 Chemometrics Experiments for estimating analytical errors
Walking tour of the plan. Comparison of methods experiment. A value of indicates perfect correlation between the results of two methods. Other statistics (such as slope, intercept, and standard deviation of the residuals) can also be calculated from the same data to estimate the size of errors occurring between the methods.

20 Chemometrics Analytical errors Random error
Random error, RE, or imprecision Can be either positive or negative. Direction and exact magnitude cannot be predicted. Imprecision is quantitated by calculating the standard deviation (SD) from the results of a set of replicate measurements. Random error

21 Chemometrics Analytical errors Random error
Random error, RE, or imprecision Can be either positive or negative. Direction and exact magnitude cannot be predicted. Imprecision is quantitated by calculating the standard deviation (SD) from the results of a set of replicate measurements. As the SD often increases as the concentration increases, the coefficient of variation (CV) is calculated to express the SD as a percentage of the mean concentration from the replication study. Maximum size of a random error is commonly expressed as a 2 SD or 3 SD estimate to help understand the potential size of the error that might occur.  Random error

22 Chemometrics Analytical errors Systematic error
Systematic error, SE, or inaccuracy Is always in one direction. A systematic shift displaces the mean of the distribution from its original value. In contrast to random errors that may be either negative or positive and whose direction can not be predicted, systematic errors are in one direction and cause all the test results to be either high or low.  Systematic error

23 Chemometrics Analytical errors Systematic error
Systematic error, SE, or inaccuracy How high or how low can be described by the bias, which is calculated as the average difference, or the difference between averages, between the value by the "test" method and a "comparative" method in a comparison of methods experiment. Alternatively, the expected systematic difference may be predicted from the equation of the line that best fits the graphical display of test method values on the y-axis vs comparative method values on the x-axis. Systematic error

24 Chemometrics Analytical errors Systematic error
Systematic error, SE, or inaccuracy How high or how low can be described by the bias, which is calculated as the average difference, or the difference between averages, between the value by the "test" method and a "comparative" method in a comparison of methods experiment. Alternatively, the expected systematic difference may be predicted from the equation of the line that best fits the graphical display of test method values on the y-axis vs comparative method values on the x-axis. SE may stay the same over a range of concentrations, in which case it can also be called constant error, or it may change as concentration changes, in which case it can be called proportional error. Systematic error

25 Chemometrics Analytical errors Total error Total Error, TE,
It is the net or combined effect of random and systematic errors, as shown in the accompanying figure. Total error

26 Chemometrics Analytical errors Total error Total Error, TE,
It is the net or combined effect of random and systematic errors, as shown in the accompanying figure. It represents a "worst-case" situation, or just how far wrong a test result might be due to both random and systematic errors. Because laboratories typically only make a single measurement for each test, that measurement can be in error by the expected SE, or bias, plus 2 or 3 SD, depending on how you quantitate the effect of RE.  Total error

27 Chemometrics

28 Chemometrics

29 Chemometrics


Download ppt "Chemometrics Method comparison"

Similar presentations


Ads by Google