1. Multi-laboratory Method Validation
ทิพวรรณ นิ่งน้อย
สคอ.

2. Method Validation Definition (VIM)
Verification, where the specified requirements are adequate for an intended use
Example: A measurement procedure, ordinarily used for the measurement of mass concentration of nitrogen in water, may be validated also for measurement in human serum.
[Verification:
Provision of objective evidence that a given item fulfills specified requirements
Item = a process, measurement procedure, material, compound, or measuring system.]
23/05/62

3. Method Validation Definition (ISO…..)
The confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fulfilled.
23/05/62

4. Method Validation Single laboratory validation
Within a laboratory
Multi-laboratory method validation
Collaborative study (inter-laboratory study)
23/05/62

5. Inter-laboratory study
Evaluation and monitoring of laboratory performance; (PT)
Identification of problems in laboratory and initiation of actions for improvement; (PT)
Establishment of the effectiveness and comparability of test methods; (PT)
Provision of additional confidence to laboratory customers; (PT)
Identification of interlaboratory difference; (PT)
Education of participating laboratories; (PT)
Validation of uncertainty claims; (PT)
Evaluation of performance characteristics of a method; (collaborative trial-method validation)
Assignment of values to reference materials; (certification)
Support for statements of the equivalence of measurements of NMIs. (key comparison)
23/05/62

6. Method Validation by Inter-laboratory study
Collaborative Study-a test of the method
To evaluate how a method will perform in the future in actual practical applications in real life settings
To provide confidence in the performance statistics
To demonstrate the reproducibility of the method
To provide opportunities for improvements of the method
23/05/62

7. Guidelines for Collaborative Study
Preliminary work (within one-lab)
Design of the collaborative study
Preparation of materials for collaborative studies
Submission of test samples
Statistical analysis
AOAC/IUPAC
23/05/62

8. 1. Preliminary work (within one-lab)
Determine purpose and scope of the study and method
Optimize either new or available method
Develop within-lab attributes of optimized method
Prepare description of method
Invite participation
Instruction and report forms
Familiarization or practice samples
23/05/62

9. 1. Preliminary work (within one-lab)
Determine purpose and scope of the study and method
Purpose …..
Type of the method …..
Probable use of the method ..…
Design of the study (attributes) ..…
Optimize either new or available method
Develop within-lab attributes of optimized method
Prepare description of method
Invite participation
Instruction and report forms
Familiarization or practice samples
23/05/62

10. 1. Preliminary work (within one-lab)
Determine purpose and scope of the study and method
Optimize either new or available method
Ruggedness
Interferences
Within lab reproducibility
Develop within-lab attributes of optimized method
Prepare description of method
Invite participation
Instruction and report forms
Familiarization or practice samples
23/05/62

11. 1. Preliminary work (within one-lab)
Determine purpose and scope of the study and method
Optimize either new or available method
Develop within-lab attributes of optimized method
Calibration function
Specificity
Bias
Precision
Matrices
Prepare description of method
Invite participation
Instruction and report forms
Familiarization or practice samples
23/05/62

12. 1. Preliminary work (within one-lab)
Determine purpose and scope of the study and method
Optimize either new or available method
Develop within-lab attributes of optimized method
Prepare description of method
Format and style
Clear explanation
Imperative directions; avoid subjunctive and conditional expression as options as far as possible.
Invite participation
Instruction and report forms
Familiarization or practice samples
23/05/62

13. 1. Preliminary work (within one-lab)
Determine purpose and scope of the study and method
Optimize either new or available method
Develop within-lab attributes of optimized method
Prepare description of method
Invite participation
Selection of collaborators/candidate laboratories
Letter of invitation
Laboratory coordinator
Instruction and report forms
Familiarization or practice samples
23/05/62

14. 1. Preliminary work (within one-lab)
Determine purpose and scope of the study and method
Optimize either new or available method
Develop within-lab attributes of optimized method
Prepare description of method
Invite participation
Instruction and report forms
Carefully design
Send independently from study materials
“THIS IS A STUDY OF THE METHOD, NOT OF LABORATORY. THE MTHOD MUST BE FOLLOWED AS CLOSELY AS PRACTIICABLE, AND ANY DEVIATIONS FROM THE METHOD AS DESCRIBED, NO METTER HOW TRIVIAL THEY MAY SEEM, MUST BE NOTED ON THE REPORT FORM.”
Familiarization or practice samples
23/05/62

15. 2. Design of the collaborative study
General principles
Laboratories
Test materials
Replication
Other design considerations
23/05/62

16. Minimum Criteria Quantitative stuay No. Note Material/ matrix 5 3
If 1 level 1 matrix
Laboratory 8
Competent and fully capable
Special cases
Replicate 2 1
If within-lab repeatability not desired
23/05/62

17. Matrix/material
Represent common matrices that the method is likely to be used on in practice
Materials with naturally occurring concentrations are preferred over spiked samples
Homogeneous and stable
Minimize variance estimate
23/05/62

18. Laboratory
At least 8 labs for statistical validation of the performance parameters
Should start with 12 labs
Error
Non-participation
Unforeseen difficulties
Should be representative of the labs where you expect your methods will be used
23/05/62

19. Concentrations Test the full range of the method’s scope as written
Prepare samples of analyte at levels to bracket and cover this area of interest
23/05/62

20. 3. Preparation of materials for collaborative studies
General principles
Materials suitable for collaborative studies
Blanks
Limit of detection/quantitation
controls
23/05/62

21. 3. Preparation of materials for collaborative studies
General principles
Materials suitable for collaborative studies
Material and analyte stability
Single batch of homogeneous, stable product
Reference materials
Synthetic materials
Spiked materials
Blanks
Limit of detection/quantitation
controls
23/05/62

22. 3. Preparation of materials for collaborative studies
General principles
Materials suitable for collaborative studies
Blanks
Matrix blanks
Reagent blanks
Limit of detection/quantitation
controls
23/05/62

23. 3. Preparation of materials for collaborative studies
General principles
Materials suitable for collaborative studies
Blanks
Limit of detection/quantitation
Definition
False positives and false negatives
controls
23/05/62

24. 3. Preparation of materials for collaborative studies
General principles
Materials suitable for collaborative studies
Blanks
Limit of detection/quantitation
Controls
Prepare more sets of laboratory samples (e.g. 25% more)
Some packages may never arrive
Some materials may spoil
Some may be lost or the container broken
23/05/62

25. 4. Submission of test samples
Sending collaborative study material
Obligations of collaborators
23/05/62

26. Mini-collaborative Study
To see any readily identifiable and fixable problems
To ensure homogeneity and stability of the samples
Unofficial
2-3 friendly labs
Data included in the whole collaborative study
23/05/62

27. 5. Statistical analysis Initial review of data (data audit) Outliers
Bias (systematic deviation) of individual results
Precision
HORRAT
Incorrect, improper, or illusory values (false positive and false negative values)
Final collaborative study manuscript
23/05/62

28. Outlier Consideration
Excessive number of outlier is more than two-ninths of the data
A method which produces outliers at greater than two to nine ratio is considered to be unstable
Outliers observed in the course of the collab must be investigated
Sources of outliers
Simple mistake
Unusual chemical reaction or condition
23/05/62

29. Calculation Outlier tests Cochran test for individual measurements
1-tail test: P = 2.5%
Grubbs test for lab outliers (extreme average)
Single value test : 2-tail; P = 2.5%
Pair value test (2 at the highest end)
Pair value test (2 at the lowest end)
Pair value test (one at each end)
23/05/62

30. Performance Parameters
Bias (systematic deviation) of individual results (Mean or average value)
Recovery study
Marginal %Rec = 100RM = 100(Cf-Cu/CA)
Total %Rec =100RT = 100(Cf)/(Cu+CA)
Precision (Standard deviation)
repeatability
Reproducibility
Relative standard deviations
Repeatability value
Reproducibility value
HorRat
23/05/62

31. Repeatability and Reproducibility
Repeatability standard deviation (Sr)
within laboratory standard deviation
Reproducibility standard deviation (SR)
includes both the within and between laboratory variance
23/05/62

32. Calculation
Performance statistics
ANOVA
23/05/62

33. Repeatability and Reproducibility
Repeatability value (r)
Internal precision
Two single results obtained within a lab under repeatability conditions should not differ by more than r
r = 2.8xSr
Reproducibility value (R)
External precision
Two single results obtained within a lab under reproducibility conditions should not differ by more than R
R = 2.8xSR
23/05/62

34. HORRAT Ratio of the RSDR, % to the PRSDR, % HORRAT = RSDR, % /PRSDR, %
PRSDR, % = 2C
C = the estimate mean concentration
HORRAT value
< 0.5
>1.5
>2.0
23/05/62

35. Case Study
Method performance study for total solids and total fat in coconut milk and products
23/05/62

36. 23/05/62

37. Inter-laboratory study
Table 3: Conclusion of method performance for total solids determination
Test materials
Light coconut milk
Coconut milk
Coconut cream
Coconut milk powder
No. of laboratory
(No. of outlier)
17(1)
17(3)
17(0)
Mean (g/100g)
8.84
10.57
20.90
27.01
28.97
98.82
99.25
Repeatability standard deviation, Sr (g/100g)
0.02
0.03
0.12
0.49
0.15
0.04
0.10
Repeatability relative standard deviation, %RSDr
0.21
0.30
0.59
1.83
0.53
Repeatability limit, r=2.8 Sr
0.05
0.09
0.35
1.38
0.43
0.28
Reproducibility standard deviation, SR(g/100g)
0.11
0.22
0.60
0.31
Reproducibility relative standard deviation, %RSDR
0.50
1.05
2.24
0.95
0.32
Reproducibility limit, R=2.8 SR
0.61
1.69
0.77
0.88
23/05/62

38. Inter-laboratory study
Table 4: Conclusion of method performance for fat determination
Test materials
Light coconut milk
Coconut milk
Coconut cream
Coconut milk powder
No. of laboratory
(No. of outlier)
17(2)
17(4)
17(1)
17(3)
Mean (g/100g)
5.88
8.28
16.85
21.83
23.02
42.90
42.44
Repeatability standard deviation, Sr (g/100g)
0.03
0.16
0.76
0.23
0.28
0.11
0.17
Repeatability relative standard deviation, %RSDr
0.59
1.88
4.49
1.04
1.22
0.26
0.41
Repeatability limit, r=2.8 Sr
0.10
0.44
2.12
0.63
0.79
0.31
0.49
Reproducibility standard deviation, SR(g/100g)
0.15
0.89
0.74
0.46
0.34
0.40
Reproducibility relative standard deviation, %RSDR
2.7
1.77
5.30
3.4
2.02
0.80
0.94
Reproducibility limit, R=2.8 SR
2.50
2.08
1.30
0.97
1.11
23/05/62

39. Mini-collaborative Study
To see any readily identifiable and fixable problems
To ensure homogeneity and stability of the samples
Unofficial
2-3 friendly labs
Data included in the whole collaborative study
23/05/62

40. Qualitative Study Minimum criteria 10 Labs 2 levels/matrix 1 matrix
Analyte level 2
Analyte level 1
negative control
6 samples
6 samples
6 samples
23/05/62