5Quality Terminology QA - Quality Assurance Planned, systematic actions providing confidence that a quality output will be producedLaboratory ProceduresQC - Quality Control (QC)Procedures use to assess validity of results in real time, controls release of resultsQC material run with patient samplesEQA - External Quality Assessment (PT)Procedures operated by an external agency which allow retrospective review of performanceRCPA-AACB
6Running a QC Program Selection of material (matrix) Selection of levels (decision points)Setting of targets and rangesDecision on frequency (batch vs RA)Decision on number of QC samples (n)Decision on rules and interpretationResponse to out-of-range valuesQuality planning
7QC Quiz Your new trainee scientist asks you the following question: “In our lab, how far can the results of an assay vary from the actual concentration in the sample?”What is the answer:+/- 1SD; 2 SD; 3 SD; 4 SD; 5 SD ?
8Assay Characteristics Stable assays:Performance defined by mean and SDQC never failsResults “always” within +/- 2SD
9Stable Assay Mean = 20, SD = 1 95% of QC results between 18 and 22. Interpretation: A result of 20 has a 95% confidence interval of 18 to 22.
10Unstable Assays Mean drifts over time (fluctuating bias) QC process used to detect driftsVariation in results due to scatter plus drift
11Unstable Assay Mean = 20, SD = 1, Plus fluctuating mean. Interpretation: Result of 20 has 95% confidence limit of18 – 22, PLUS bias at time of assay.
12Unstable assaysHow bad can this be?How can we measure this?
19Power Function Chart N=2 13s/22s/R4s 90% error detection at 3.2 x SD Probability of Rules FiringShift in Mean (multiples of SD)90% error detection at 3.2 x SD
20Shifts and Results (unstable assay) Imprecision: up to 2 SD.Undetected shifts in mean: 3 SDTotal spread: up to 5 SDWith the assay still “in control”!+3 SD+2 SD+5 SD
21QC Quiz The result may differ from the correct result by addition of: random variation of the assay (up to 2 x SD)the undetected bias at the time of the assay (up to 3 x SD).ExampleCV cholesterol assay: 2.0%At 6 mmol/L, 5 x SD = 0.6 mmol/LAccumulation of errors all in the same direction is rare, but can happen.
22Understanding our assays For any assay, with the QC protocol in place, we should be able to say how much analytical error may occur.“These rules have the power to cause a STOP 90% of the occasions when there is a shift in the assay of 2.8 x LSD and cause a PAUSE 90% of the occasions when there is a shift in the assay of 2.6 x LSD.”- SydPath Quality Control SOP
23Westgard - Quantifying QC With the rules I have in place, what shifts in assay performance can I detect.orHow can I be sure that I can detect important changesCapabilitySetting QC protocolsWhat are “important changes”
24Capability Capable assays are easily able to “do their job” Capable assays (almost) never produce results outside important limits.Poorly capable assays will produce results outside the set limits.Capable assayIncapable assay
25CapabilityGood assays (capable) have an analytical performance (SD) which is much less than the clinically important change.This can be quantified as the Capability Index: Cp=ALP/SD(ALP=Allowable limit of Performance)>6 great; OK; <4 poor
27CapabilityCapability is the concept we use to discuss quality of assay performance.Relates assay precision to required precision.
28Reverse Engineering QC If we have limits to our assay performance we want QC protocols which allow us to detect assay problems before “wrong” results may be issued.Choose QC protocols which allow appropriate error detection.Use Power Function Charts….
30Setting QC Protocols Capable assays – simple rules Poorly capable assays – Need:More rulesMore QCIncapable assays – will not achieve target performanceHave to accept less chance of finding shiftsOr choose better assay
31Quality Specifications Hierarchy How good do we need to be:1. Proven analyte-specific data on clinical decision making2. General-clinical decision makingBased on biological variationBased on medical opinions3. Professional recommendations4. Regulations or EQA targets5. Published state-of-the-art data
32Within-subject biological variation Variability in patient results due to changes in the patient.ArchivesReference Material (near bottom on right)Biological Variation Databaseeg Sodium: 0.7%; ALT 24.3%
33Analytical v Biological CV CVa/CVbRelative total CV