Generalities & Qualitative Testing Plans May 8-10, 2006 Iowa State University, Ames – USA Jean-Louis Laffont Kirk Remund

ISTA Statistics Committee2 Introduce Acceptance Sampling –review assumptions –definitions –understand strengths & limitations Use with a qualitative assay –zero tolerance plans –plans that allow deviants –purity testing Objectives

ISTA Statistics Committee3 0.07% 0.12% 0.09% 0.05%0.11% Seed Lot Sample Challenges: random sampling variability

ISTA Statistics Committee4 Challeges: Sampling & Assay Variability 0.12% 0.15% Seed Lot Sample Sampling Error 0.09% Sample Prep Assay (PCR) < 0.10% Assay System Error

ISTA Statistics Committee5 1. Manage sampling variability & assay errors 2. Maintain flexibility: seed pooling schemes, single or double stage testing 3. Maintain confidence in decisions –“ We are 95% confident that the GMO presence in this lot is < 0.1% ” Benefits of acceptance sampling approach

ISTA Statistics Committee6 Definition 1 –“Obtain sample so that each seed has an equal and independent chance of being selected [called a simple random sample (SRS)]” –Index every seed, pick random numbers, obtain indexed seeds –Good idea? Definition 2: mimic SRS sample –bag sampling (ISTA rules) –probe sampling (uniform grid) –systematic sampling ,000,000, Assumption: “ Representative ” Sample

ISTA Statistics Committee7 Probe sampling

ISTA Statistics Committee8 Sample a flow of seed on regular time interval –flow from hopper bottom truck –flow from a silo More samples as heterogeneity increases Sample collect from cut through entire stream of flowing seed Caution: Make sure that there is not cyclic behavior in flow that correlates with sampling interval Systematic sampling

ISTA Statistics Committee9 … seed lot primary samples composite sample submitted sample seed pools (bulks) for testing Mix well! Obtaining Pools to Evaluate Bulk Characteristics Obtain sample

ISTA Statistics Committee10 Sample size should be no larger than 10% of population This condition must hold to use Seedcalc or Qalstat If this assumption is not met we must use methods based on the hypergeometric distribution Assumption: Seed lot is large

ISTA Statistics Committee11 SEED SEED LOT SEED SAMPLE OF SEEDS X DEVIANT SEEDS FOUND X>C XCXC ACCEPT LOT REJECT LOT Acceptance sampling for qualitative assays Number of deviant seeds is distributed binomial

ISTA Statistics Committee12 Definitions LQL = lower quality limit –highest level of impurity that is acceptable to consumer –“95% confident that seed impurity is below 1%” (LQL=1%) AQL = acceptable quality level –level of impurity that is acceptable to producer and consumer –Some definitions Conservative: producer can produce seed at this impurity level or below Practical: process average Set in relation to threshold –generally, AQL less than or equal to 1/2 LQL

ISTA Statistics Committee13 Definitions, cont. 0% % impurity 0.5% LQL 0.2% Most production between 0% & this value % production process average 0.15% AQL

ISTA Statistics Committee14 Consumer Risk = chance of accepting “bad” lot (lot impurity = LQL) also called beta (  ) Producer Risk = chance of rejecting “good” lot (lot impurity = AQL) also called alpha (  ) Definitions, cont.

ISTA Statistics Committee15 want these whatever don ’ t want these Operating characteristic (OC) curve

ISTA Statistics Committee16 OC curves, cont.,

ISTA Statistics Committee17 LQL = threshold AQL = what producer can deliver LQL = 2 x threshold AQL = ½ x threshold (similar to tolerance approach) LQL & AQL in relation to threshold threshold

ISTA Statistics Committee18 Reducing Costs: Testing Seed Pools Rather than Individuals 300 seeds per pool Works well in testing for adventitious presence Assay must be able to detect one GM seed in pool of all conventional seed with high confidence 5 seed pools

ISTA Statistics Committee19 Challenge: setting the threshold Option 1: require true zero threshold result: test all seed in entire lot….. Option 2: “zero tolerance” in sample result 1: hidden non-zero threshold Example: USDA recommendation for Starlink (Cry9c), test 2400 seeds and allow zero positives yields a 0.19% threshold rather than zero. result 2: high cost to producer Throw away a lot of good seed due to false positives and sampling variability

ISTA Statistics Committee20 Challenge: setting the threshold, cont. Option 3: set reasonable non-zero threshold, allow for some positives result 1: manage consumer and producer risks to acceptable levels result 2: better manage impact of assay errors on results result 3: most seed approved for sale will be much lower than threshold (e.g., 3 or 10 times lower)

ISTA Statistics Committee21 Zero Tolerance Plans

ISTA Statistics Committee22 1% threshold Reject 0% of “ Good ” Lots Accept 0% of “ Bad ” Lots The Perfect Plan True Lot Impurity

ISTA Statistics Committee23 1% threshold Reject ~20% of “ Good ” Lots Accept <1% of “ Bad ” Lots Zero Tolerance Plan - Test one pool of 300

ISTA Statistics Committee24 1% threshold Reject 5% of “ Good ” Lots Accept <1% of “ Bad ” Lots Almost Perfect Plan: Test 6 pools of 300, accept 4 deviants pools or less

ISTA Statistics Committee25 OC curves for two testing plans threshold

ISTA Statistics Committee26 Hypothetical situation: “ Ten seed pools of 300 seeds each are tested from a conventional seed lot and 5 pools test positive for adventitious presence. The lot is labeled as having less than 1% adventitious presence and it is shipped. ” Should they have shipped the lot?

ISTA Statistics Committee27 Yes. INTERPRET WITH CARE!!

ISTA Statistics Committee28 OC Curves for two testing plans threshold

ISTA Statistics Committee29 False negative rate (FNR) –probability that a positive sample tests negative –PCR failures, DNA problems, … False positive rate (FPR) –probability that a negative sample tests positive –DNA contamination, … More definitions

ISTA Statistics Committee30 Assay Error Impact (pool size =1) 20%false negative rate 2% false positive rate 1% false positive rate No Errors 10% false negative rate

ISTA Statistics Committee31 Double Stage Testing Plan N1N1 X1X1 N2N2 X2X2 REJECT LOT ACCEPT LOT

ISTA Statistics Committee32 Trait Purity Testing Example: Testing RR Soybeans are above 98% trait purity Must test individual seeds DNA or protein assay detects intended trait rather than unintended trait in AP testing FNR has larger effect on testing plan than FPR Roles of FNR & FPR reverse in Seedcalc6 and Qalstat programs No Pooling Allowed!!

ISTA Statistics Committee33 Introduction to Seedcalc