1. High-Confidence, Low- Probability-of-Failure Screening January 11, 2014 RLGM? P[P[Failure|Eq]<0.05]  95% Magnitude

2. HCLPF depends on whom you believe Which components need seismic safety improvements? – If HCLPF < RLGM (Review Level Ground Motion), improve them – “HCLPF is defined as the earthquake [ground] motion level at which there is a high (95 percent) confidence of a low (at most 5 percent) probability of failure” Interim Staff Guidance, DC/COL-ISG-020 HCLPF: SoV vs. CDFM – A m Exp[-2.32  C ] “CDFM” (Conservative Deterministic Failure Margin, “because engineers understand it”) [Kennedy et al.] – A m Exp[-1.65(  R +  U )] [“SoV” (Separation of Variables), (EPRI TR-103959, Ravindra, and others] A m should be the a “High-Confidence” limit on the natural logarithm of median strength -1.65 and -2.32 are the number of 5% and 1% standard deviations below the normal distribution [log] mean A m The  -parameters represent randomness or aleatory variability (R), epistemic uncertainty (U), or combined uncertainty (C) Notice any sample uncertainty?

3. Objective of Seismic Screening: Component HCLPF < RLGM? Guidance unspecific, encourages industry initiative, abhors statistics? – ASCE/SEI 41-06 and -13, FEMA E-74 Chapter 4.1, and IEAE NS-G- 1.6 – RG 1.208, “A Performance-based Approach to Define the Site- specific Earthquake Ground Motion” (NRC, 2007b) “The desired performance being expressed as the target value of 10 -5 for the mean annual probability of exceedance (frequency) of the onset of significant inelastic deformation “ Plants with higher seismic risk are already shut down – Vallecitos, Humboldt Bay, Zion, Fukushima Daiichi, San Onofre – Older plants have little time left, except for 20-or-30-year extensions!

4. Alternative Uncertainty Model Q = P[f < f’ | a]; i.e., the subjective probability (confidence) that the conditional probability of failure, f, is less than f’ for a peak ground acceleration a (e.g., RLGM)  -1 (.)= the inverse of the standard Gaussian cumulative distribution function.

5. How about some Rational Economic Resource Allocation? Cost-benefit analysis and budget constrained resource allocation is not under current consideration – Cost of alternatives? Fix some? Shut down plant? How to allocate resources to aleatory (random) or epistemic (unknown) uncertainty? – Is reducing epistemic uncertainty [unknown unknowns] a scam? Quants use scenarios. Elizabeth Paté-Cornell says insure or hedge (Allin’s wife) Price-Anderson act provides insurance to nuclear utility industry NRC insures industry for costs incurred by delays caused by NRC – How to allocate to high confidence vs. low probability of failure “The public appears to heavily value confidence and places a much smaller, although still positive, emphasis on accuracy.” [Smith and Wooten]

6. Legal, Precautionary Principle, and Social Equity “…if an action or policy has a suspected risk of causing harm to the public or to the environment, in the absence of scientific consensus that the action or policy is harmful, the burden of proof that it is not harmful falls on those taking an action.” http://en.wikipedia.org/wiki/Precautionary_principle http://en.wikipedia.org/wiki/Precautionary_principle Search for “Kip Viscusi and Risk Equity” for guidance ISO 26000 Social Responsibility, ISO 16269-6 “Determination of Statistical Tolerance Intervals”

7. HCLPF Borrows Credibility To statisticians, HCLPF is a tolerance limit or a confidence limit on a sample percentile (ISO 16269-6 and http://en.wikipedia.org/wiki/Tolerance_limit http://en.wikipedia.org/wiki/Tolerance_limit – The “High-Confidence” part of HCLPF corresponds to the statistical confidence limit, typically 95% – The “Low-Probability-of-Failure” part of HCLPF corresponds to the percentile, typically 5-th percentile of the fragility function – Tolerance limits are supposed to be estimated from data, perhaps censored, perhaps extrapolated To reliability engineers, HCLPF is a R95C95 reliability demonstration test: 95% confidence that reliability > 95% – RnnCmm demonstration uses sample test or field (observed) data HCLPF uses subjective opinions or response analysis to relate component fragility to RLGM instead of local component response – Awkwardly related to material strength distributions [ASME et al.]

8. Methods: Economics Minimize seismic risk subject to budget constraint – Allocate resources to biggest bang-per-buck =  Seismic Risk/  $$$(component fragility parameter) Seismic risk = E[Discounted future costs due to earthquakes] – =  exp(-dt)*P[Eq of magnitude m at age t]*E[Cost|Eq of magnitude m at age t]dF(m,t) – integrate from now to license expiration – Conditional on components to be improved $$$(component fragility parameter) = cost per unit change in component fragility parameter – Use chain rule (next slide) – Use law of diminishing marginal returns: the more you spend on one component, the less you get

9. Chain Rule  Seismic Risk /  $$$(component fragility parameter) = –  Seismic Risk /  Subsystem Risk*  Subsystem Risk /  $$$(component fragility parameter) –  Subsystem Risk /  $$$(component fragility parameter) =  Subsystem Risk /  Component type risk *  Component Type Risk /  $$$(component fragility parameter) –  Component Type Risk /  $$$(component fragility parameter) =  P[Component Failure|Eq] /  $$$(component fragility parameter) – Where component parameters include fragility correlations!

10. Why is HCLPF = A m Exp[-1.65  ]? Assume strength at failure RV is A m  R  U – Where A m is lower 5-th confidence limit on median component strength at failure and – ln(  R ) and ln(  U ) are normally distributed, independent random variables with means 0 and (logarithmic) standard deviations  R and  U Then P[Stress > Strength] =  [ln(Stress/A m )/  R 2 +  U 2 )] and – Screen is P[RLGM > HCLPF?] = 5%  – HCLPF? =  -1 [0.05, ln(A m ),  R 2 +  U 2 )] or – A m Exp[  1.64485  R 2 +  U 2 )] where standard normal z(0.05) =  1.64485 – i.e., P[Stress/(A m  R  U ) > 1] =  [ln(Stress/A m  > 0] = 0.05

11. Methods: Data and Subjective Opinions to Obtain HCLPF Data: ASME and other material strength distributions, component test data, and post-earthquake failure observations – Maximum likelihood, Bayes, and method of moments – “The seismic capacity of such equipment in regard of their functionality during and after an earthquake is impossible, difficult or unreliable to evaluate by other methods” [Other than seismic test, Tengler] Subjective opinions on median, percentiles, (logarithmic) standard deviations, and Y1|Y2 [NUREG/CR-3558 and others] – Least squares and weighted least squares – Output A m and  from “High-Confidence” fragility function, and 

12. Excel Workbook Implementations HCLPF….xlsx – Computes HCLPFs and P[RLGM > HCLPF] for various components from various input parameters SubjFrag.xlsx – Computes High-Confidence fragility functions from test and seismic observation data, subjective opinions, and multiple-failure-mode fragility functions – Estimates fragility correlations from subjective opinions of P[X1|X2] NoFail.xlsm – Estimates lognormal fragility function parameters, including correlation, from earthquake responses and component no-failure observations

13. HCLPF…xlsx Spreadsheets Spreadsheet NameContents IP2ESELJim Moody’s list of components BoMFLEX PWR list of components InherentESEL Summary table list of components HCLPFCompute HCLPF and P[RLGM > HCLPF] from component lists and fragility parameters HELParametersBandyopadhyay fragility parameters StructuresStructure CDFM fragility parameters EquipmentEquipment CDFM fragility parameters ToleranceIntervalEstimate correct tolerance limit from sample data 95% confidence on 5-th percentile

14. HCLPF Inputs List of component candidates for screening, QPA, structure function (series, parallel, RBD, fault tree) Design basis and RLGM and their units Method(s) (CDFM or SoV?) Component “fragility” parameters A m and  – Strength-at-failure distributions for materials and some components or “High-Confidence” A m (median) and  R (logarithmic) standard deviation assuming lognormal strength – Uncertainties due to components’ strength and responses to seismic ground motions,  U – Safety and other fudge factors for CDFM computation of structure and equipment fragility parameters

15. HCLPF…xlsx:HCLPF Spreadsheet HCLPF spreadsheet does the HCLPF and P[RLGM>HCLPF]? and other computations – From component lists, parameter lists, and parameter computations in other spreadsheets – Table 1 contains notes – Table 2 originated from AREVA proposal form Added QPA, Units, Method, Factors, Am, , and computation columns Computations include: z(p)A m , RLGM>HCLPF?,  (ln(RLGM/A m )/  ),  ({ln(RLGM/A m )+  U  -1 (Q(RLGM)))/  R ), and P[All QPA components survive] Could use VLOOKUP() function or links to enter parameters from other spreadsheets

16. HCLPF Spreadsheet Table 2

17. HCLPF Computations Include… A m Exp[z(p)  : HCLPF = A m Exp[z(p)  ] etc. for alternative z(p) and  (p = “Low-Probability-of-Failure” 5% or 1%) RLGM>HCLPF?: TRUE or FALSE  (ln(RLGM/A m )/  ) = P[RLGM > HCLPF] for alternative   ({ln(RLGM/A m )+  U  -1 (Q(RLGM))}/  R ) [Ravindra, Kennedy, et al.] P[min>RLGM] = P[All QPA components survive] – Assumes QPA co-located, series components with iid fragilities – Change the formula for parallel components or other configuration (FTA) or correlated fragilities

18. SubjFrag.xlsx Spreadsheets Spreadsheet NameContains ToleranceIntervalSame as HCLPF…xlsx:ToleranceInterval 19expertsCompute least-squares lognormal fit to subjective lower 5 th percentiles 19expertsWeightedDitto with weighted least squares 10expertsBootstrapBootstrap 95% “High-Confidence” lognormal fit with fewer than 19 experts SubjCorr Estimate  from subjective distribution of Y1|Y2 by least squares

19. Tolerance Limit on Lognormal RV Fragility Function Estimate correct tolerance limit on a lognormal distribution from a sample of means  (= ln(Am)) and (logarithmic) standard deviations  Input desired “High-Confidence” and “Low-Probability-of- Failure) and a sample of means and standard deviations from tests or subjective opinions – "Exposure Assessment: Tolerance Limits, Confidence Intervals, and Power Calculation…" K. Krisnamoorthy et al. Confidence limit for  + z(p)*  is constructed of the form…  +Q(p,alpha)*  from estimates  and  are the MLEs and Q(p,alpha) is the tolerance factor determined so that … P[  +Q(p,alpha)*  1  alpha… where (something  )/  is approximately ~ (z(p)  )/  Output is tolerance limit or HCLPF (table 2 is simulated)

20. “High-Confidence” Subjective Fragility Function Estimation Suppose 19 experts give 19 opinions on fragility median A m and (logarithmic) standard deviation For each (discrete) value of strength y, find maximum of 19 cdfs and connect with a curve F max (y) – P[Strength < y|Expert A m and  ], (or A m and some percentile) and – Assume each represents the upper 95% confidence limit (“High-Confidence”) – Fit a lognormal distribution to minimum curve to represent a 95% “High-Confidence” fragility function using (weighted) least squares Note that P[F(y) ≤ F max (y) for all y]  0.95

21. Lognormal Fit to Lower 5-th Percentiles

22. By Weighted Least Squares

23. What if there aren’t 19 experts? Bootstrap Correlation estimate requires at least one subjective opinion of distribution of X1|X2 Use least squares to combine experts’ subjective distribution information – Sum of squared errors indicted magnitude of experts’ deviation from lognormal distribution

24. Bootstrap 10 experts

25. NoFail.xlsm Spreadsheets Spreadsheet NameContents FreqMle of lognormal fragility parameters from 19 non-failure responses, assuming 20-th would be failure BayesMoM estimates of lognormal fragility parameters of a-posteriori distribution of P[Failure|Eq] from 19 non-failure responses assuming non-informative prior BayesCorrSame as Bayes, including estimate of fragility correlations Imagine inspections after earthquakes indicate component responses and failure or non-failure

26. Freq Spreadsheet Input iid responses for which no failures occurred – 19 responses were simulated for example and convenient interpretation of mean and standard deviation estimates as “High-Confidence” – Assume ln[Stress]-ln[Strength} ~ N[muX-muY, Sqrt(sigmaX^2+sigmaY^2)] Use Solver to maximize log likelihood of  P[Non- failure|Response] subject to constraint – Either constrain CV or P[Failure] Output is ln(Am) and 

27. Bayes Spreadsheets Bayes estimate of reliability r = P{ln[Stress]- ln{strength] > 0] – Non-informative prior distribution of r – Same inputs as Freq: responses and non-failures – Use MoM to find ln(Am) and  to match posterior E[r] = n/n+1 and Var[r] = n/((n+1)^2*(n+2)) – Ditto to find correlation  from third moment of a- posteriori distribution of r Bayes posterior P[ESEL component life > 72 hours|Eq and plant test data]

28. Parameter Estimates from 19 Non-Failure Responses Given 19 earthquake responses with ln(Median) = 0.5 and  = 0.1 and reliability P[ln(Stress) < ln(strength)] ~ 95% Bayes non-informative prior on reliability P[Response posterior distribution Use Method of Moments to estimate parameters for a-posteriori distribution of reliability ParameterAssume 20-th is failure BayesBayes Correlation  (ln(strength)) 0.7814040.7813890.950047891  ln(strength) 0.1481320.1481230.226336118  ln(strength) N/A 0.752266708

29. What is the correlation of fragilities? See SubjFrag.xlsx:SubjCorr and NoFail.xlsm:BayesCorr spreadsheets to estimate correlations from subjective opinions on Y1|Y2 or from no-failure response observation HCLPF ignores fragility correlation Risk doesn’t ignore it

30. What if multiple, co-located components? Responses are same (Refer to work for Howard last year) In series? Parallel? RBD? Fault tree? Using event trees, Jim Moody argues that HCLPF for one component is representative of all like, co-located components

31. What if like-components are dependent? Fragilities could be dependent too! But not necessarily all fail if one fails – True, P[Response > strength] may be same for all like, co-located components – But what is P[g(Stress, strength) = failure] for system structure function g(.,.)?

32. More References NAP, “Review of Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and the Use of Experts (1997) / Treatment of Uncertainty,” National Academies Press, http://en.wikipedia.org/wiki/Quantification_of_margins_and_uncertainties http://en.wikipedia.org/wiki/Quantification_of_margins_and_uncertainties Viscusi, W. Kip, “Risk Equity,” ISSN 1045-6333, (2000) http://www.law.harvard.edu/programs/olin_center/papers/pdf/294.pdf http://www.law.harvard.edu/programs/olin_center/papers/pdf/294.pdf Der Khiureghian, Armen and Ove Ditlevsen, “Aleatory or Epistemic? Does It Matter?” (2007), Risk Acceptance and Communication Workshop, Stanford Mannes, Albert and Don Moore, “I Know I’m Right, A Behaviourial View of Overconfidence,” Significance, vol. 10, issue 4, August 2013 Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol. 10, issue 4, August 2013 Tengler, Marek, “Seismic Qualification of NPP Structures, Systems and Other Components,” Seminar, Nov. 2011 Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol. 10, issue 4, August 2013