The Changepoint Approach to SPC Douglas M. Hawkins, Peihua Qiu University of Minnesota Chang-Wook Kang Hanyang University

Changepoint approach to SPC2 Background to SPC Have stream of process readings X 1, X 2,…X n,…. Need to decide whether all follow common statistical model, versus Isolated (transient) special causes (affect individual readings) or Persistent special causes that remain until detected and fixed.

Changepoint approach to SPC3 The simplest statistical model In control the X n are iid N( 2 ) Isolated special causes change mean and/or variance then revert. Persistent special cause shifts the mean and/or variance. For example, step change in mean to

Changepoint approach to SPC4 Standard SPC methods Shewhart Xbar and R/S chart used for isolated special causes. Persistent causes need memory – cumulative sum (cusum) or exponentially weighted moving average (EWMA) chart. For now we concentrate on latter.

Changepoint approach to SPC5 Designing a chart An upward cusum is defined by where K is reference value or allowance. The chart signals a change if where H is the decision interval.

Changepoint approach to SPC6 The things you need to know Cusum is the optimal way to detect step shift if K is halfway between in-control and out-of-control means. So you must know and You decide H by setting acceptable in- control average run length (ARL). To do this, you also need to know

Changepoint approach to SPC7 Who told you the Greek stuff? Very rarely, you do actually know it. More commonly, –do a Phase I study to estimate and –carefully check data for control (can use fixed- sample-size methods for this) –pick a big enough to matter, small enough not to be easy to see.

Changepoint approach to SPC8 An estimate is not a parameter But sample estimates are not population parameters. So you have a target ARL, but your actual ARL will be a random variable. For sensitive methods like cusum with small K, EWMA with small, resulting uncertainty in your ARL can be large.

Changepoint approach to SPC9 What cusum optimality? On top of this, cusum is optimal only for shift it is tuned for. Get a much different shift, you lose performance. Similarly for EWMA.

Changepoint approach to SPC10 The changepoint-in-mean model For this model –X i ~ N( 2 ) for i <= ~ N( for i > None of the Greeks is known a priori. Suppose we are at observation number n.

Changepoint approach to SPC11 Likelihood approach Write If we knew changepoint was (say) k then MLEs for would be 2 MLE would be S k,n = (V 0,k + V k,n )/(n-2) (after the usual bias adjustment

Changepoint approach to SPC12 …. continued Two-sample t for H 0 : =0 (no change) is Finally, estimate as k maximizing |T k,n | And diagnose step change if T max,n > h n

Changepoint approach to SPC13 Phase II use Changepoint formulation for fixed-sample (Phase I setting) is classical. For Phase II SPC use n is not constant. Modify the procedure to: –If T max,n, < h n, diagnose in control, continue –If T max,n, > h n, conclude out of control. Use the MLEs to diagnose time of change and pre- and post-change means.

Changepoint approach to SPC14 Getting the control limits We need sequence of control limits h n. Fixed-sample theory not much help. A conceptual objective: Pick the h n so that Pr[T max,n > h n | no signal before time n] =. With such a sequence, in-control RL would be geometric (like Shewhart), and with –In-control ARL = 1/

Changepoint approach to SPC15 How to get the h n Big simulation: 16 million data sets. Estimated h n for several values. All on web site

Changepoint approach to SPC16 So why have a Phase I? Dont need in-control parameter estimates, and so dont need Phase I data gathering, Can get up and running in Phase II. As time goes by in control, ever-growing data base gives ever-better estimates (unlike conventional Phase I/II dichotomy)

Changepoint approach to SPC17 ….continued But most folk would dry run at least some readings before turning on testing. For lack of obvious best choice, suggest starting testing at n=10 (but Web tables give cutoffs for starts of n=3 through 21) For example, for =0.005:

Changepoint approach to SPC18

Changepoint approach to SPC19 The cutoff The cutoffs seem to tend to around 3.2 This corresponds roughly to the two-sided point of a N(0,1) This Bonferroni multiplier of 5 is what you pay for the multiple testing.

Changepoint approach to SPC20 Do we need the Shewhart? Changepoint formulation with compares latest X with mean of all previous data; this includes Shewhart I chart as one of its tests. Asymptotic cutoff of 3.2 is close to European standard. and tests the newest mean against grand mean of all previous data; this includes Shewhart Xbar chart for rational groups of any and all sizes.

Changepoint approach to SPC21 How does method perform? Compared to what? Methods that fix IC ARL with unknown parameters scarce. Self-starting cusum doesnt need IC parameter values. Also seamless from Phase I to Phase II. Does however need size of shift for tuning purposes.

Changepoint approach to SPC22 A method comparison Three cusums, k=0.25, 0.5, 1 (tuned for shifts of 0.5, 1, 2 sds) Two in-control ARLs – 100, 500 Shift occurring early (observation 10) or later (observation 100) a: ARL 100, early; b: ARL 100, later c: ARL 500, early; d: ARL 500, later

Changepoint approach to SPC23

Changepoint approach to SPC24 Results Changepoint is sometimes best. Mostly is second best (no surprise, given cusums theoretical optimality). Where not best, it is a close second best and has by far most robustly good performance.

Changepoint approach to SPC25 Example – triglyceride data Data set kindly supplied by Dr. Dan Schultz, Rogasin Institute, New York. Assay triglyceride standard every week. Use as a QC check on unknowns. Triglyceride reading should be constant (doesnt much matter what its value is). Heres one year of data (given as I chart):

Changepoint approach to SPC26 Outlier? Upward shift at end?

Changepoint approach to SPC27 First clear exceedance is at week 40

Changepoint approach to SPC28 What are estimates of the changepoint?

Changepoint approach to SPC29 and of the before- and after-change means

Changepoint approach to SPC30 Focus Dont interpret estimate of changepoint or of separate means in non-significant bit. First signal is 5 weeks after apparent shift. Pre-change mean estimate is 117 mg/dL Post-change mean estimate is124 mg/dL Right from first signal, all three estimates highly stable.

Changepoint approach to SPC31 Conclusions Conventional Shewhart, cusum, EWMA calibrated assuming known parameters. Random errors of estimation in parameters become systematic distortions in run distribution of any particular chart making IC and OOC ARLs random. Ugly tradeoff between Phase I sample size and control over IC RL distribution.

Changepoint approach to SPC32 … The unknown-parameter changepoint formulation lets you fix in-control run length distribution exactly, with or without sizeable Phase I sample. Furthermore, interval alternative means performance competitive regardless of size of the shift.

Changepoint approach to SPC33 References Hawkins, D. M., Qiu, P., and Kang, C.-W. (2003) The Changepoint Model for Statistical Process Control to appear in Journal of Quality Technology. Pollak, M. and Siegmund, D., (1991), 'Sequential Detection of a Change in a Normal Mean When the Initial Value Is Unknown', Annals of Statistics, 19, Siegmund, D, (1985), Sequential analysis : tests and confidence intervals, Springer-Verlag, New York. Siegmund, D. and Venkatraman, E. S., (1995), 'Using the Generalized Likelihood Ratio Statistic for Sequential Detection of a Change- point', Annals of Statistics, 23,