2Agenda Introduction SPC MSPC Passive optimization (E-MSPC) Active optimization (MSPO)ConclusionsThe topic of this lecture is statistical process control and optimization. The first word – control is well known now.The multivariate statistical process control (MSPC) is now a very popular approach that helps to understand and to run the real-world technological processes. It combines the old-known statistical methods like the statistical process control with the modern multivariate data analysis techniques in order to produce a new knowledge about the process in question. This knowledge gives an easy way to monitor and to control the process, but it does not offer a method to optimize the process performance. However, we think that the general aim of any statistical analysis of technology is to improve the quality of the final product or/and to reduce the production costs. In other words there should be a method that could optimize the process.
3Statistical Process Control (SPC) SPC ObjectiveTo monitor the performance of the processSPC ConceptTo study historical data representing good past process behaviourSPC MethodConventional statistical methodsIn this slide I show you the main features of SPC.SPC ApproachTo plot univariate chart in order to monitor key process variables
4Historical Process Data (Chemical Reactor) …Production cycles s1, s2, ... ,s54Key process variables (sensors) X1, X2, ... , X17Here we show our first example that is chemical industrial process. It is described by 17 key process variables, which actually are various sensor indicators. We have a set of historical data about this process represented by the 54 rows in the table. Each row represents a time point where sensor indices were fixed.
5Shewart Charts (1931)Using Shewart approach we can represent the data of sensor 1 by the following chart. Here we can see sensor indices versus time as well as two limits. The first is the boundary for the normal and the second one is the limit for out-of –control process state. That is the first sensor chart and this is the second sensor chart and now we can see both sensors together.Owing to the different natural sensor scales such a plot is very inconvenient, so we performed scaling and plot all sensors together. Now all sensors have common control limit that is equal to 1.
6Panel Process Control (just a game) This, however, is still inconvenient to control and now we present the process model in column diagram view.This is just a computer game, which simulates the process behavior. Playing this game could give us some ideas about the process. New each sensor is represented by the bar and control levels are shown by these red lines. Each bar should be inside and do not exceed the limits. Watching these bars operator can control the process and even manage it.Let we try to do it too. Using these handles we can increase and decrease a variable value. However we cannot operate them independently. If we increase sensor 10 index, we see that other indices are changed too. Some become higher, other become lower. Thus we see they are correlated. No wonder as everybody should expect that temperature and pressure be highly correlated in chemical reactor.
7Multivariate Statistical Process Control (MSPC) MSPC ObjectiveTo monitor the performance of the processMSPC ConceptTo study historical data representing good past process behaviorMSPC MethodProjection methods of Multivariate Data Analysis (PCA, PCR, PLS)This idea moves us to the Multivariate Statistical Process Control.It has the same objective and concept as the conventional SPC. The main distinction is that now we will take into account internal links that present in the system. I am sure that everybody here knows how to do it.MDA in general and Projection technique in particular are the tools we need.MSPC ApproachTo plot multivariate score plots to monitor the process behavior
8Projection Methods Initial Data Data Plane Data Center PCs Data ProjectionsUsually we spend a lot of time in explanation of these pictures for the industry people. Now I may relax a little and give you a chance to enjoy our advanced approach in teaching PCA for engineers.
9Low Dimensional Presentation Now we can represent our data in a low dimensional subspace.
10MSPC Charts (Chemical Reactor) SamplesKey VariablesWe applied projection method (namely PCA) to our example and obtained the following results. In this plot we see the score representation of all samples in two dimensional space. The red ellipse (T-Hotelling) stands for in-state-of-control process limit. The second plot shows all sensors in loading space. Here we can clearly see two groups of similar sensors as well as negative correlation between sensor X7 and the first group.
11Panel Process Control (not just a game) This is our game again but now it is more complicated. Besides the bar chart we show the Score chart, where current process state is represented by the dot within the control ellipse. Watching this plot, operator could control and manage process behavior in easy way.Moreover, now, having the PCA model of the process, we can actively interfere in the management and to suggest a stabilization procedure that moves the process into stabile state.This is a first attempt to optimize the process. More advanced approach will be considered with the next example which is the famous Norwegian cruise ship.
12Cruise Ship Control (by Kim Esbensen) Here is the example, earlier presented by Kim Esbensen and recently by Oxana Rodionova.We will use it in order to explain what process optimization is and for this purpose some background details will be necessary. This is a cabotage vessel that runs along the seashore in Norway. In order to study this process a special agent was introduced into the crew. Actually he was a PhD student who gathers information for his thesis.
13Key Process VariablesThis student could collect the following information that is shown in the table. Each row represents a small trip from one port to another. This trip is characterized by uncontrolled variables that are weather conditions occur during the way and by the controlled variables that are captain's orders about the propeller set up. The last group combines variables that represent the results of that trip – fuel consumption and ship’s speed.
14PLS1 Prediction of Fuel Consumption SamplesPredicted vs. MeasuredUsing these data the student constructed a PLS model that can predict the fuel consumption on the base of the given weather conditions and the known (or snooped) cap’s setup.Now, we came very close to the concept of passive and active optimizations.Weather conditions X1, X2, X3, X4Cap’s setup X5, X6, X7PLS1Fuel Consumption Y
15Passive Optimization 42 42 Captain Computer Student censored Weather conditionsPrediction ?X1, X2, X3, X4X5, X6, X7X5, X6, X7censoredFuckOrder!!!Order!!!Prediction !Prediction !4242To explain them we will show you a 3-act drama, where 3 personages are involved. They are the captain (or technologist, manager, in other words the person in charge), the student (scientist, chemometrician or in other words absolutely irresponsible person) and a computer.The first act. Passive optimization.First scene. Cap thinks about the weather, student thinks about uncontrolled variables and cap gives an order.Second scene. Student thinks about the cap’s setup, computer calculates and student makes a predictionThird scene Cap thinks about the prediction and gives another order. Student thinks about the new order, computer calculates and student makes a new prediction. Cap delivers a speech about the student’s mother, about the proper usage of computers, and about the extraordinary utility of chemometrics for navigation.The curtain falls, entr’acte.CaptainComputerStudent
16Active Optimization 42 Student Computer Captain Weather conditions X5 X6, X7Advice!!!X1, X2, X3, X4CensoredOptimalX5, X6, X7Order?42The second act of the play. Active optimization.Forth scene Student thinks about the weather conditions, computer thinks about uncontrolled variables and calculates the optimal variables for propeller set-up.Fifth scene. Student thinks about the optimized set-up variables and advices the captain. Cap gives an order and computer thinks about the captain’s mother, about the proper usage of students, and about the extraordinary utility of chemometrics for computers.StudentComputerCaptain
17In Hard Thinking about PC and PCs Forty twocensoredThe last Act. The cap is thinking about PC and PCs.The drama is over.
18Multivariate Statistical Process Optimization (MSPO) MSPO ObjectiveTo optimize the performance of the process (product quality)MSPO ConceptTo study historical data representing good past process behaviorMSPO MethodsThat was a play. Now we shall see how it works in practice.MSPO objective is not to control, but to optimize the process performance, i.e. the the final product quality. To reach this target we use historical records as a knowledge base. These data are analyzed with conventional projection technique and (this is a novel point!) with a Simple Interval Calculation method.Projection methods and Simple Interval Calculation (SIC) methodMSPO ApproachTo plot predicted quality at each process stage
19Technological Scheme. Multistage Process To illustrate our ideas, we consider a multi-stage continuous technological process. It is represented by 25 key variables x and by one output variable y, which is the final quality of the product. The whole cycle is divided into seven stages numbered by the Roman numerals.First stage (I) is represented by six input variables (W1, W2, W3 and S1, S2, S3) that stand for the properties of the raw components S and W. At the second stage (II) component W is refining and variables WR1 and WR2 characterize this process. Variables CW1, CW2, and CW3 (Stage III) represent the properties of the outcome product CW. The next stage (IV) is mixing of the raw component S and the refined component CW. The result M is characterized by variables M1, M2, and M3. Afterward, blend M is also refined (Stage V) with the process characteristics MR1 and MR2, and the properties of outcome CM are presented by variables CM1, CM2, and CM3 (Stage VI). The last stage (VII) stands for the ultimate amendments, which are done with additives A1,…, A6. The output variable (P=y) is the final product quality.
20Historical Process Data X preprocessingY preprocessingWe have a collection of historical data measured for 154 samples that characterize proper process performance. Each sample corresponds to the entire production cycle shown in previous slide. The whole data set is divided horizontally (by samples) in two parts: the training set (102 objects) and the test set (52 object). All data are also divided vertically (by variables) into 7 blocks in conformity with the technological stages:For example, block XIV is the (102*3) matrix, which includes 3 variables: M1, M2, and M3, and 102 calibration samples.The data are centered and scaled in such a way that each variable, including quality measure y, varies within range (–1,+1) and that all values outside this interval are not valid.
21Quality Data (Standardized Y Set) It is also assumed that the highest product quality corresponds to y=+1, as well as the lowest one corresponds to y=–1
22General PLS ModelDue to classical MSPC approach we take the whole calibration data set with 25 X-variables and construct the general PLS1 regression model employing 6 PLS components. After that, we perform an additional data scaling, multiplying some columns by factor –1, in order to make all regression coefficients positive. This is done to standardize the process reply, because now increasing of any process variable x leads to the improvement of quality y.
23SIC Prediction. All Test Samples On the base of the PLS-model we also construct the SIC-model of the entire process. Here you can see the predicted quality Y for all test samples together with SIC intervals of uncertainty.For illustration purposes we select five samples from the test set. They present the most typical cases with respect to the product quality as well as to the SIC-status i.e. sample’s relationship towards the process MSPC model.
24SIC Prediction. Selected Test Samples OutsidersAbs. OutsidersInsidersSample NoQuality statusSIC Status1NormalInsider2HighOutsider3Absolute outsider4Low5Let’s look at these selected samples narrowly and talk about SIC object classification. To understand this approach we need keep in mind two items. The first is that SIC method gives a measure of calibration error. It is shown in the left plot by the black error bounds around the referenced test values. We call them test intervals. The second item is the SIC prediction intervals that are shown by blue bars.Inspecting this plot one can see that the blue bars sometimes are inside the black bars. These are samples 1 and 5. It means that prediction error is less then calibration error. Good case.Sample 3 demonstrates the inverse case – blue bar is outside the black one. This is a bad case as prediction is worse, then calibration. At last, samples 2 and 4 represent cases, when prediction is biased against the test.SIC object status plot shows these relations in a compact form. The SIC leverage is actually the width of prediction interval, divided by calibration error, so it shows precision. The SIC residual is a difference between prediction and test, so it a bias characteristics.Samples lying in this triangle are similar to the calibration samples, so they are insiders – in other words they are inside the model. They completely agree with the model, thus insiders are the most trusted objects in predictionSamples that are out of triangle, lie outside the model, therefore they are called outsiders.Outsiders do not contradict the model and, being added to the calibration set, improve the calibration accuracy. However, while they are not in calibration set and they are less-than-perfect in prediction. There are may be two reasons: the width of prediction interval (that is, of course, the SIC-leverage) is greater than the calibration error, or there is a bias (that is characterized by SIC-residual).Usually working with a new sample, we do not know its reference, y value. In this case, it is impossible to calculate the SIC-residual, but we can calculate the SIC-leverage. Such a sample, which leverage is greater than one, could not become an insider for any response value. These samples form a special class of objects, which are called absolute outsiders. Even having no information about their reference values, we can vigorously state that prediction error of such samples will be more than the obtained calibration error.Now we see that selected test samples have the following properties with respect to the quality of final product and regarding to their SIC status.
25Passive Optimization in Practice ObjectiveTo predict future process output being in the middle of the processConceptTo study historical data representing good past process behaviourMethodSimple Interval PredictionWe will show how passive optimization can be implemented in this example using selected samples for illustration.The main objective of passive optimization is to predict future process output being in the middle of the process. We will use historical records as the base and SIC method as the main tool.ApproachExpanding Multivariate Statistical Process Control (E-MSPC)
26Expanding MSPC, Sample 1To make the passive optimization we apply a method of expanding MSPC, which appears to be a new approach in this area.At the end of first stage that is actually the beginning of the process we can predict y using the upper block as the training X set and the lower block as new X set. We obtain PLS point estimates ( red dots) and the SIC-intervals ( blue bars). We also show actual X-variables (dotted line) that were used in the production of these samples. They can be presented in the same plot due to the uniform scaling performed before modeling.At the next stage we may expand the X data block and obtain new prediction model. Using this calibration we can try different values the next forthcoming stage. They are variables WR1 and WR2 that represent set-up of stage II. We can select the best set-up and continue the process with these corrections.When stage II is over we start to adjust the next stage variables, using new expanded model.Thus, we build a series of PLS1 regressions where X matrix is expanding together with a process and Y matrix is fixed.
27Expanding MSPC , Samples 2 & 3 The results of expanded modeling and correspondent SIC-modeling are presented in these slides where 5 plots correspond to five selected samples. In each plot we demonstrate the results of prediction of future quality y that are obtained on each process stage.Green rhombus in the right part of the plots show the quality value, y that was actually obtained in the production.The SIC-prediction intervals are decreasing along the process, but the reference value is always located inside them.Sample 2 (outsider) has a high predicted quality at the beginning of the cycle and later actions improve it.Sample 3 (absolute outsider) has the largest prediction intervals in the course of the process, but no corrections could improve its normal quality status
28Expanding MSPC , Samples 4 & 5 Sample 4 (outsider) initially has a low predicted quality and all correction actions do not recover it, yet make it worse.Sample 5 (insider) keeps its normal status along the cycle without changes.The width of the SIC-interval (i.e. the degree of uncertainty) is smaller for the insiders (Samples 5) and is the largest for the outsiders (Sample 3).
29Active Optimization in Practice ObjectiveTo find corrections for each process stage that improve the future process output (product quality)ConceptCorrections are admissible if they are similar to ones that sometimes happened in the historical data in the similar situationMethodSimple Interval Prediction and Status ClassificationYou have seen how passive optimization was done. In the course we didn’t suggest optimal process corrections but just checked how good were captain’s (operator’s) actions.Now we shall consider how to find the optimal correcting actions that could be performed at the end of each stage, i.e. the active optimization. Actually, this means the proper choice of the controlled variables that become the input variables for the next forthcoming stageApproachMultivariate Statistical Process Optimization (MSPO)
30Linear OptimizationLinear function always reaches extremum at the border. So, the main problem of linear optimization is not to find a solution, but to restrict the area, where this solution should be found.All our models are linear. The main problem in linear optimization is not to find a solution, but to restrict the area where this solution is sought . Optimization of a linear model on unreasonable region of predictors always gives the senseless decision, where optimized characteristic tends to infinity.
31Optimization Problem Model Weather conditions X1, X2, X3, X4PLS1Fuel Consumption YCap’s setup X5, X6, X7Fixed variables XfixPLS1Quality measure YOptimized XoptY = X*a = Y0 + Xopt*a2, where Y0 = Xfix*a1 = ConstModelFor given Xfix and a1 to find Xopt that maxi(mini)mizes YTaskLet’s remember the problem of cruise ship. There we had the model that consisted of two X blocks: fixed weather conditions and free setup values. The data were used to predict the quality measure, which was fuel consumption. In general can we can express this problem in such a way.Our linear prediction model may be represented as the two terms sum. We have the following task. For a given subset of fixed X and for a given parameter values a , to find the second X subset values that optimize the quality Y.The solution is easy as optimum always occurs at the boundary of optimized variable area. In our case it is the upper limits, as we have corrected all coefficient a to be positive.Thus we reduce the optimization problem to another problem, which is to restrict the area of optimized variables.max (Y) = Y0 + max (Xopt)*a2, as all a > 0 (by g factor)Solution
32Interval Prediction of Xopt PLS2XoptLet us consider (just for example) optimization of the forth stage of the process. Here we have 11 fixed variables into 3 past stages. Our aim is to find optimal setup for the next, fourth block.The first, rough restrictions on the predictor area come from the physical or the technological meaning of the process variables. In our case they are +- 1.To find the more sensible range, we suggest applying the main MSPC concept again. We use this block of historical data as X matrix and that block as Y matrix and built PLS2 model that can predicts the forth block variables in the course of production. These red dots presents PLS prediction of variable M1 for all selected test samples.To get the intervals we could draw PMSEP intervals around these points. However, we prefer to use the SIC intervals that take into account individual features of samples.Here you can see the ultimate results where optimal X values (they are upper bounds) are marked with green squares.
33Dubious Result of Optimization Applying this method to all stages of the process we obtain the optimized process variables (blue line) that are above the old test X values (black line). We can also build the prediction of quality based on this optimized process setup.Oops, we are in a mess. Predicted quality is out of sense. No drink could be stronger then 100 degrees!What’s up? The point is that predicted Xopt variables are out of model!Predicted Xopt variables are out of model!
34Adjustment with SIC Object Status ConceptCorrections are admissible if they are similar to ones that sometimes happened in the historical data in the similar situation.Optimal variables Xopt should be within the model !Let come back to the predicted X intervals. To get a reasonable prediction we have to follow this concept. What model do we mean? This is PLS2 model, which predicts the next block of X variables using all previous blocks as X data.If we look at the SIC-intervals for each model together with their object status plot, we see that some samples are out of the insiders area, in other words they are out of model. We have to put them in reducing their intervals in such a way that they are located completely inside the test intervals.Now all samples are insiders and corresponding intervals could give a rational forecast.
35Sample 1 Normal Quality Insider At last you can see the reasonable output. This is optimization of samples 1. It was Normal Quality Insider and it became a little bit better.
36Sample 2 High Quality Outsider Sample 2 (outsider of high quality) did not change its quality, but also did not became worse
37Sample 3 Normal Quality Abs. Outsider Sample 3 (absolute outsider of normal quality) turned into a high quality object
38Sample 4 Low Quality Outsider Sample 4 (outsider of low quality) became a normal quality object
39Sample 5 Normal Quality Insider Sample 5 (insider of normal quality) did not change its quality; it kept its normal statusThe presented results give some reasons for general conclusions. Active optimization shows that status of a sample regarding the general calibration model determines its chances to be optimized. All insiders (test Samples 1 and 5) are the worst objects for improvements. These samples are so average, so normal, that no actions could enhance them. Apparently, this comes from the common technological wisdom that if something is going well let it go at that. This historical experience collected in the actual process records, constricts the area of available actions with x-variables to a very narrow range, which does not permit any enhancements.On the other hand, the outsiders, i.e. the outstanding objects that are far from the common zero level, give an excellent chance to improve or to worsen them. If we undertake some correcting actions with such marginals, they immediately respond with improvements. This happens because the area of possible X-variables is much wider for these unusual, extreme samples.The special case is an absolute outsider (Sample 3), which is an extraordinary object with the abnormal initial properties of the raw components. The historical data contain no knowledge about such samples, so the area of possible corrections is always so wide that it requires some shrinkage to put it in the routine mainstream. Nevertheless, even restricted, such a sample demonstrates excellent possibilities to ‘upbringing’ and it could become a high quality object.Further generalization, association, and parallels to a human inclinations and their development are, unfortunately, out of scope today.
40Philosophy of MSPO. Food Industry Home-made quality MSPOPhilosophy of MSPO. Food IndustryFood QualityProduction EffectivenessHome-made quality Intuitive (expert) controlRestaurant quality Standard (descriptive) controlFast Food quality ISO-9000Leaving this interesting topic, let us however talk a little about the philosophy of MSPO. The food industry, in general, and haute cuisine in particular give us a cause.Who is the best cook in the world? The answers (if I ask you) will be – it’s my granny, or it’s my wife, or even it’s me!Well, no doubt that the best food is the home-made meals produced by an experienced cooker. However, this way of production is very ineffective.The next level is the restaurant-made food. It is worse but more effective.The highest effectiveness has the fast-food. But the quality is ... no words.Now think about the cooking process control. In home kitchen we have the high level control. Granny tastes the meal at each stage. She knows how to correct it and how to improve it. This is MSPO in practice because she uses her knowledge base, her large experience and I am sure some intuitive PLS model in her mind. Yes, I think that humans use projection patterns while thinking. This is however another story for another talk.Could anybody implement such high class control in the restaurant or moreover in fast food service? Of course not, it is impossible. They could apply the descriptive method with cookery books or the method of standardization. And what kind of food we have? These are meals with absolutely standard (from Miami to Barnaul) repulsive taste.What I am talking for? MSPO is the solution. Let us record granny’s action in cooking, collect a base of historical knowledge about her action and build a MSPO model for granny. This will be a computer cook that could produce home made quality meals at the high effectiveness. Now replace granny with captain, or with technologist and you will catch my thought – The MSPO approach is a way to high quality production with minimum costs.If somebody thinks that I am trying to give you a warning about the forthcoming banquette quality, I must to say that this is not the truth. Valentina Ivanovna, our restaurant cook, does belong to the first category so I wish you
41Thanks and ... Bon Appetite! Conclusions The Expanded Multivariate Statistical Process Control (EMSPC) approach has possibilities to support of the on-line improvement of production. It helps to predict which effect on the product quality could be caused by the planned actions, and thus it assists in the passive quality control.The Multivariate Statistical Process Optimization is a technique that proposes the real actions for the quality improvement in the course of production. This is an active optimization of quality, which takes into account the actual history of the process and suggests the available corrections while production process is not over yet.