Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright © 2010 SAS Institute Inc. All rights reserved. Root Cause Investigations Graphical Approaches Byron Wingerd, Systems Engineer, JMP/SAS.

Similar presentations


Presentation on theme: "Copyright © 2010 SAS Institute Inc. All rights reserved. Root Cause Investigations Graphical Approaches Byron Wingerd, Systems Engineer, JMP/SAS."— Presentation transcript:

1 Copyright © 2010 SAS Institute Inc. All rights reserved. Root Cause Investigations Graphical Approaches Byron Wingerd, Systems Engineer, JMP/SAS

2 2 Copyright © 2010, SAS Institute Inc. All rights reserved. Agenda Vaccine Process Overview Visualizing Process Changes Decision Trees for Process Investigations

3 3 Copyright © 2010, SAS Institute Inc. All rights reserved. 3 Vaccine Production Process Bulking and Formulation Downstream Purification Fill/Finish Operation Labeling Packaging Inspection Media Preparation Seed Fermentation Upstream Fermentation Inoculum Preparation

4 4 Copyright © 2010, SAS Institute Inc. All rights reserved. Major Inputs to a Biological Process Equipment Process Equipment Support Equipment Facility Utilities Materials Chemicals, Gasses, Filters, Biological Personnel Procedures and other Documents (SOPs etc.) Shifts, Teams and Individuals Measurements At-Line and On-line sensors and assays Off-Line measurements and assays Materials, Personnel, Equipment and Instruments… Process Investigation

5 5 Copyright © 2010, SAS Institute Inc. All rights reserved. The Despair Scenario 5 Process Investigation Investigation Burn out Special Cause Variation Common Cause Variation ? Noisy Response Interacting Systems Changing Inputs Process Investigation

6 6 Copyright © 2010, SAS Institute Inc. All rights reserved. Many Possible Process Inputs Lots of Possible Inputs Good data Bad data Horrible Data Multiple Input Changes Before Each Run Documented in multiple locations Many parts, different owners Many Systems Interact Unintended consequences Goal: Spend the least amount of time and effort excluding branches Process Investigation

7 7 Copyright © 2010, SAS Institute Inc. All rights reserved. When did the Problem Start? Identify the Start of the trend as narrowly as possible Equipment Materials Personnel Measurements

8 8 Copyright © 2010, SAS Institute Inc. All rights reserved. Markers are colored by material type. Each row is an individual lot Are Inputs Changing When the Trend Starts? Conclusion: Materials are Excluded in First Round Equipment Materials Personnel Measurements Date of Run Material 1 Lot 2 Lot 1 Lot 3 Lot 4 Event Marker

9 9 Copyright © 2010, SAS Institute Inc. All rights reserved. When did the Problem Start? Clean cut change events are really convenient Boundaries of the event are easy to investigate specifically Changes in cause correlated with change in process

10 10 Copyright © 2010, SAS Institute Inc. All rights reserved. Possible Trends in Measurement Results What Happens When the Trends are Messy? Change in the Question: Which Xs might be Driving Y

11 11 Copyright © 2010, SAS Institute Inc. All rights reserved. Case Study: Problem: All of my inputs are changing How can I visualize the changes Do changes in inputs affect variation? What is most important to look at first? Approach: Bubble plots for visualization Partition Platform to Analyze Data

12 12 Copyright © 2010, SAS Institute Inc. All rights reserved. Changes in Materials Over Time Each color change in a row represents a change in the lot number of the material

13 13 Copyright © 2010, SAS Institute Inc. All rights reserved. What are the key Drivers of Variation?

14 14 Copyright © 2010, SAS Institute Inc. All rights reserved. Pain Point: Higher frequency of OOS runs Data colored to highlight the extreme high and low values. Color format is the same on the next slide

15 15 Copyright © 2010, SAS Institute Inc. All rights reserved. Recursive Partition (Decision Tree) Systematic method for looking at relatively large data sets Current structure of the partition and its effect on the responses Higher Y values shifted to the right X values are arranged randomly within each category

16 16 Copyright © 2010, SAS Institute Inc. All rights reserved. Decision Trees Also known as Recursive Partitioning, CHAID, CART Models are a series of nested IF() statements, where each condition in the IF() statement can be viewed as a separate branch in a tree. Commonly used for credit scoring, fraud detection, marketing promotion target generation, … Also used to help discover the hot Xs in historical data

17 17 Copyright © 2010, SAS Institute Inc. All rights reserved. Under the hood To find the next branch in the tree For every branch or split For every X »Search through each unique value of X »Split the branch into two groups –e.g. X1 = X1_split –Record »the difference in the response average between the groups, »calculate the logworth = - log10(p-value) Select the split that maximizes the logworth (minimized the p-value) and add a branch based on that split

18 18 Copyright © 2010, SAS Institute Inc. All rights reserved. Under the hood Keep building tree until Minimize size (number of data points) in a branch is met Other criteria can also be imposed

19 19 Copyright © 2010, SAS Institute Inc. All rights reserved. Partition Conclusions Identified Key Materials Investigation Direction Can you investigate everything? Recursive partition points helped to narrow down the potential list of candidates to investigate in depth. If your Xs dont explain your Ys Youre measuring the wrong thing

20 20 Copyright © 2010, SAS Institute Inc. All rights reserved. Follow on Investigation Deeper investigation reveals explanation Started with available data Added hard to get data Implemented changes Variation dropped, and costly unusual events did too.

21 21 Copyright © 2010, SAS Institute Inc. All rights reserved. Application to Future Investigations Run ID Number (sequential runs) Start of Trend List of materials Each material is listed on a separate row. (Intentionally left of graph) The Usual Suspect Lot change in the Usual Suspect is not aligned with the start of the trend Conclusion: Move on to other potential causes

22 22 Copyright © 2010, SAS Institute Inc. All rights reserved. Case Study Fixed Production Schedule Pound the wall until the problem goes away. Train 1 Train 2 Train 3 Running Schedule Train N… Problem FixedTrend Begins

23 23 Copyright © 2010, SAS Institute Inc. All rights reserved. Exclude Categories Quickly Data was readily available Define, Measure, Analyze and Hope Process Investigation

24 24 Copyright © 2010, SAS Institute Inc. All rights reserved. Group Measurements by Suspect Material Control chart phased by suspect material lot numbers Corrective action was implemented quickly Minimized impact of failure mode. Process Investigation

25 25 Copyright © 2010, SAS Institute Inc. All rights reserved. Conclusions While its a lot of work to keep manufacturing databases up to date. The pay off in an emergency is worth it. Depth of data collection can be shallow as long as: Resources are available to troll through non electronic records Intermediate data is sufficient for an indirect diagnosis Recursive Partition, decision trees can quickly yield actionable results in root cause investigations. Best when the relationship between Xs and Ys are unknown Good where there are many Xs to wade through Sparse data could cause problems, other tools like Random Forrest (Bootstrap Forrest in JMP) may be necessary.

26 Copyright © 2010 SAS Institute Inc. All rights reserved.

27 Methods Byron Wingerd, Systems Engineer, JMP/SAS

28 28 Copyright © 2010, SAS Institute Inc. All rights reserved. Plots for Showing Sequential Changes in Categorical Variables Raw material lots change frequently in many of the processes we investigate Need a standard graphic: Quickly compare lot turnover between materials Show change events relative to raw material lot changes Drill down from all materials to specific materials. In JMP, the Bubble Plot graph type can be easily formatted to generate simple and informative plots

29 29 Copyright © 2010, SAS Institute Inc. All rights reserved. Data Structure Single Material (Flat Table) One row for each process run Individual columns for each raw material »The lot number for each material is recorded for each run »Graphics are intolerant of missing data, empty cells will be blank. Multiple Materials (Stacked Table) One Column for run ID One Column for Material type One Column for Lot Numbers These graphics are intolerant of missing data, empty cells will be blank.

30 30 Copyright © 2010, SAS Institute Inc. All rights reserved. Plot 1: For One Material Bubble Plot Dialog: Graph/Bubble Plot

31 31 Copyright © 2010, SAS Institute Inc. All rights reserved. Plot 2: For Multiple Materials Need to Stack Materials. (Tables/Stack) The lot numbers of materials are in separate column so the first step is to stack the Lot Number Columns Name for the column that will contain the lot numbers Name for the column that will contain the material names

32 32 Copyright © 2010, SAS Institute Inc. All rights reserved. Plot 2: For Multiple Materials Optional Step: Clean up Material or Lot names Clean up names using the Recode tool in the columns menu Concatenate the material type and lot number. This column is used for the graph label. Note: The Concatenate character is a pair of double tubes, Shift-Backslash (the button over the enter key). Concatenate only works on character columns or numbers that are forced to be characters using Char(:colname).

33 33 Copyright © 2010, SAS Institute Inc. All rights reserved. Plot 2: For Multiple Materials Bubble Plot Dialog: Graph/Bubble Plot

34 34 Copyright © 2010, SAS Institute Inc. All rights reserved. Plot 2: For Multiple Materials Format Axis Double Click Y axis to edit the scale or to add reference lines

35 35 Copyright © 2010, SAS Institute Inc. All rights reserved. Plot 3: More Color for Multiple Materials In this plot each material is on one row and the color of the row changes with each lot change

36 36 Copyright © 2010, SAS Institute Inc. All rights reserved. Setting up the Plot Data Structure Run or Date Column Material ID column Lot ID column Making the Graph Graph/Bubble Plot X, Run Number (or date) Y, Material Coloring, Lot Number/ID Details Use the Red Triangle Menu to change the shape to a Square

37 37 Copyright © 2010, SAS Institute Inc. All rights reserved. Scripting The Bubble Plot The JMP Scripting Language (JSL) can be used to generate graphs automatically. JMP writes the script for you. Red Triangle Menu, select Script, then save the script to the script window. Add a couple of edits, like an Open statement and a send to (<<) and add your columns names to your captured script. dt=Open(c:\filepath\filename.jmp); dt< { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/7/1651321/slides/slide_37.jpg", "name": "37 Copyright © 2010, SAS Institute Inc. All rights reserved.", "description": "Scripting The Bubble Plot The JMP Scripting Language (JSL) can be used to generate graphs automatically. JMP writes the script for you. Red Triangle Menu, select Script, then save the script to the script window. Add a couple of edits, like an Open statement and a send to (<<) and add your columns names to your captured script. dt=Open(c:\filepath\filename.jmp); dt<

38 38 Copyright © 2010, SAS Institute Inc. All rights reserved. Running a Recursive Partition From the Analyze menu, select Modeling, Partition Add response and factors to the dialog and click OK

39 39 Copyright © 2010, SAS Institute Inc. All rights reserved. Running a Recursive Partition Click the Split and Prune button to find the best splits The Red Triangle menu contains an option to view the column contributions For automatic splitting, choose the k-fold cross validation option, or exclude rows to use in a for a validation subset.

40 Copyright © 2010 SAS Institute Inc. All rights reserved.

41 41 Copyright © 2010, SAS Institute Inc. All rights reserved. Process of Statistical Discovery Reporting Analysis/Graphics Data Management Data Access Big Time Savings, But Not Flashy The Ahas Occur Here Interactive Flash Output Data Information Knowledge Understanding So decision makers can take Action!

42 42 Copyright © 2010, SAS Institute Inc. All rights reserved. Process of Statistical Discovery Getting Data into JMP is Easy JMP, Excel, Text, SAS & other data formats SAS Data Server Database Internet/html Reporting Analysis/Graphics Data Management Data Access

43 43 Copyright © 2010, SAS Institute Inc. All rights reserved. Process of Statistical Discovery Shaping the Data for Analysis - Big Time Savings Tables Menu Spend time here today Cols Menu Column Info… Column Properties Formula… Rows Menu Reporting Analysis/Graphics Data Management Data Access

44 44 Copyright © 2010, SAS Institute Inc. All rights reserved. Process of Statistical Discovery Many Analyses & Graphs – Range of Stat. Expertise Exploratory Data Analysis, Statistics, Modeling Design of Experiments Interactive Data Mining Visual Six Sigma, Quality, Reliability Business Visualization Profiler, Simulator, Data Filter Reporting Analysis/Graphics Data Management Data Access

45 45 Copyright © 2010, SAS Institute Inc. All rights reserved. Process of Statistical Discovery Wide Range of Outputs Available Graphs & Tables in: Data Tables, Reports, Journals, Projects Paste Special into MS Word, PPT, Excel Flash Objects Profiler Distribution Bubble Plots Print to PDF Reporting Analysis/Graphics Data Management Data Access

46 46 Copyright © 2010, SAS Institute Inc. All rights reserved. All Data is Contextual… Only people understand context, relevance and utility. Making new discoveries is not algorithmic, and never can be. JMP allows informed users to explore data in flexible ways to make new useful discoveries. This happens in the same head, with no division of labor to confuse things. JMP, in continual development for more than twenty years, is designed and architected to support this process of Statistical Discovery.


Download ppt "Copyright © 2010 SAS Institute Inc. All rights reserved. Root Cause Investigations Graphical Approaches Byron Wingerd, Systems Engineer, JMP/SAS."

Similar presentations


Ads by Google