Measurement Systems Analysis Six Sigma Foundations Continuous Improvement Training Six Sigma Foundations Continuous Improvement Training Six Sigma Simplicity

Key Learning Points  Data Needs to be:  Verifiable  Reliable  Correct  Stakeholders Believe Data  Data Needs to be:  Verifiable  Reliable  Correct  Stakeholders Believe Data

AGENDAAGENDA  What is Measurement Systems Analysis?  Reproducibility and Repeatability  Summary  What is Measurement Systems Analysis?  Reproducibility and Repeatability  Summary

What is measurement systems analysis  Whatever it takes to ensure that measurements are reliable and credible  Anticipate ‘doubting Thomas’ and share the pre-checks on your measurements  Whatever it takes to ensure that measurements are reliable and credible  Anticipate ‘doubting Thomas’ and share the pre-checks on your measurements

Measurement Systems Analysis– Terms  Repeatability Chance that same person (operator) and process will give consistent measurement  Reproducibility Chance that different person (operator) will give consistent measurement  Attribute Qualitative e.g. Pass/Fail, Hot/Cold, Employee name, work order number  Variable Quantitative Numbers, % Pass, Cost, Cycle Time  Repeatability Chance that same person (operator) and process will give consistent measurement  Reproducibility Chance that different person (operator) will give consistent measurement  Attribute Qualitative e.g. Pass/Fail, Hot/Cold, Employee name, work order number  Variable Quantitative Numbers, % Pass, Cost, Cycle Time

MSA - Who should create it  Process owner/team guided by Black or Green Belt  Quality group  May be part of Quality Management System (QMS)  Process owner/team guided by Black or Green Belt  Quality group  May be part of Quality Management System (QMS)

ExerciseExercise 12 months ago you agreed to the installation of a $1Million MRP software package with the aim of improving on time delivery Now a sister plant is looking at introducing a different package to solve the same problem The reason that they give for using a different package is because they say that your attempt to improve delivery failed They quote the data shown on the next graph.

MRP system ‘improvement’

MRP System An Answer  Comparing the data from before and after there is no relationship between the month of the year and the on time delivery

 Before the MRP system there was more spread than afterwards  After the system introduction the average on time delivery fell  But difference is small 1.2/88 (delta/sigma) and we only had monthly figures!  Because of our measurement system we would be unwise to assume that the observed sample average (before and after) is an accurate prediction of future on time delivery  Before the MRP system there was more spread than afterwards  After the system introduction the average on time delivery fell  But difference is small 1.2/88 (delta/sigma) and we only had monthly figures!  Because of our measurement system we would be unwise to assume that the observed sample average (before and after) is an accurate prediction of future on time delivery

Sample Rates  It is unsafe to say whether the project worked or not because the project to introduce the system was poorly set up!!  In future the Green belt should check  The source of the data  The speed and size of sampling  The rate of changes you introduce elsewhere which may have a domino effect  The delta sigma of improvement you want to see

NoiseNoise  Was on-time delivery the right thing to try and change?  If ‘it depends’ on many factors such as: order clauses, customers, capacity, changing volumes of orders, seasonal delivery demands and backlog of orders…this is called noise  Was on-time delivery the right thing to try and change?  If ‘it depends’ on many factors such as: order clauses, customers, capacity, changing volumes of orders, seasonal delivery demands and backlog of orders…this is called noise  In future, check …  That the metric is as free of noise as possible  That there is not a less noisy measure - such as % of orders delivered within 8 weeks, or RTY through a bottleneck (such as test)

LinearityLinearity  A measurement is linear if there is a straight line relationship between the observed or measured value and the actual value

Impact of Linearity  Is 95% on time delivery only 5% better than 90%?  If lead time is 50% do sales go up by 50%  Is it necessary to train 100% of employees or just 50%  Do all the engineers need a computer?  Is 95% on time delivery only 5% better than 90%?  If lead time is 50% do sales go up by 50%  Is it necessary to train 100% of employees or just 50%  Do all the engineers need a computer?  In each case we will be more in control if we fully understand the relationship between  Y and X  and how the magnitude of X or Y effects our decisions

Repeatability and Reliability  T o determine the accuracy of our measurement system and the limits on the usefulness of the measurement device

The necessity of training farmhands for first- class farms in the fatherly handling of farm livestock is foremost in the eyes of farm owners. Since the forefathers of the farm owners trained the farmhands for first-class farms in the fatherly handling of farm livestock, the farm owners feel they should carry on with the family tradition of training farmhands of first class farmers in the fatherly handling of farm livestock because they believe it is the basis of good fundamental farm management.  Task: You have 60 seconds to document the number of times the 6th letter of the alphabet appears in the following text. Repeatability Exercise Number: ____

 You receive a complaint from a salesman that the on time delivery of your plant is low. The last 20 units shipped to a customer were late.  The plant quotes 100% on time. Every departmental manager is 100% on time according to their schedule  How can this be?  You receive a complaint from a salesman that the on time delivery of your plant is low. The last 20 units shipped to a customer were late.  The plant quotes 100% on time. Every departmental manager is 100% on time according to their schedule  How can this be? Repeatability - Discussion

R&R - Why do we need it  To identify how much the chosen measuring system limits our ability to improve  Because measurement systems are often significantly flawed  To guarantee the truth of improvements made  To identify how much the chosen measuring system limits our ability to improve  Because measurement systems are often significantly flawed  To guarantee the truth of improvements made

How to do R&R 1. Select a minimum of 30 parts/documents from the process and at least 2 operators 2. Half should have defects some of which are marginally defective, half defect free 3. Each operator examines each piece at least twice in a random order (ensure understanding of type of data being collected - Variable or Attribute) 4. Plot measurement against operator, run, part and standard 5. If R&R are not acceptable adjust process and repeat 1. Select a minimum of 30 parts/documents from the process and at least 2 operators 2. Half should have defects some of which are marginally defective, half defect free 3. Each operator examines each piece at least twice in a random order (ensure understanding of type of data being collected - Variable or Attribute) 4. Plot measurement against operator, run, part and standard 5. If R&R are not acceptable adjust process and repeat

Attribute Example

Attribute Analysis  Agrees with himself 14/14, 11/14, 14/14  Agrees with standard 26/28, 21/28, 22/28  Overall agree with standard = /(3*28)

Presenting Results  Operator 2 was inconsistent No operator was perfect

Presenting Results  Overall accuracy of measurement 82%  What is it about some parts that we can not measure them correctly?

Using computer software Some of the initial conclusions that you can make are limited If you want more precise information the team should seek advice from a specialist

Attribute R&R - Graphical output

Variable R&R - Graphical Analysis In this example 3 operators measured twice the diameter of 10 turned components. What would the ideal graph look like for each?

R&R Class Exercise  Now its your turn  Grab a bag of m&m’s

Measurement Systems Analysis Six Sigma Foundations Continuous Improvement Training Six Sigma Foundations Continuous Improvement Training