Presentation is loading. Please wait.

Presentation is loading. Please wait.

Establishing the Integrity of Data: Measurement Systems Analysis prepared by Jeffrey T. Luftig, Ph.D. W. Edwards Deming Professor of Management Lockheed.

Similar presentations


Presentation on theme: "Establishing the Integrity of Data: Measurement Systems Analysis prepared by Jeffrey T. Luftig, Ph.D. W. Edwards Deming Professor of Management Lockheed."— Presentation transcript:

1 Establishing the Integrity of Data: Measurement Systems Analysis prepared by Jeffrey T. Luftig, Ph.D. W. Edwards Deming Professor of Management Lockheed Martin Engineering Management Program University of Colorado at Boulder

2 Topics  Measurement Scales and Types of Data  Establishing the Reliability and Validity of Instrumentation, or Precision and Accuracy of Instrumentation

3 Measurement Scales and Data

4

5 Measurement as a Process  As in any process, regardless of the nature of data collected or generated, measurement systems must demonstrate Stability through time, or control Minimal variation as a proportion of specifications, or capability Minimal variation as a proportion of process variation

6 Measurement as a Process MEASUREMENT PROCESS Equipment Standard Procedure Operator Measurement Ambient Environmental Characteristics Product or Process to be Measured

7 Definition of Terms  Reference Value The theoretically or agreed upon correct value of the characteristic being measured, traceable to some standard  Resolution The smallest increment, or unit of measure, available from a measurement process Generally at least 1/10 th of the specification range

8  Precision The degree of agreement (or variability) between individual measurements or test results from measuring the same specimen(s)  Accuracy (Bias) The difference between the average of the measurement error distribution and the reference value of the specimen measured Definition of Terms

9 Precision vs. Accuracy Accuracy Precision

10  Repeatability The variation in repeated measurements of the same items with a single measurement system Within appraiser/system variation  Reproducibility The variation in the average measurements by different appraisers or systems measuring the same items Between appraiser/system variation Definition of Terms

11 Measurement Error Reference Value Accuracy (Bias) Distribution of repeated measures on a single specimen or part Precision - Repeatability - Reproducibility

12 Terms  Linearity The degree to which bias changes with changes in the magnitude of the characteristic measured  Stability The dependability, or consistency of the measurement process over time

13 Measurement Systems Capability  The variability resulting from measurement error must not exceed a significant proportion of the intended specifications said to be capable  In addition, it is not desirable for measurement error to exceed a significant proportion of the total process variability  Capability is not the same as acceptability, acceptability must be determined on a case by case basis

14 Measurement Systems Capability LSLUSL 5.15  E (USL - LSL) Measurement Error Distribution

15 Measurement System Studies  Potential Studies Assess potential of a measurement system to be capable over the long term 10 parts measured 2–3 times by one or more appraisers A “quick and dirty” study to find out if you are in the ballpark Assesses repeatability and reproducibility Often called an R&R study

16 Measurement System Studies  Potential Studies Error Through Time Bias Through Time

17 Measurement System Studies Potential Studies (continued)

18 Results & Conclusions: Evaluating the Precision & Accuracy of the Measurement System  This result of the previous analysis allows us to calculate the average variance of the repeated measures, which when we take its square root gives us the estimate of the standard deviation due to measurement error:  2 =  = 8.15

19 Results & Conclusions: Evaluating the Precision & Accuracy of the Measurement System  Using the estimate of measurement error, we can calculate the Precision-Tolerance ratio, which in the case of short-term studies, should be less than 10%. Assuming the engineering tolerance for this process is 470 (USL) – 450(LSL) = 20:  2 =  = 8.15 P/T = Precision-Tolerance Ratio = = 6(  ) / USL-LSL = 6(8.15) / 470 – 450 = 2.44 = 244% > 10% Requirement (S-T)

20 Results & Conclusions: Evaluating the Precision & Accuracy of the Measurement System  Likewise, we can estimate the Accuracy (amount of Bias) in the scale by calculating the average of the differences between the Means of the Repeated Measures and the True Values for the associated specimens:  = Estimate Bias at 3.09 Grams; as compared to the Precision estimate, this is arguably an inconsequential value.

21 Measurement System Studies  Short-term Studies 25 parts measured 5-8 times by one or more appraisers A more thorough short-term assessment  Long-term Studies 8-10 parts measured 25+ times by one or more appraisers Assesses through time stability

22 Measurement System Studies  Long-term Studies

23 Measurement System Studies  Long-term Studies

24 Measurement Systems Requirements  Summary Regardless of the type of data gathered by an instrument, and the assessment methodology employed, the instrument or device utilized to obtain criterion data must meet three requirements before the experiment should proceed:  The instrument must be precise or reliable;  The instrument must be accurate or valid; and  The instrument should be / must be operating in a state of statistical control.

25 25 Sources and References  The material used in the PowerPoint presentations associated with this course was drawn from a number of sources. Specifically, much of the content included was adopted or adapted from the following previously-published material:  Luftig, J. A Quality Improvement Strategy for Critical Product and Process Characteristics. Luftig & Associates, Inc. Farmington Hills, MI, 1991  Spooner-Jordan, V. Understanding Variation. Luftig & Warren International, Southfield, MI 1996  Luftig, J. and Petrovich, M. Quality with Confidence in Manufacturing. SPSS, Inc. Chicago, IL 1997  Littlejohn, R., Ouellette, S., & Petrovich, M. Black Belt Business Improvement Specialist Training, Luftig & Warren International, 2000  Ouellette, S. Six Sigma Champion Training, ROI Alliance, LLC & Luftig & Warren, International, Southfield, MI 2005  Luftig, J. An Overview of Total Quality Management, Luftig & Warren, International, 1992  Luftig, J. Dr. Deming’s Theory of Profound Knowledge as a Foundation for Strategic Planning and Policy Deployment, Luftig & Warren, International, 1997  Luftig, J. and Jordan, V. Design of Experiments in Quality Engineering, McGraw-Hill/Irwin Publishing Company, 1998


Download ppt "Establishing the Integrity of Data: Measurement Systems Analysis prepared by Jeffrey T. Luftig, Ph.D. W. Edwards Deming Professor of Management Lockheed."

Similar presentations


Ads by Google