1. 1 Module One: Measurements and Uncertainties No measurement can perfectly determine the value of the quantity being measured. The uncertainty of a measurement needs to be estimated A typical report of a measurement has the form: (Best Estimate) (k*Uncertainty)  The best estimate is usually the average of the observed data.

2. 2  The multiple of the uncertainty, k, is determined to either 1.achieve a certain level of confidence for the reported interval based on some underline probability distribution or 2.achieve the pre-determined specification limits based the physical property of the subject being measured.  The uncertainty is a measure of the spread among observations. It is estimated either 1. using the empirical data (Type A uncertainty) or 2. Using some predetermined allowable uncertainty based on the physical property of the subject, the manufacturer’s specification, reference data from handbooks (Type B uncertainty)

3. 3 e.g.,: On average, how far do people commute from home to office? 45 km 10 km How much confidence for this claim? Why? How? The quantity 45 km is usually the average distance from the sample data. This uncertainty of 10 km is usually refers to some multiple of the sample standard error that measures the precision of the estimate. A typical case: We claim 95% of chance that the average distance commute is between Average 2 * Standard Error of Average

4. 4 e.g.: The chemical compound of a specimen is 5%.02%. The.02% is determined based on the chemical property of the specimen by the manufacturer. If the compound is not in this interval, the specimen is not manufactured. This is a Type B uncertainty. Give an example of Type A uncertainty Give an example of Type B uncertainty

5. 5 Key issues for consideration of a measurement process 1.Validity of the variables to be measured Will the variables reflect the purpose of the study? E.g., A study shows that the IQ of 3 to 5 years old children is highly related to their foot size. The larger the foot size, the higher the IQ. Does Foot Size a valid variable to model IQ for children? To compare a procedure for testing a chemical component in the water between two laboratories, what should be measured? To find out how far people commute from home to office, what variables should be measured?

6. 6 2. Representativeness of the sample to the targeted population To compare a procedure for testing a chemical component in the water between two laboratories, how do we take water samples? How many samples are needed? To study how far people commute in Taipei, how the sample should be obtained? How many samples?

7. 7 3. Potential errors during the process of measurement To compare a procedure for testing a chemical component in water samples between two laboratories, what are the possible errors that may be introduced into the process of measurement? To study how far people commute in Taipei, how the sample should be obtained? How many samples?

8. 8 4. Additional variables (covariates) that may have impact to the purpose of the study or may be confounded with the response variables, e.g., Environmental and background variables. Other than the variables of interest, are there other variables that may have impact to the measurement or may be confounded with the measurement?  List some covariates that may be important for the water testing study.  List some covariates for the commute study.

9. 9 5. Reliability of the data Can the observed data be reasonably reproduced in different labs? This is the issue of reproducibility in inter-laboratory testing. What factors may affect reproducibility for the water testing study?

10. 10 Reliability of Data (continued) Can the observed data be repeated in the same lab? This is the issue of repeatability in inter-laboratory testing. What factors may affect repeatability for the water testing study? If we conduct another survey, are the results from the second survey similar to the first survey? What factors are associated with the reliability of survey data?

11. 11 Sources of uncertainty in an inter-laboratory testing study The sources of uncertainty may be from everywhere during the testing process. They may depend on: 1.The purpose of the study 2.The process of the testing procedure 3.The skills of the operator 4.The facility of the laboratory 5.The tested material(s) (samples) 6.The time delay in the testing process 7.The types of measurements 8.Random error 9.Others:

12. 12 Depending on the purpose of the study, it is crucial to minimize the sources of uncertainty for any lab testing study. Some useful approaches include: 1.Use the similar material or split the same sample for different labs to minimize the variability due to material. 2.Use similar experienced and skillful operators. 3.Use statistical process control tools to minimize the variability and errors introduced in the process. 4.Use appropriate experimental design to obtain adequate data that will maximize the signal and minimize the variability. 5.Others:

13. 13 Important components of uncertainty Bias and Variance Bias refers to the deviation of the observed data from the target (or standard, or reference value). Variability refers to spread of the observed data. Target is the hole. Bias = the average differences from the ball to the target. Variance = the degree of spread among the balls. It is usually measured by the averaging the squared deviations between each ball and the average center of the balls.

14. 14 The first player is consistent, but, makes a systematic error that introduces an large bias. (Precise, but less accurate) The player II is consistent and is accurate as well. (Precise and accurate). Player III is very inconsistent. (Imprecise, some what accurate). In any process, including inter-laboratory testing studies, similar situation may happen. The goal of a process quality control study is to find out the causes that may be associated with the outcomes, and take appropriate actions to improve the process, that is, to minimize the bias and reduce the variance, so that the process is under statistical control.

15. 15 One major task for most inter-laboratory studies is To minimize the system Error and To reduce sources of random error For an inter-laboratory testing study, it is critical that the testing process is under statistical control prior to planning the testing study. So that the results from the testing study are able to reveal the true outcomes due to the factors of interests rather than some uncontrollable causes.

16. 16 Assuming that the participating laboratories and testing processes are under statistical control, one may be interested in measuring some or all of the components of uncertainty, depending on the purpose of the study: Day-to-day variability Between laboratory variability Within laboratory variability Testing differences between methods Testing the homogeneity between material Interaction between testing material and laboratory Interaction between testing method and testing material Interaction between testing method and laboratory Others:

17. 17 A hands-on activity to simulate a testing study 1.Draw a line of 2 cm. based on your best guess. Do not use any ruler or reference. 2.Repeat the process ten times in the order as listed in the next page. 3.Measure the length of each line to two decimal places. Go to next page and begin your activity. Measure each line and record your measurements in table One. Pass your paper to the neighbor on your right side. Now measure the lines and record them in table Two. Tear Table Two off and give it back to the person who draws the lines. Now, record the table Two data to the second row of your table One.

18. 18 1:2: 4: 8: 5: 9: 7: 3: 10: 6:

19. 19 Table One Your own measurements of your drawing 12345678910 Your neighbor’s measurements of your drawing Table Two (Your measurements of your neighbor’s drawing) 12345678910

20. 20 Identify some sources of uncertainties from this drawing activity.. Draw a cause-effect diagram of this drawing activity.

21. 21 Find one or two inter-laboratory testing studies from your working experience that are similar to this activity

22. 22 How to analyze the data such as the one we collected from the above activity? Such an inter-laboratory testing study usually is a comparative study, that is, the purpose is typically to make some type of comparison: Between lab comparison, Testing method comparison, Comparison with standards, Interaction investigation, etc.

23. 23 A Simple Graph Summary of the Measurements 2 cm 1.5 2.5 1 2 3 4 5 67 8 9 10

24. 24 Summarize your observation of the uncertainties Discuss the uncertainty associated with each source: Between individuals (Reproducibility) Repeats within the same individuals (Repeatability) Between different measurement methods (Measurement methods)

25. 25 Modeling, trend analysis, response surface modeling In addition to comparative analysis, there are situations, we are interested in predicting a response using some related variables, determining the maximum/minimum responses, analyzing the trend changes in time. For example, one may be interested in testing the strength of a fiber product under different temperature settings. To conduct this test, the fiber strength will be testing using several temperature settings, say, five temperature setting at –15 o, 0 o, 15 o, 30 o and 45 o. One purpose is to compare the strengths among different temperatures. An even more important goal is to determine the temperature at which the strength is maximized, or minimized. This is a trend analysis. It is also a modeling problem.

26. 26 A hands-on activity for model building and investigating relationship Is your palm size related to your height? How well can palm size predict height? 1.Determine a measurement method to measure palm size. 2.Measure your palm size and height, and record them in the following table. 3.What do your think the relationship would be? (Highly positive, moderately positive, little correlation) Palm sizeHeight (in cm)

27. 27 Scatter plot for demonstrating the relation between Height and Palm Size Ht Palm Size

28. 28 The Process of Measuring Uncertainty Generally speaking, estimation of uncertainty involves with four steps: 1.Specify measurand: Clearly define what is to be measured, and give a clear operational definition of instruments, the physical properties of the measurand, the input and anticipated output. 2.Identify uncertainty sources: Simple tools such as brainstorming, Cause-Effect diagram, Relational matrix mappings can be used to identify possible sources for uncertainty. 3.Quantify uncertainty components: In the process of a study, the major uncertainty components should have been identified. A measurement of each uncertainty component should be measured. Some are determined by the prior knowledge, the physical properties, or the predetermined specification limits. These are the Type B uncertainty. In determining the Type B uncertainty components, the level of confidence is usually based on a conservative probability distributions such as Uniform distribution (assuming equal probability for a possible outcomes), or a somewhat little less conservative distribution such as Triangular distribution. These two distributions are simple and can approximate most of unknown distribution of a characteristic to be measured without underestimating the uncertainty. In some cases, normal distributions may be assumed for determining the confidence intervals. Type A uncertainty is empirical and is usually measured through a process of data collection as well as data analysis to estimate the uncertainty. Most commonly used distribution of the characteristic to me measured is a normal distribution.

29. 29 4. Calculate combined uncertainty: The uncertainty components needs to be combined to determine the overall uncertainty for the measurand. Some mathematical and statistical tolls are needed. Some times, this combination can be rather complicated, especially when uncertainty components are dependent. 5.Review and assess if there is a need to revisit a certain uncertainty components, especially when some unusual uncertainty components are observed. This may happen more likely for Type A empirical uncertainty. An appropriate measurement uncertainty should be repeatable and reproducible under similar conditions. 6.Uncertainty is usually reported in interval depending on the level of confidence. A typical level of confidence is 95% level of confidence.