1. Workshop on Providing the traceability of measurements Application practice of provisions GUM in the performance of comparison of national standards Workshop on Providing the traceability of measurements in the national metrology institutes and accredited calibration and testing laboratories Kyiv, Ukraine – January 2011

2. Workshop on Providing the traceability of measurements Concept of uncertainty When reporting the result of a measurement of a physical quantity, it is obligatory that some quantitative indication of the quality of the result be given so that those who use it can assess its reliability. Without such an indication, measurement results cannot be compared, either among themselves or with reference values given in a specification or standard.

3. Workshop on Providing the traceability of measurements Concept of uncertainty It is therefore necessary that there be a readily implemented, easily understood, and generally accepted procedure for characterizing the quality of a result of a measurement, that is, for evaluating and expressing its uncertainty.

4. Workshop on Providing the traceability of measurements Concept of uncertainty The concept of uncertainty as a quantifiable attribute is relatively new in the history of measurement On the other hand error and error analysis have long been a part of the practice of measurement science or metrology.

5. Workshop on Providing the traceability of measurements Concept of uncertainty It is now widely recognized that, when all of the known or suspected components of error have been evaluated and the appropriate corrections have been applied, there still remains an uncertainty about the correctness of the stated result, that is, a doubt about how well the result of the measurement represents the value of the quantity being measured

6. Workshop on Providing the traceability of measurements Concept of uncertainty Just as the nearly universal use of the International System of Units (SI) has brought coherence to all scientific and technological measurements, a worldwide consensus on the evaluation and expression of uncertainty in measurement would permit the significance of a vast spectrum of measurement results in science, engineering, commerce, industry, and regulation to be readily understood and properly interpreted.

7. Workshop on Providing the traceability of measurements Concept of uncertainty In this era of the global marketplace, it is imperative that the method for evaluating and expressing uncertainty be uniform throughout the world so that measurements performed in different countries can be easily compared.

8. Workshop on Providing the traceability of measurements Method for evaluating and expressing the uncertainty It should be: universal: the method should be applicable to all kinds of measurements and to all types of input data used in measurements.

9. Workshop on Providing the traceability of measurements Method for evaluating and expressing the uncertainty The actual quantity used to express uncertainty should be: internally consistent: it should be directly derivable from the components that contribute to it, as well as independent of how these components are grouped and of the decomposition of the components into subcomponents;

10. Workshop on Providing the traceability of measurements Method for evaluating and expressing the uncertainty The actual quantity used to express uncertainty should be: transferable: it should be possible to use directly the uncertainty evaluated for one result as a component in evaluating the uncertainty of another measurement in which the first result is used.

11. Workshop on Providing the traceability of measurements Method for evaluating and expressing the uncertainty Further, in many industrial and commercial applications, as well as in the areas of health and safety, it is often necessary to provide an interval about the measurement result that may be expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the quantity subject to measurement.

12. Workshop on Providing the traceability of measurements Method for evaluating and expressing the uncertainty The ideal method for evaluating and expressing uncertainty in measurement should be capable of readily providing such an interval, in particular, one with a coverage probability or level of confidence that corresponds in a realistic way with that required…

13. Workshop on Providing the traceability of measurements What is uncertainty of measurement – basic terminology Measurement uncertainty Parameter characterizing the dispersion of the quantity values being attributed to a measurand, based on the information used

14. Workshop on Providing the traceability of measurements What is uncertainty of measurement – basic terminology Measurand Value of a physical quantity Uncertainty To make a result of a measurement complete Result of a measurement Measurand and uncertainty

15. Workshop on Providing the traceability of measurements What is uncertainty of measurement – basic terminology Measurements are not exact! Repetition of a measurement Equipment Measuring method Observer Different results Varying parameters influencing the measurement

16. Workshop on Providing the traceability of measurements What is uncertainty of measurement – basic terminology Correction The value to be added to the measurement result Correction factor The value to be multiplied with the measurement result

17. Workshop on Providing the traceability of measurements What is uncertainty of measurement – basic terminology Deviation Difference between the known value and the value shown by the measuring instrument Nominal value Not a value Name that is being used to describe characteristics of a measuring instrument

18. Workshop on Providing the traceability of measurements What is uncertainty of measurement – basic terminology Error Difference between measured value and true value Absolute or relative Systematic Can be determined by calibration

19. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement uncertainty Measurement uncertainty includes components arising from systematic effects, such as components associated with corrections and the assigned quantity values of measurement standards. Sometimes known systematic effects are not corrected for but are instead treated as uncertainty components

20. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement uncertainty The parameter may be, for example, a standard deviation called standard measurement uncertainty (or a specified multiple of it), or the half-width of an interval, having a stated coverage probability

21. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement uncertainty Measurement uncertainty comprises, in general, many components. Some of these may be evaluated by Type A evaluation of measurement uncertainty from the statistical distribution of the quantity values from series of measurements and can be characterized by experimental standard deviations. The other components, which may be evaluated by Type B evaluation of measurement uncertainty, can also be characterized by standard deviations, evaluated from probability density functions based on experience or other information

22. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement accuracy Closeness of agreement between a measured quantity value and the true quantity value of the measurand The concept ”measurement accuracy” is not a quantity and is not given by a numerical quantity value. A measurement is said to be more accurate when it offers a smaller measurement error

23. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement accuracy The term “measurement accuracy” should not be used for measurement trueness” and the term „measurement precision“ should not be used for 'measurement accuracy', which, however, is related to both these concepts 'Measurement accuracy' is sometimes understood as closeness of agreement between measured quantity values that are being attributed to the measurand.

24. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement trueness Closeness of agreement between the average of an infinite number of replicate measured quantity values and a reference quantity value

25. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement trueness A reference quantity value can be a true quantity value of the measurand or an assigned quantity value of a measurement standard with negligible measurement uncertainty Measurement trueness cannot be expressed numerically, but measures are given in ISO 5725

26. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement trueness Measurement trueness is inversely related to only systematic measurement error The term “measurement trueness” should not be used for ‘measurement accuracy‘ and vice versa

27. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement precision Closeness of agreement between indications obtained by replicate measurements on the same or similar objects under specified conditions

28. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement precision Measurement precision is usually expressed numerically by measures of imprecision, such as standard deviation, variance, or coefficient of variation under the specified conditions of measurement The ‘specified conditions’ can be repeatability conditions of measurement, intermediate precision conditions of measurement, or reproducibility conditions of measurement (see ISO 5725-5:1998)

29. Workshop on Providing the traceability of measurements What is uncertainty of measurement Measurement precision Measurement precision is used to define measurement repeatability, intermediate measurement precision, and measurement reproducibility Sometimes “precision” is erroneously used to mean ‘measurement accuracy

30. Workshop on Providing the traceability of measurements GUM Concept Type A evaluation of measurement uncertainty, Type A evaluation Evaluation of a component of measurement uncertainty by a statistical analysis of quantity values obtained under defined measurement conditions

31. Workshop on Providing the traceability of measurements GUM Concept Type B evaluation of measurement uncertainty, Type B evaluation Evaluation of a component of measurement uncertainty determined by means other than a Type A evaluation of measurement uncertainty

32. Workshop on Providing the traceability of measurements GUM Concept Type B evaluation of measurement uncertainty, Type B evaluation Evaluation based on information associated with authoritative published quantity values; associated with the quantity value of a certified reference material; obtained from a calibration certificate and incorporation of drift; obtained from the accuracy class of a verified measuring instrument; obtained from limits deduced through personal experience

33. Workshop on Providing the traceability of measurements GUM Concept Standard measurement uncertainty, standard uncertainty of measurement, standard uncertainty Measurement uncertainty expressed as a standard deviation Combined standard measurement uncertainty, combined standard uncertainty Standard measurement uncertainty that is obtained from the measurement results of the input quantities in a measurement model Uncertainty budget Statement of a measurement uncertainty, of the components of that measurement uncertainty, and of their calculation and combination The uncertainty budget should include the measurement model, estimates and measurement uncertainties of the quantities in the measurement model, covariances, type of applied probability density functions, degrees of freedom, type of evaluation of measurement uncertainty, and any coverage factor

34. Workshop on Providing the traceability of measurements GUM Concept Expanded measurement uncertainty, expanded uncertainty Product of a combined standard measurement uncertainty and a factor larger than the number one Expanded measurement uncertainty is termed “overall uncertainty” in paragraph 5 of Recommendation INC-1 (1980) and simply “uncertainty” in IEC documents The term ‘factor’ in this definition refers to a coverage factor Coverage interval Interval containing the set of true quantity values of a measurand with a stated probability, based on the information available Coverage probability Probability that the set of true quantity values of a measurand is contained within a specified coverage interval Coverage factor Number larger than one by which a combined standard measurement uncertainty is multiplied to obtain an expanded measurement uncertainty

35. Workshop on Providing the traceability of measurements GUM Concept Calibration measurement capability The CMC is the best measurement capability that is ordinarily available to customers under normal conditions, for example, as published in an NMI.s service list and available, in principle, at any time. It should be: performed according to a documented procedure and an established uncertainty budget under the quality system of the NMI performed on a regular basis; and available to all clients This is also stated in Paragraph T.7 of the CIPM MRA.s Technical supplement:.... The calibration and measurement capabilities referred to in this paragraph are those that are ordinarily available to the customers of an institute through its calibration and measurement services; they are sometimes referred to as best measurement capabilities.

36. Workshop on Providing the traceability of measurements GUM Concept CMC are calibration and measurement capabilities that is ordinarily available to customers under normal conditions a that are: Published in the Database of Key Comparisons of BIPM (BIPM KCDB) in the frame of CIPM MRA Described in the accreditation certificate annex issued by the signatory of ILAC MLA Meaning of both terms (CMC and BMC) is identical and they should be similarly and in a consistent way interpreted in their contemporary fields of use

37. Workshop on Providing the traceability of measurements GUM Concept Measurement and calibration with CMC should be: Performed according to documented procedure with stated uncertainty budged in the frame of the laboratory management system Performed on a regular basis (including contractual calibrations or in a time-schedule during the year) Available to all customers

38. Workshop on Providing the traceability of measurements GUM Concept – calibration The measurands are the particular quantities subject to measurement. In calibration one usually deals with only one measurand or output quantity Y that depends upon a number of input quantities X j (j = 1, 2,…, N) according to the functional relationship Y = f(X 1, X 2, …, X N ) The set of input quantities Xi may be grouped into two categories: Quantities whose estimate and associated uncertainty are directly determined in the current measurement Quantities whose estimate and associated uncertainty are brought into the measurement from external sources An estimate of the measurand Y, the output estimate denoted by y, is obtained from the above equation using input estimates x j for the values of the input quantities X j y = f(x 1,x 2, …,x N )

39. Workshop on Providing the traceability of measurements GUM Concept – calibration For a random variable the variance of its distribution or the positive square root of the variance, called standard deviation, is used as a measure of the dispersion of values The standard uncertainty of measurement associated with the output estimate or measurement result y, denoted by u(y), is the standard deviation of the measurand Y It is to be determined from the estimates x i of the input quantities X j and their associated standard uncertainties u(x j )

40. Workshop on Providing the traceability of measurements GUM Concept – calibration Type A evaluation of standard uncertainty arithmetic mean or the average of the individual observed values q j ( j = 1, 2, …, n) An estimate of the variance of the underlying probability distribution is the experimental variance s²(q) of values q j that is given by

41. Workshop on Providing the traceability of measurements GUM Concept – calibration Type A evaluation of standard uncertainty The best estimate of the variance of the arithmetic mean q is the experimental variance of the mean given by

42. Workshop on Providing the traceability of measurements GUM Concept – calibration Its (positive) square root is termed experimental standard deviation of the mean. The standard uncertainty associated with the input estimate q is the experimental standard deviation of the mean

43. Workshop on Providing the traceability of measurements GUM Concept – calibration Type B evaluation of standard uncertainty The Type B evaluation of standard uncertainty is the evaluation of the uncertainty associated with an estimate x i of an input quantity X j by means other than the statistical analysis of a series of observations

44. Workshop on Providing the traceability of measurements GUM Concept – calibration Type B evaluation of standard uncertainty The standard uncertainty u(x j ) is evaluated by scientific judgement based on all available information on the possible variability of X j. Values belonging to this category may be derived from: Previous measurement data Experience with or general knowledge of the behaviour and properties of relevant materials and instruments Manufacturer’s specifications Data provided in calibration and other certificates Uncertainties assigned to reference data taken from handbooks

45. Workshop on Providing the traceability of measurements GUM Concept – calibration Type B evaluation of standard uncertainty A well-based Type B evaluation of standard uncertainty can be as reliable as a Type A evaluation of standard uncertainty The following cases must be discerned: A single value is known for the quantity X j A probability distribution can be assumed for the quantity X j Upper and lower limits a + and a – can be estimated for the value of the quantity X j

46. Workshop on Providing the traceability of measurements GUM Concept – calibration Calculation of the standard uncertainty of the output estimate (combined uncertainty of uncertainties of the input estimates of different types) For uncorrelated input quantities the square of the standard uncertainty associated with the output estimate y is given by:

47. Workshop on Providing the traceability of measurements GUM Concept – calibration Uncertainty budget Quanti ty (X j ) Estimat e (x j ) Standard uncertain ty u(x j ) Probability distribution Sensitivi ty coeff. c j Contributio n to the standard uncertainty u j (y) X1X1 x1x1 u(x 1 )normalc1c1 u 1 (y) X2X2 x2x2 u(x 2 )triangularc2c2 u 2 (y) ::::: XNXN xNxN u(x N )normalcNcN u N (y) Yyu(y)

48. Workshop on Providing the traceability of measurements GUM Concept – calibration Expanded uncertainty of measurement Within EAL it has been decided that calibration laboratories accredited by members of the EAL shall state an expanded uncertainty of measurement U, obtained by multiplying the standard uncertainty u(y) of the output estimate y by a coverage factor k U = ku(y)

49. Workshop on Providing the traceability of measurements GUM Concept – calibration In cases where a normal (Gaussian) distribution can be attributed to the measurand and the standard uncertainty associated with the output estimate has sufficient reliability, the standard coverage factor k = 2 shall be used. The assigned expanded uncertainty corresponds to a coverage probability of approximately 95%. These conditions are fulfilled in the majority of cases encountered in calibration work

50. Workshop on Providing the traceability of measurements GUM Concept – calibration Statement of uncertainty of measurement in calibration certificates In calibration certificates the complete result of the measurement consisting of the estimate y of the measurand and the associated expanded uncertainty U shall be given in the form (y ± U) The reported expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k = 2, which for a normal distribution corresponds to a coverage probability of approximately 95%.

51. Workshop on Providing the traceability of measurements GUM Concept – calibration In cases where the above mentioned possibility is not relevant: The reported expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k = XX, which for a t- distribution with ν eff = YY effective degrees of freedom corresponds to a coverage probability of approximately 95%

52. Workshop on Providing the traceability of measurements GUM Concept – calibration Statement of uncertainty of measurement in calibration certificates The numerical value of the uncertainty of measurement should be given to at most two significant figures The numerical value of the measurement result should in the final statement normally be rounded to the least significant figure in the value of the expanded uncertainty assigned to the measurement result For the process of rounding, the usual rules for rounding of numbers have to be used. If the rounding brings the numerical value of the uncertainty of measurement down by more than 5%, the rounded up value should be

53. Workshop on Providing the traceability of measurements GUM Concept – calibration Rules for significant figures Nonzero digits are always significant Examples: 45 - 2 sig. figs, 1,37 - 3 sig. figs Captive zeros are always significant Examples: 1001 - 4 sig. figs, 1,0005 - 5 sig. figs Leading zeros are not significant Examples: 0,004 - 1 sig. digit, 0,0045 2 sig. digits Trailing zeros are significant if there is a decimal comma or point Examples 0,00400 - 3 sig. figs, 1000, - 4 sig. figs Zeros at end, no decimal point – they are not significant Example: 1000 - 1 sig. digit Use exponential notation Examples: 1E03 – 1 sig. digit, 1,001E03 – 4 sig. figs

54. Workshop on Providing the traceability of measurements GUM Concept – calibration Rules for significant numbers For addition and subtraction. The answer has the same number of decimal places as there are in the measurement with the fewest decimal places Example 1: 83,5 +23,28=106,78=106,8 Example 2: 865,9 –2,8121=863,0879=863,1 For multiplication and division. The number with the least certainty limits the certainty of the result. Therefore, the answer contains the same number of significant figures as there are in the measurement with the fewest significant figures Example: 9,26,80,3744=23,4225=23

55. Workshop on Providing the traceability of measurements GUM Concept – calibration Rules for rounding of numbers If the digit removed is greater than 5, the preceding number increases by 1 – 5,379 rounds to 5,38 if three significant figures are to be retained and to 5.4 if two significant figures are to be retained If the digit removed is less than 5, the preceding number is unchanged 0,2413 rounds to 0,241 if three significant figures are to be retained and to 0,24 if two significant figures are to be retained If the digit removed is 5, the preceding number increases by 1 if it is odd and remains unchanged if it is even 17,75 rounds to 17,8, but 17,65 rounds to 17,6. If the 5 is followed only by zeros, rule 3 is followed; if the 5 is followed by non-zeros, rule 1 is followed: 17,6500 rounds to 17,6, but 17,6513 rounds to 17,7 Be sure to carry two or more additional significant figures through a multistep calculation and round off only the final answer

56. Workshop on Providing the traceability of measurements GUM Concept – calibration Step by step procedure for calculating the uncertainty of measurement Express in mathematical terms the dependence of the measurand (output quantity) Y on the input quantities X j Identify and apply all significant corrections List all sources of uncertainty in the form of an uncertainty analysis Calculate the standard uncertainty for repeatedly measured quantities For single values, e.g. resultant values of previous measurements, correction values or values from the literature, adopt the standard uncertainty where it is given or can be calculated If no data are available from which the standard uncertainty can be derived, state a value of u(x j ) on the basis of scientific experience For input quantities for which the probability distribution is known or can be assumed, calculate the expectation and the standard uncertainty u(x j )

57. Workshop on Providing the traceability of measurements GUM Concept – calibration Step by step procedure for calculating the uncertainty of measurement Calculate for each input quantity X j the contribution u j (y) to the uncertainty associated with the output estimate resulting from the input estimate x i according to above stated equations and sum their squares to obtain the square of the standard uncertainty u(y) of the measurand. If input quantities are known to be correlated, apply a special procedure Calculate the expanded uncertainty U by multiplying the standard uncertainty u(y) associated with the output estimate by a coverage factor k Report the result of the measurement comprising the estimate y of the measurand, the associated expanded uncertainty U and the coverage factor k in the calibration certificate

58. Workshop on Providing the traceability of measurements GUM Concept – calibration Sources of uncertainty of measurement Incomplete definition of the measurand Imperfect realisation of the definition of the measurand Non-representative sampling — the sample measured may not represent the defined measurand Inadequately known effects of environmental conditions or imperfect measurements of these Personal bias in reading analogue instruments Finite instrument resolution or discrimination threshold Inexact values of measurement standards and reference materials Inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithm Approximations and assumptions incorporated in the measurement method and procedure Variations in repeated observations of the measurand under apparently identical conditions These sources are not necessarily independent

59. Workshop on Providing the traceability of measurements Simple example of comparison Comparison of 500 kilogramg standard EUROMET.M.M-S1 Fifteen participants (including the pilot laboratory) determined the mass of the standard.

60. Workshop on Providing the traceability of measurements Simple example of comparison Values of mass and uncertainties For each participant the results have been determined as the difference between the reported mass value (m) and the nominal mass value (mo).

61. Workshop on Providing the traceability of measurements Simple example of comparison

62. Workshop on Providing the traceability of measurements Simple example of comparison Pilot laboratory’s mass value Estimate of the pilot laboratory’s mass value is the mean of three measurements made during the measurement of the all participants As an alternative, the reference value has been also calculated to be the median of the reported measured mass difference from the nominal value of each participant (including the pilot laboratory).

63. Workshop on Providing the traceability of measurements Simple example of comparison Pilot laboratory’s mass value As an alternative, the reference value has been also calculated to be the median of the reported measured mass difference from the nominal value of each participant (including the pilot laboratory). The reference value m ref can therefore be 0,174g The reference value taken as weighed mean is 0,212g.

64. Workshop on Providing the traceability of measurements Simple example of comparison Mass difference and uncertainty between participants and reference value The mass difference the reference value and each participant is calculated from:

65. Workshop on Providing the traceability of measurements Simple example of comparison The uncertainties have been calculated in accordance with the international guide. The uncertainty of the difference between the reference value and a participant’s measurement is generally made up of the following components:

66. Workshop on Providing the traceability of measurements Simple example of comparison the uncertainty in the participant’s measurement, u c (m i ) the uncertainty due to the drift or instability of the transfer standard, (negliged) the uncertainty in the reference value, u(m ref )

67. Workshop on Providing the traceability of measurements Simple example of comparison The uncertainty is therefore calculated from:

68. Workshop on Providing the traceability of measurements Simple example of comparison

69. Workshop on Providing the traceability of measurements Simple example of comparison

70. Workshop on Providing the traceability of measurements Simple example of comparison Mass differences and uncertainties between participants Mass differences between participants X and Y are calculated by subtracting the difference between the reference value and a participant X from the difference the reference value and a participant Y. These differences are given in Table 5, together with their associated uncertainties which have been calculated as followes. The mass difference and uncertainty are therefore given by:

71. Workshop on Providing the traceability of measurements Simple example of comparison

72. Workshop on Providing the traceability of measurements Simple example of comparison The uncertainty of the difference is made up of the following components: the uncertainty of the difference between a participant X and the reference value the uncertainty of the difference between a participant Y and the reference value

73. Workshop on Providing the traceability of measurements System of prepackages control Thank you for your attention Ivan Kříž ČMI ikriz@cmi.cz www.cmi.cz