Introduction to the “WGFF Guidelines for CMC Uncertainty and Calibration Report Uncertainty” Chun-Min Su CMS/ITRI, Chinese Taipei Nov. 25, 2013 Taipei, Taiwan

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Contents Background Introduction to the Document Unsettled Issues and Future Versions Usage and Dissemination of the Document 2

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Calibration and Measurement Capabilities in the context of the CIPM MRA Paragraph N5 of MRA-D-04 says “… Contributions to the uncertainty stated on the calibration certificate include the measured performance of the device under test during its calibration at the NMI or accredited laboratory. CMC uncertainty statements anticipate this situation by incorporating agreed-upon values for the best existing devices. …” BED 3 CIPM MRA-D-04

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial JCRB-8/9 “Uncertainty contributions of the device under calibration or measurement” Recommendations in DOCUMENT JCRB-8/9 by Ad hoc JCRB Working Group CMC Uncertainties: 1.It is recommended to exclude contributions to the CMC uncertainty caused by the customer device before or after its calibration or measurement at the institute. 2.It is recommended to include contributions to the CMC uncertainty caused by the best ordinarily available customer device during its calibration or measurement at the institute. In general, these values will be published in Appendix C of the CIPM MRA. For individual certificates the actual characteristics of the customer device must be considered. 4 (Feb. 2002)

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial ILAC Policy for Uncertainty in Cal. Clause 5.4 –…. In the formulation of CMC, laboratories shall take notice of the performance of the “best existing device” which is available for a specific category of calibrations. –A reasonable amount of contribution to uncertainty from repeatability shall be included and contributions due to reproducibility should be included in the CMC uncertainty component, when available. … ILAC-P14:01/2013 5

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial ILAC Policy for Uncertainty in Cal. –It is recognized that for some calibrations a “best existing device” does not exist and/or contributions to the uncertainty attributed to the device significantly affect the uncertainty. If such contributions to uncertainty from the device can be separated from other contributions, then the contributions from the device may be excluded from the CMC statement. For such a case, however, the scope of accreditation shall clearly identify that the contributions to the uncertainty from the device are not included. –NOTE: The term “best existing device” is understood as a device to be calibrated that is commercially or otherwise available for customers, even if it has a special performance (stability) or has a long history of calibration. 6 ILAC-P14:01/2013

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial ILAC Policy for Uncertainty in Cal. Clause 6.4 –Contributions to the uncertainty stated on the calibration certificate shall include relevant short-term contributions during calibration and contributions that can reasonably be attributed to the customer’s device. … uncertainty components evaluated for the best existing device shall be replaced with those of the customer’s device. Therefore, reported uncertainties tend to be larger than the uncertainty covered by the CMC. … Clause 6.5 –As the definition of CMC implies, accredited calibration laboratories shall not report a smaller uncertainty of measurement than the uncertainty of the CMC for which the laboratory is accredited. 7 ILAC-P14:01/2013

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial CCM-WGFF Resolutions At its 10 th meeting, WGFF resolved “DUT uncertainty will be agreed upon by the WGFF, based on uncertainties of KC transfer standards, and included in the CMC uncertainty value.” The document “Proposed Definition of DUT Uncertainty” was generated on Dec. 6, 2011 and circulated among members of the WGFF 8

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Initial Proposal (considered/rejected) The same values of DUT uncertainties for different instruments, derived primarily from the KC reports, was proposed Measurand / Device Under Test DUT Uncertainty (k=2)UnitsComments K1 Water flow0.02% Based on CCM.ff-K1 report figures 5 to 8 and figures 19 to 22. K2 Hydrocarbon liquid flow 0.02% Based on CCM.ff-K2 report figures 2 and 4 to 7 showing repeated cals of TS at NEL. K3 Air speed /u(m/s ) % Based on APMP.ff-K3 TS reproducibility, see "Air Speed" worksheet. K5 High-pressure gas flow 0.05% Based on CCM.ff-K5b report table 2. K6 Low-pressure gas flow 0.05% From Wright Flomeko 2007 paper on the uncertainty of the CCM K6 TS. 9

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial New Proposal (accepted) Feedbacks to the Dec. 6, 2011 proposal –The numbers suggested by WGFF are sensible values but the inclusion of fixed figure is not acceptable. –Development of guidance on the expression of uncertainty in CMCs and how to include repeatability, reproducibility and inter-comparison contributions to uncertainty, and uncertainty budget template. –… 10

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Formation of the Document “WGFF Guidelines for CMC Uncertainty and Calibration Report Uncertainty” –6 page document, written over a 2-year period, > 20 versions! –Sub ‐ group: Wright, Smits, Terao, Batista, Paton, Su, Arias, Mickan, van der Beek, and Shimada –Can serve as a model for accreditation of commercial labs 11

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Contents Background Introduction to the Document Unsettled Issues and Future Versions Usage and Dissemination of the Document 12

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial WGFF Guidelines for CMC Uncertainty (1) Define Calibration and Measurement Capability uncertainty (U CMC ) as the root-sum-of-squares (RSS) of: 1.type B base uncertainty of the reference standard 2.type A repeatability of n calibration results measured using the Best Existing Device (BED) 13 U CMC represents the 95 % confidence level uncertainty of the average of n calibration results for the BED using the reference standard. The number of measurements n should match the normal procedures recommended by the lab during calibration of a customer’s device or that may be used when providing services to clients.

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial 14 WGFF Guidelines for CMC Uncertainty (2)

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial 15 WGFF Guidelines for CMC Uncertainty (3)

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Two methods are acceptable for calculating a 95 % confidence level result from the finite number of repeatability measurements –using the Welch ‐ Satterthwaite method to find the effective degrees of freedom and k 95 as explained in Annex G of the GUM, or –using the 95 % confidence level t ‐ value for n ‐ 1 degrees of freedom, divided by 2 and assuming k 95 = 2 16 where s is the sample standard deviation of n repeatability measurements WGFF Guidelines for CMC Uncertainty (4)

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial 17 WGFF Guidelines for CMC Uncertainty (5)

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial The Guidelines provide procedures for assessing repeatability for: –flow calibration –volume calibration The CMC uncertainty can be stated as: 1) the largest value over the operating range of the facility or calibration system or 2) a function describing the uncertainty across the operating range or 3) a table showing how uncertainty varies over the operating range or volume range. 18 WGFF Guidelines for CMC Uncertainty (6)

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial 19 Guidelines for Reported Uncertainty in a Calibration Report (1) Performance Indicator (PI): –error, meter factor, K ‐ factor, discharge coefficient, etc. Reported Uncertainty for the PI (U PI ) –Include additional components over u base due to: instrumentation and characteristics associated with the DUT u AI fluid properties (if applicable) u prop repeatability or short-term reproducibility for the customer’s DUT u repeat or reprod, DUT Note: U PI must be ≥ U CMC

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial 20 Guidelines for Reported Uncertainty in a Calibration Report (2)

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Contents Background Introduction to the Document Unsettled Issues and Future Versions Usage and Dissemination of the Document 21

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Issues and Future Versions Remove t ‐ test approach and recommend Welch ‐ Satterthwaite only? How to handle uncertainty for n = 1. 22

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Contents Background Introduction to the Document Unsettled Issues and Future Versions Usage and Dissemination of the Document 23

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial Usage and Dissemination NMIs should now apply the Guidelines during normal, periodic updates of facilities and their uncertainty analyses. Use for a few years and gather experience. Have the sub-group write a journal paper (perhaps for Metrologia or an accreditation journal) In the short term, post the Guidelines on the BIPM and ILAC web site where they can be publicly accessed – htmlhttp:// html NMIs disseminate information to accrediting bodies 24

Copyright 2012 ITRI 工業技術研究院 建議字型：中文微軟正黑體，英文 Arial 25 Thank you for your attention!