1. CCM Working Group on CMCs 1 Report on 11 May 2011 meeting of CCM-WGCMC & on Related Issues Chris Sutton, Chair Measurement Standards Laboratory of New Zealand Prepared for TCM 2011 5 & 6 December 2011, Kobe, Japan v29-11-2011

2. CCM Working Group on CMCs 2 Contents  CCM-WGCMC meeting 11 May 2011  Background to WGCMC  Main Roles of WGCMC  List of services  Oversight of CMC review process  Update from JCRB  Guidelines for authorship of comparison reports  Plan for BIPM Workshop on CMC review  Rationalisation of CIPM MRA documents  The role of supplementary comparisons  CMC submission & Review

3. CCM Working Group on CMCs 3 Background to CCM-WGCMC  Formed in April 2005 at request of JCRB  To facilitate the inter-regional CMC Review Process  Follows Terms of Reference set by JCRB  Except does not conduct inter-regional reviews of CMCs  Members  Chairs of CCM WGs (except WGAv and WGSI-kg)  Representatives from RMOs (~TCM and ~TCFF)  CCM President & Executive Secretary  JCRB Executive Secretary & KCDB Coordinator

4. CCM Working Group on CMCs 4 Main Roles of WGCMC - List of Services  Current List of Services  Go to KCDB, CMCs, Mass and Related Quantities http://kcdb.bipm.fr/AppendixC/search.asp?reset=1&met=M http://kcdb.bipm.fr/AppendixC/search.asp?reset=1&met=M  Agreed previously:  to restrict the list of services to those services normally offered by NMIs,  to avoid changing the current structure of the list,  to add specific instruments/artefacts to the list where it helps to define the best existing device uncertainty, and  for NMIs in developing economies, to allow CMCs for services outside those normally offered by NMIs.

5. CCM Working Group on CMCs 5 List of Services (continued)  Planned changes to List of Services  Expand to include “best existing device uncertainty”  Agreed to use comments column  WGs on Low Pressure, Fluid Flow progressing  Incorporate molar flow rates (gas leak rates)  under Fluid Flow  Leak rate key comparison CCM.P-K12 to be completed 2012

6. CCM Working Group on CMCs 6 Main Roles of WGCMC - Oversight of CMC review process  CMCs efficiently reviewed  by RMO TCs/WGs - grateful for efforts of TC/WG chairs  Relatively low level of CMC activity  40 submissions in 3 years to Feb 2011  Few problems  About 10 CMC-related problems since 2007  Generally resolved promptly  by WGCMC chair and/or JCRB Executive Secretary  Problems mostly due to lack of knowledge about  Preparing CMCs for submission  Required level of supporting evidence  The need to use publicly available results to support CMCs  The process of analysing & linking comparisons

7. CCM Working Group on CMCs 7 Update from JCRB - 1  Guidelines for authorship of reports  In the context of the CIPM MRA  Recently approved by the CIPM (October 2011)  Not yet in CIPM MRA documents  Key points In order to qualify as an author of the project/comparison report every individual shall have made at least a substantial intellectual contribution in at least one of the following activities:  conception, experimental design and evolution of the project/comparison;  scientific performance of the research, having executed at least one or more significant aspects of the project/comparison;  creative analysis, interpretation and calculations of the measurement data;  creative writing up of the manuscript and documenting the project with all its data and results.

8. CCM Working Group on CMCs 8 Update from JCRB - 2  Private companies that are not NMIs or DIs  are not allowed to participate in comparisons conducted under the CIPM MRA  BIPM are planning a March 2012 workshop  on the best practice for the review of CMCs  The JCRB has rationalised many documents  Key are  CIPM MRA-D-04 Calibration and Measurement Capabilities in the context of the CIPM MRA  CIPM MRA-D-05 Measurement Comparisons in the CIPM MRA  CIPM MRA-G-02 Guidelines for the monitoring and reporting of the operation of quality systems See http://www.bipm.org/en/committees/jc/jcrb/.http://www.bipm.org/en/committees/jc/jcrb/

9. CCM Working Group on CMCs 9 Supplementary comparisons - 1  Question raised at WGCMC meeting  Answer  Provided by Mr Altan, JCRB Executive Secretary  See CIPM MRA-D-05 and MRA Glossary “A supplementary comparison is a comparison, usually carried out by an RMO to meet specific needs not covered by key comparisons (e.g. regional needs), for instance measurements of specific artefacts, or measurements of parameters not within the “normal” scope of the Consultative Committees”.

10. CCM Working Group on CMCs 10 Supplementary comparisons - 2  Supplementary comparison key points  Do not need an extensive protocol  Are not linked to CIPM or RMO key comparisons  Because they are explicitly meant to cover techniques/ areas not covered by key comparisons,  Are usually undertaken to support CMC claims of participating institutes  Through allowing demonstration of capabilities in areas that would not usually be subject to key comparisons,  Do not require the computation of degrees of equivalence,  Can be used as support for CMCs  after the Final Reports are approved and published in the KCDB, and  Protocol and Final Report agreed within particular RMO.

11. CCM Working Group on CMCs 11 Supplementary comparisons - 3  Example:  Comparison of conventional (basis 8000 kg/m 3 ) mass  Rather than true mass as in key comparison  NMI-A and NMI-B calibrate the same standard weight  Using their normal calibration and reporting procedures  The results can be used to support the CMCs of NMI-B  If the results of the two NMIs are consistent within the reported uncertainties, and  NMI-A has CMCs for mass which have smaller uncertainties than those reported in the supplementary comparison by NMI-B  An important role of supplementary comparisons  Is supporting the CMCs of developing countries  For example: APMP.M.M-S1 100 mg, 20 g, 1 kg Sri Lanka, Nepal, Bangladesh, India, Pakistan

12. CCM Working Group on CMCs 12 CMC Submission & Review  Procedure:  Follow  CIPM MRA-D-04 and  APMP-QS2 - Guidelines & CMC Flow Diagram  To speed up the process  Provide the quality system & CMC information at the same time  Such as in a.zip file  For modified CMCs  Make the changes to the published Excel file  Available from the link “Get published CMCs” See Section 10 of CIPM MRA-D-04  Advice  is available from APMP TC Chair and/or WGCMC Chair

13. CCM Working Group on CMCs 13 APMP Flow Diagram for CMC submission  NMI Actions: 1, 4, 8, 15  Peer Review before CMC submission  Submit QS and CMC info together

14. CCM Working Group on CMCs 14 Review criteria for CMCs - Section 3 of CIPM MRA-D-04  …. the JCRB requires that the range and uncertainty of the CMCs submitted be consistent with information from some or all of the following sources: 1. Results of key and supplementary comparisons 2. Documented results of past CC, RMO or other comparisons (including bilateral) 3. Knowledge of technical activities by other NMIs, including publications 4. On-site peer-assessment reports 5. Active participation in RMO projects 6. Other available knowledge and experience  While the results of key and supplementary comparisons are the ideal supporting evidence, all other five sources listed above may be considered to underpin CMCs not directly related to the available comparison results and those for which comparison results are not yet available…

15. CCM Working Group on CMCs 15 Any questions?