Conference on Data Quality for International Organizations Newport, Wales, United Kingdom, 27–28 April 2006 Session 3: Collection, management and dissemination of current best methods, minimum standards, and best practices “The role of best practices in building a global statistical system” Presenter: Stefan Schweinfest, UNSD

Outline of presentation 1.The “What”: Standards/Best Methods/Best Practices 2.The “How”: Collection/Management/ Dissemination 3.The “Why”: Transparency/ Harmonization/Capacity Building 4.Three UNSD examples of dissemination of ‘best practices’ 5. Points for discussion.

The “What”: Standards/Best Methods/ Best Practices Surprising lack of terminological clarity: ‘standard’, ‘code’, ‘system’, ‘recommendation’, ‘principle’, ‘best methods’, ‘best practices’, ‘good practices’, ‘current practices’…. Surprising lack of terminological clarity: ‘standard’, ‘code’, ‘system’, ‘recommendation’, ‘principle’, ‘best methods’, ‘best practices’, ‘good practices’, ‘current practices’…. abstract – concrete (‘standards’ – ‘best practices’) abstract – concrete (‘standards’ – ‘best practices’) ‘Standard’ = norm / model ‘Standard’ = norm / model  Statistical Standards: Approved by legislative body eg. Statistical Commission – question of ‘enforcement’;  Minimum vs. maximum standards ‘Best Practices’ = applied procedures; value judgement ‘Best Practices’ = applied procedures; value judgement

The “How”: Collection/Management/ Dissemination Collection Collection  Scope: metadata - complex process descriptions  Timing: time of data collection – separate  Mode: questionnaire – assessment mission Management Management  difficult to organize and store (textual information)  difficult to maintain up-to-date Dissemination Dissemination  whether or not? (sensitive country information?)  Mode: paper (static) or Internet (dynamic)

The “Why”: Transparency/ Harmonization/Capacity Building Questions: What is the primary purpose of disseminating “Best Practices”? Transparency:Extension of metadata Transparency:Extension of metadata Capacity building: sharing of knowledge Capacity building: sharing of knowledge Harmonization: Convergence of practices, towards pre-established norm (‘soft enforcement’). Harmonization: Convergence of practices, towards pre-established norm (‘soft enforcement’).

Three UNSD examples of Dissemination of Best Practices Trade Index Numbers Trade Index Numbers “National Practices in Compilation and Dissemination of External Trade Index Numbers – A Technical Report” UN, New York 2005, ST/ESA/STAT/SER.F/86 Poverty Statistics Poverty Statistics “Handbook on Poverty Statistics: Concepts, Methods and Policy Use”, UN, New York, forthcoming, ST/ESA/STAT/SER.F/99, Statistical Organization Statistical Organization “Official Statistics - Good Practices Website”

Trade Index Numbers (1) Prompted by a requested by Statistical Commission Prompted by a requested by Statistical Commission Collection mode: questionnaire with follow-up ( ) Collection mode: questionnaire with follow-up ( ) Response: 76 countries and one customs union; Response: 76 countries and one customs union; Relatively simple and straightforward questions on 8 categories; Relatively simple and straightforward questions on 8 categories; Comprehensive response from respondents; Comprehensive response from respondents; One-time paper publication (2005): standard description approx. 1 page per country One-time paper publication (2005): standard description approx. 1 page per country

Trade Index Numbers (2) CategoryResponse Rate 1 Index number series produced100 2 Source of information100 3 Index calculation methods100 4 Limitations of indices, problems and other methodological observations 75 4 Release dates 84 5 Revision policy 75 6 Dissemination 99 7 Compiling agency and contact information100

Poverty Statistics (1) Part of work on Handbook poverty statistics (inventory of methods) Part of work on Handbook poverty statistics (inventory of methods) Collection mode: paper/electronic questionnaire with follow-up during regional workshops (2005) Collection mode: paper/electronic questionnaire with follow-up during regional workshops (2005) Responses used for analysis: 65 countries; Responses used for analysis: 65 countries; Relatively complex questionnaire with approx. 50 detailed questions; also contained request for actual data; Relatively complex questionnaire with approx. 50 detailed questions; also contained request for actual data; Uneven responses from respondents; Uneven responses from respondents; One-time paper publication (2006): tabular summary presentation of main findings (16 pages) One-time paper publication (2006): tabular summary presentation of main findings (16 pages)

Poverty Statistics (2) Category Response Rate 1. National poverty rate (data) Poverty rate absolute or relative97 3. Income vs. expenditure97 4. Level of calories threshold68 5. Use of equivalent scales98 6. Poverty specific CPI used to costing the basket of goods65 7. Regional patterns in food habits taken into account68 8. Number of food items in the basket69 9. Length of recall period(s) Diaries method used to collect consumption data?35 [….]

Statistical Organization (1) Part of follow-up activities to Fundamental Principles of Official Statistics (Information sharing) Part of follow-up activities to Fundamental Principles of Official Statistics (Information sharing) Collection mode: Initial call for provision of information (2000); Collection mode: Initial call for provision of information (2000); Selected information on 86 countries; Selected information on 86 countries; No questionnaire; posting of original material; assignation of keywords based on 10 principles and related categories; No questionnaire; posting of original material; assignation of keywords based on 10 principles and related categories; Availability of information related to categories very uneven; Availability of information related to categories very uneven; Internet based data base (ad hoc updates by UNSD). Internet based data base (ad hoc updates by UNSD).

Statistical Organization (2) Principle Category“Response Rate” Advisory bodies49 User consultation81 Organizational planning and operation78 Presentation of statistics95 Dissemination and marketing55 Release policies95 Dealing with the media92 [….]

Summary observations Three ‘collections’ of best practices prompted by different circumstances (data/no data; stage of methodological development); Three ‘collections’ of best practices prompted by different circumstances (data/no data; stage of methodological development); Availability of information for about 30-40% of member countries; Availability of information for about 30-40% of member countries; Direct communication crucial to improve response rate and response quality; Direct communication crucial to improve response rate and response quality; Labour intensive exercises; challenge of sustainability Labour intensive exercises; challenge of sustainability Limited knowledge of use of information. Limited knowledge of use of information.

Points for Discussion What other examples of collection of ‘best practices’ are there? What other examples of collection of ‘best practices’ are there? What are the experiences with the cost-benefit analysis for the exercise? What are the experiences with the cost-benefit analysis for the exercise? Under which circumstances are collection efforts of ‘best practices’ most useful? Under which circumstances are collection efforts of ‘best practices’ most useful? Do you agree that the collection of ‘best practices’ is an effective tool to promote ‘convergence’ among countries? Do you agree that the collection of ‘best practices’ is an effective tool to promote ‘convergence’ among countries?