1. Experiences in Developing Statistical Quality Frameworks
Michael Colledge, Statistical Consultant
27 June 2018
Session 2

2. Context for Paper and Presentation
Need for and benefits of a statistical quality framework
Systematic mechanism for identification of quality problems,
Basis for creating and maintaining a quality culture
Source of reference material for quality training.
Makes transparent the processes by which quality is assured
Reinforces image of office as a credible provider of quality statistics.
Facilitates exchange of ideas on quality management with other national and international producers of statistics
Statistical offices undertake a wide range of quality initiatives with or without an overall framework
What happens
Do the quality initiative meet their objectives?
What are the lessons learned?
Based on response by 7 statistical offices
I have been personally involved in some of the initiatives and wanted to hear what happened

3. Structure of Presentation
Introducing and maintaining a QAF
Support for QAF implementation
Quality reviews of processes and outputs
Support for other national agencies producing statistics
Quality management and other cross-cutting functions
General messages

4. Introducing and maintaining a Quality Assurance Framework
QAF at Statistics Canada
First released 1999
Objectives
Identify quality management and risk mitigation strategies
within context of StatCan's priorities and values
Reference quality practices in place
Agency-wide, and by
individual program areas
Most recently updated (third version) 2017
Challenge to keep relevant
New data sources, technologies, practices
SCF=QAF
Satcan QAF was the first

5. Introducing and maintaining a QAF (continued)
UN Statistical Quality Assurance Framework (SQAF)
Established by Committee for Chief statisticians of UN System
Comprises template and guidelines
Completed March 2018
For use by UN agencies without QAF, or as basis to revise QAF
UN Commission for Trade and Development SQAF
UNCTAD has decentralised statistical system
Need common understanding of quality across organisation
Statistics Branch previously developed own QAF
Awaited implementation to have support of UN SQAF

6. Support for QAF implementation
Data quality network
Established 2009
Comprised three groups
Core team
Forum for Data Quality Unit to present strategy
Quality circle
Subject matter heads
Relate quality principles back to staff
Quality working group
Quality managers from regional offices
Only quality working group is still active

7. Support for QAF implementation
Implementation of Code of Good Practice (Peru)
Lessons learned
An organisational unit must be assigned overall responsibly
Need to interpret commitments in terms of measurable indicators
Need quality support tools to facilitate implementation
Quality Training (Statistics Canada)
Pilot course on quality management
Converted to self-directed learning

8. Quality reviews of processes and outputs
Quality review program (Statistics Canada)
Started 2008
Independent peer review
Objectives
spread knowledge of quality assurance practices
and risks they can mitigate
identify where resources needed to reduce risks to quality
About 5 reviews per year
Discontinued 2014
Sufficient corporate oversight by other means
Corporate IT solutions
project, financial and human resource management frameworks
Raised staff awareness.

9. Support for other national agencies producing statistics
Statistical Regulations (Statistics Sweden)
Three regulations
Updated quality concept
Updated quality declaration/report to accompany outputs
Annual self evaluation form
Thee sets of guidelines
Handbook on quality for official statistics of Sweden (2016)
Handbook on evaluation of quality of official statistics in Sweden (2017)
What identifies official statistics in Sweden (in preparation)
Data Quality Toolkit (Statistics Canada)

10. Support for other national agencies producing statistics
Statistical Clearing House (Australian Bureau of Statistics)
Started 1997
Objectives
to promote good survey practice
and to minimise respondent burden
by requiring clearance of any survey of more than 50 businesses
by staff of SCH
In 2016/17
About 150 surveys reviewed
75 full review
Reduced respondent burden by 3900 hours
Improved quality
Ceased operations mid 2017

11. General messages for quality initiatives
Evolve or expect to discontinue
Involve staff on whom activities have direct impact
Provide quality tools to support quality initatives
Especially to other ONAs
Coordinate with other cross-cutting frameworks
Quality assurance can be embedded within them

12. Experiences in Developing Statistical Quality Frameworks
Thank you for your attention
Michael Colledge, Statistical Consultant