1 The South African Statistical Quality Assessment Framework (SASQAF) Presentation made at the Conference on Data Quality for International Organisations Helsinki, Finland, 6–7 May 2010 Seble Worku National Statistics System Division Statistics South Africa

2 Background The broader National Statistics System (NSS) is characterized by three gaps  Capacity gap - Insufficient statistical skills - Inappropriate placement of skilled personnel  Quality gap - Preponderant usage of data of uncertain or unknown quality  Information gap - Insufficient supply of statistical information - Insufficient supply of data on second economy To address the Quality Gap the Statistician General has gazetted through parliament the South African Statistical Quality Assessment Framework (SASQAF) on the 23 rd September 2009 SASQAF quality requirements must be met by all statistics producers in the NSS to qualify as “Official Statistics”.

3 The Statistics Act No. 6 of 1999 Stats SA is governed by the Statistics Act (No. 6 of 1999) The Stats Act covers “Official” Statistics and “Other” statistics Implies, Stats Act covers all statistics produced that informs policy, planning and monitoring of government performance. Section 14(7) of the Act, empowers the Statistician- General to “designate as official statistics any statistics or class of statistics” produced by Stats SA or any organ of state  Required that a rational, sustainable and transparent framework for assessing the quality of those statistics being developed  South African Statistical Quality Assessment Framework (SASQAF) has been developed for this purpose

4 Definition Official statistics’ definition is statutory – see Statistics Act [No. 6 of 1999] Practical criteria of official statistics –Must be used in the public domain –Are from organs of state and other agencies that are partners in the National Statistics System [NSS] –Are sustainable –Have met quality criteria as defined by the Statistician- General [SASQAF] National statistics’ definition is implicitly statutory Official statistics are statistics designated as official statistics by the Statistician-General within the provisions of the Statistics Act National statistics are statistics not designated as official Statistics by the Statistician-General

5 Structure of the framework Provides a structure for the assessment of statistical products based on 1.Dimensions of quality, 2.Indicators, 3.Standards, 4.Benchmarks Each of the 8 quality dimensions consists of number of indicators Within the indicators a number of benchmarks are identified relating to a 4-point scale; 1. Quality statistics (4) 2. Acceptable statistics (3) 3. Questionable statistics (2) 4. Poor statistics (1)

6 Relevance Accuracy Coherence Timeliness Interpretability Accessibility Meeting real needs of clients Correctly describes phenomena it is designed to measure Info available at desired reference point Ease of obtaining info from agency Availability of supplementary info and metadata Harmonisation of different info within broad analytical and temporal framework Integrity Free from political interference: Adherence to objectivity, professionalism, transparency, ethical standards Methodological soundness Sound methodologies: International standards and guidelines - good practice Agreed practices Dataset-specific The Dimensions of Quality SASQAF

7 Indicators and Standards 1 st Edition2 nd Edition DimensionIndicators Standards Prerequisites of Quality 7821 Relevance 755 Accuracy 7736 Timeliness 5410 Accessibility Interpretability 333 Comparability and Coherence 5512 Methodological soundness 6614 Integrity 666 Quality

8 Quality dimensionDescription 0.Prerequisites of qualityRefers to the institutional and organisational conditions that have an impact on data quality. Key componentsIndicator Assessment Levels Quality StatisticsAcceptable Statistics Questionable Statistics Poor Statistics Level 4Level 3Level 2Level 1  Legal and institutional environment (including Memoranda of Understanding (MoUs) or Service Level Agreements (SLAs)  Privacy and confidentiality  Resources are commensurate with the needs of statistical programmes  Quality is the cornerstone of statistical work The responsibility for producing statistics is clearly specified. The responsibility for producing statistics is explicitly specified through a legal framework. The responsibility for producing statistics is specified through a legal framework. The responsibility for producing statistics is implied through a legal framework. The responsibility for producing statistics is not specified. Standards and policies are in place to promote consistency of methods and results. All standards and policies are in place to promote consistency of methods and results, and are adhered to. The majority of standards are in place to promote consistency of methods and results. Some standards are in place to promote consistency of methods and results. No standards are in place to promote consistency of methods and results. Data sharing and coordination among data-producing agencies is clearly specified and adhered to. Data sharing and coordination among data- producing agencies is explicitly specified through a legal framework. Data sharing and coordination among data- producing agencies is specified through a legal framework. Data sharing and coordination among data- producing agencies is implied through a legal framework. Data sharing and coordination among data- producing agencies is not specified. Layout of the framework

9 Quality Standards Indicator: The responsibility of producing statistics is clearly specified. Standard: A legal arrangement exists that explicitly mandates the production of statistics. Benchmarks: Quality statistics: A law or legal arrangement exists that explicitly provides the mandate for the production of statistics; Poor statistics: No arrangement exists.

10 Purpose of the framework SASQAF provides an universal framework for assessing the quality of statistics within Stats SA and the NSS Reviews within the NSS: certification of data Self-assessment by data-producing agencies: rate own performance Assessment by data users based on quality declaration: brings trust in data Assessment by international agencies: ease international comparison

11 Purpose of the framework SASQAF provides an universal framework for assessing the quality of statistics within Stats SA and the NSS Reviews within the NSS: certification of data Self-assessment by data-producing agencies: rate own performance Assessment by data users based on quality declaration: brings trust in data Assessment by international agencies: ease international comparison

12 Conclusion Adherence to SASQAF will encourage compliance to the agreed standards, procedures and guidelines  resulting in improvement of quality, closing the quality gap ensure that more statistics are certified as official  closing the supply gap assist the users in assessing the quality of data and products  promoting transparency ensure that all published products include statements about data quality (quality declaration)  informing users about the quality of data and products ensure that more statistical products produced in the NSS are declared as “fit for use”

13 Thank you