Copyright 2010, The World Bank Group. All Rights Reserved. QUALITY ASSURANCE AND EVALUATION Part 1: Quality Assurance 1
Copyright 2010, The World Bank Group. All Rights Reserved. QUALITY ASSURANCE Quality Assurance: is the systematic monitoring and evaluation of the various aspects of a product, service or facility 2
Copyright 2010, The World Bank Group. All Rights Reserved. QUALITY ASSURANCE Fit for purpose Right first time 3
Copyright 2010, The World Bank Group. All Rights Reserved. 6 DIMENSIONS OF QUALITY 1.Relevance 2.Accuracy 3.Timeliness 4.Accessibility 5.Interpretability 6.Coherence 4
Copyright 2010, The World Bank Group. All Rights Reserved. QUALITY Can be seen from 2 perspectives: The Producing Statistical Agency The User 5
Copyright 2010, The World Bank Group. All Rights Reserved. PRODUCER PERSPECTIVE Quality Assurance is: The sum of the efforts to produce a quality product 6
Copyright 2010, The World Bank Group. All Rights Reserved. CLIENT PERSPECTIVE Quality Assurance is: confidence in the NSO to produce quality estimates Being able to assess the data provided from the point of view of its fitness for their use 7
Copyright 2010, The World Bank Group. All Rights Reserved. QUALITY ASSURANCE FRAMEWORK Census Quality Assurance includes all activities that are aimed at ensuring quality Not one, but a wide variety of mechanisms & processes Census is an interdependent system Effectiveness dependent on collective effort Professionalism Motivation 8
Copyright 2010, The World Bank Group. All Rights Reserved. ASSURING QUALITY An NSO should make it clear to its employees that everyone has a role to play in assuring quality, from the employees working on daily production tasks to the highest level of management. 9
Copyright 2010, The World Bank Group. All Rights Reserved. USING PROJECT TEAMS Balancing of the dimensions of quality is best achieved through a project team approach 10
Copyright 2010, The World Bank Group. All Rights Reserved. QUALITY IS RELATIVE NOT ABSOLUTE 11
Copyright 2010, The World Bank Group. All Rights Reserved. DEGREE OF QUALITY In Statistics in general and the Census in particular: Achieving “perfect” quality is neither desirable or affordable (in fact it is rarely even possible) Statistics Canada, Quality Guidelines 12
Copyright 2010, The World Bank Group. All Rights Reserved. DEMONSTRATING QUALITY Oversight Professional review Independent assessment 13
Copyright 2010, The World Bank Group. All Rights Reserved. OPENNESS Don’t hide mistakes Report problems before others report on them 14
Copyright 2010, The World Bank Group. All Rights Reserved. QA PROCESS SHOULD DEMONSTRATE Methodological soundness: adherence to professional methods and (internationally) agreed standards; Efficiency: degree to which statistics are compiled in such a way that the cost and the respondent burden are minimized relative to output. 15