1. www.uis.unesco.org Application of a DQAF for Education Statistics Conference on Data Quality for International Organisations Newport, Wales April 27 - 28, 2006 Brian Buffett (b.buffett@uis.unesco.org)

2. www.uis.unesco.org lAssumptions lIntroduction lOverview of the DQAF lThe DQAF in practice lObservations & Summary Outline

3. www.uis.unesco.org lFamiliarity with the concept of Data Quality Assessment Frameworks; lAwareness of the IMF DQAF; Assumptions

4. www.uis.unesco.org 1999 ……….UNESCO Institute for Statistics (UIS) was established 2001 ……….UIS moved from Paris to Montreal, Canada 2003 ……….Statistical Capacity Building Programme (SCB) introduced UIS Mission: Internationally comparable statistics in UNESCO’s areas of competence: Education; Science & Technology; Culture; Communication Currently: 100 staff members in Montreal Regional Advisors in: Chile, Senegal, Thailand, Ethiopia, Samoa Introduction

5. www.uis.unesco.org lStatistical Capacity Building is one of the four Main Lines of Action in UNESCO Institute for Statistics statutes. lThe SCB programme serves two purposes: It supports member states to meet their own needs for production and use of statistics in UNESCO domains. It supports primary UIS data programmes. »Since countries are the source of UIS primary data – improving country data is an essential ingredient for improved international data. Why a Statistical Capacity Building Programme?

6. www.uis.unesco.org lThe SCB programme needed a mechanism in order to efficiently and effectively engage countries and assess the statistical systems within education ministries. lWithin education ministries, what was often lacking was not awareness of the need for better data and statistics – but an awareness of what the problems were and a road map for how to go about addressing them. lThe UIS desired a broad framework that focussed on the quality- related features of the governance of statistical systems, their core statistical processes, and their statistical products. Why an Education DQAF?

7. www.uis.unesco.org lThe IMF DQAF is not limited solely to timeliness and accuracy lSix dimensions to the IMF DQAF: Prerequisites of quality Integrity Methodological soundness Accuracy and reliability Serviceability Accessibility The IMF DQAF

8. www.uis.unesco.org lThe DQAF seemed to meet the overall requirements; lThe work was reduced to a fraction – and domain specific; lCost and timelines to implement were attractive; lThe IMF was willing; Why extend the IMF DQAF?

9. www.uis.unesco.org lDeveloped in 2003 in collaboration with the World Bank. lAddresses: International Standards and Classifications (ISCED) Best practices and guidelines specific to education Verifies statistical system measures and reports on: »Structure and normative characteristics of education system »Supply of education »Demand for education »Quality of learning outcomes »School environment »… Extending the DQAF to Education

10. www.uis.unesco.org lUsed to diagnose the situation of national information systems on education, paying particular attention to national information needs. lThese diagnoses are a major element to devise action plans to strengthen national capabilities on education statistics. lInternational reporting requirements are addressed but not the primary objective. lFlexibility in developing action plans: If there are significant problems related to international data reporting, ISCED is a critical element; if the major problems are related to nationally-specific challenges, other items are addressed. The DQAF in Practice

11. www.uis.unesco.org lHow have the diagnostics been carried out? Weighted the DQAF components and developed a scoring guide; Development of common methods and best practices; Scoring is to a significant extent an expert judgement. Diagnostic missions carried out by a small number of trained staff using common methods. lRegional activities can be facilitated by ensuring coherence across countries The DQAF in Practice

12. www.uis.unesco.org lIn Latin America and the Caribbean region, by the end of 2006: Completed in Honduras, Ecuador, and El Salvador; are in revision in Costa Rica, and Nicaragua, and being prepared for Guatemala, Uruguay, Paraguay, Peru and Colombia; Are being prepared for a similar number of Caribbean countries. lWill result in systematic diagnoses under a common framework for half of the region. lThe identification of common challenges will: permit grouping of countries; provide the basis for country-to-country cooperation; lWill have the necessary information to support national as well as regional purposes. The DQAF in Practice

13. www.uis.unesco.org lA useful tool to help strengthen the country’s statistical system by identifying the strengths and weaknesses of the system as well as areas to be improved. lSome results: adoption of new questionnaires better responding to country information needs; improved collection methodology – including training of respondents; more timely completion and return of completed questionnaires; more efficient data capture – with edits to ensure data quality; more efficient processing of the data and production of outputs; improved access to data; training of statisticians and policy makers in use and interpretation of the data. Summary – the Country Perspective

14. www.uis.unesco.org lReduced resource costs, timeframe, and skills required for framework development; Able to focus resources and efforts on subject-matter specificities; lThe framework and diagnostic method are effective; lInitial results have been achieved more rapidly due to this approach; lCountry quality reports on education can be comparable - providing more flexible approaches to capacity building and increasing country-to-country cooperation; lCommon best practices need to be followed, such as: Country ownership; Broad involvement; Assessments combined with audits; Consistent application of scoring guides. lUIS will benefit from future IMF work on the DQAF; Summary – the UIS Perspective

15. www.uis.unesco.org lAn example of collaboration and reuse of existing methods – adapted to International Org. environment/needs; lFactor country needs and situation into any approach; lStatistical activities outside of NSO’s can benefit from the same practices as NSO’s; lA broad definition of quality is important; Summary – the International Perspective

16. www.uis.unesco.org Thank You