Marching to the Same Drummer Overview of a Proposed Open Source Question Bank Olivier Dupriez World Bank François Fonteneau PARIS21/OECD Geoffrey Greenwell PARIS21/OECD Mark McConaghy DFID - UK International Household Survey Network

About IHSN A network of international agencies A coordinating mechanism to: –Improve quality and use of household survey data in developing countries /MDG monitoring –Harmonise international recommendations for survey design, data analysis, etc –Produce and disseminate international good practices …

The problems…. In developing countries, survey data are produced mostly by official agencies for monitoring / assessing / targeting purposes High dependency on external financial and technical assistance Poor coordination creates problems: –Accessibility and use (no re-purposing of data) –Reliability and relevance –Comparability across sources and over-time

Water & Sanitation example Access to safe water and improved sanitation –2.5 billion lack improved sanitation facilities globally –In sub Saharan Africa 58% (2006) access improved drinking water, MDG target 75%, world target 89%. –Globally 1.5 million children die every year due to poor hygiene practices –Evidence suggests we are off track to meet the MDG goal to ‘halve the proportion of people without access to basic sanitation’ worldwide (54% - 77%). Figure for 2006 is 62% projected 67% 2015.

Nigeria ‘evidence’ case study Monitored by WHO &UNICEF –Model with 3 estimates from national surveys for Access to improved rural sanitation has changed little 21% (1990) to 24% (2006) –poor hygiene practices means Diarrhoea 2 nd largest cause of child death Sluggish progress for improved rural water too 34% (1990) – 31% (2006)

Limited data points, based on 3 DHS surveys – already ‘similar’ limited syntax written to code to the common JMP standards for ‘improved’

New sources (April 2009) Nigeria data archive: 14 datasets documented (DDI) and catalogued; many have data on water/sanitation Problem: inconsistencies in questionnaire design and calculation of indicators in different sources But good documentation and access to all datasets permitted reconciliation (national workshop) and production of standard guidelines for future surveys Decision can be taken on using these new microdata, recomputed to international standards with existing sources

Inclusion of microdata – after reconciliation workshop - shifts estimate Promotes use of data which is made accessible through National Data Archive

… a common obstacle Same situation in most other sectors But in many cases, reconciliation impossible Proposed solution: harmonised questions, guidelines, classifications, methods easily usable by survey managers –A collection of recommended questions, not a repository of questions from existing surveys –Objective is to help collect better data, not to help fix existing data

Q-Bank in a suite of solutions 2005/07: DDI Microdata Management Toolkit  improved usability of data 2008/09: Survey cataloguing tool  improved visibility 2009: Dissemination / anonymization tools & guidelines  improved access 2009: Preservation guidelines (with ICPSR)  improved preservation 2009 – 2010: IHSN Question Bank  improved quality and comparability for future surveys

IHSN Q-Bank A repository of international recommendations A bank of indicators, concepts, classifications, questions and instructions, and related references Based on an XML schema (borrowed much from DDI and DCMI) Application: includes a metadata editor (developed in dot.net) and a user interface developed in Adobe Flex (web) and AIR (stand-alone); all will be open-source Content provided by many experts and specialized agencies (ILO, UNICEF, WB, consultants, etc) Work in progress; expect to cover many topics (priorities are health, education, labour …)

Concepts Latent constructs Indicators Questions Recap - questions and concepts Improved/ unimproved drinking water sources Probable access to clean/safe water What is the main source of drinking water for your household? Disease prevention & better livelihood

Each indicator provides links to the related concepts, questionnaire module(s), and classification(s) and other indicators Description of indicators (definition, methodology, limitation, references, etc)

Description of survey modules Questions - Modular structure - Multiple ‘options’ per topic but ‘best practice’ e.g. Mode, burden, regularity - Overall module instructions, purpose and link to ‘concepts’ - Discussion forum, linking user community

Detailed description and instructions for each question Automatic link to classifications Question ‘Item’ level - Answer categories - Validation checks - Coding instructions - Type - Routing and universe - Visual rendering

Inclusion of technical resources Literal question –What kind of toilet facility do members of your household usually use? Answer categories Flush/pour flush Flush to piped sewer system11 Flush to sceptic tank 12 Ventilated Improved Pit latrine 21 Pit latrine with slab 22 Pit latrine without slab/open pit 23 …. Post question text If ‘flush’ or ‘pour flush’ probe – Where does it flush to? IF NECESSARY ASK PERMISSION TO OBSERVE THE FACILITY Notes for Interviewer Pit latrines are the best conventional sanitation option when there is no source of water on or close to the plot so that water use is restricted.

A repository of classifications with detailed description Maintain complex coding and classifications separately Promote access and consistency across users Improve international comparisons

All content can be added to a “shopping cart” for easy export and re-purposing

Conclusions A tool for survey statisticians, researchers and analysts, aiming to: –Foster adoption of standards and good practices (improve reliability and comparability of data) –Reduce the cost/time of designing surveys Launch expected in September 2009 Open-source Expect some countries to develop national question banks in time

Thank you International Household Survey Network