MICS Survey Design Workshop Multiple Indicator Cluster Surveys Survey Design Workshop Overview of the MICS Process

Presentation overview Decision process Getting started Survey Plan and Budget

Decision to conduct MICS First step: Why is a survey needed? –In principle, a survey is conducted to satisfy data needs There should be a clearly defined objective of the survey - how data will be used

Data needs assessment To decide, a thorough data needs assessment needs to be carried out Assess whether MICS (or another survey) will be the answer to fill the data gap(s) in question

UNICEF’s role On monitoring-related issues, UNICEF works closely with NSOs and other relevant national organizations UNICEF will be/is working closely with governments to assess data needs, on the basis of the MICS indicator list

Decision If a MICS is decided, the process begins Participation in the Global MICS Programme

Always remember Survey planning, design, implementation, and dissemination is a continuous process Decisions made in one area influence others Some may be irreversible

Getting started Establish steering and technical committees [Template ToRs] Define roles, responsibilities, and accountabilities: the MoU (Memorandum of Understanding) [Template MoU] Identify survey coordinator and key personnel [Template UNICEF MICS Consultant ToR]

Getting started Decide on basic features: –Content [model questionnaires, indicator list] –data collecting tools (paper questionnaires or computer- assisted personal interviewing: CAPI) –reporting domains, sample size [sample calculators] –Fieldwork/data processing personnel needed [personnel calculators] Produce survey plan and budget, including timetable [Template CSP&B available]

Steering Committee Instrumental in full national ownership Senior staff from implementing agency, UNICEF, other partners in development, relevant line ministries and stakeholders Meet at critical points during implementation – funding and advocacy, review and approve MoU and survey plan and budget, review survey progress, etc.

Steering Committee Objectives/Tasks –To ensure transparent decision-making –To identify a collaborative approach to fund-raising –To promote understanding for and utilisation of survey and results –To oversee smooth implementation –Appoint/invite technical experts to a Technical Committee

Technical Committee Works under the Steering Committee Composed of technical staff from organizations represented and other stakeholders Advise Steering Committee on technical decisions and processes

Memorandum of Understanding Between government and UNICEF (and other partners) –Defines roles and responsibilities, accountabilities, –Access and use of data, –Modes and areas of collaboration

Survey Coordinator Technical person to coordinate and contribute to technical processes Typically from the implementing agency Ensures that processes run smoothly, protocols are followed, survey results are understood and used by stakeholders Coordinate work of other technical staff

Other key technical personnel Senior technical personnel/institution(s) Sampling expert Fieldwork coordinator Data processing expert

UNICEF Support Full commitment to work with government counterparts, support implementation, ensure compliance to protocols Work closely with the survey coordinator UNICEF MICS Consultant (under M&E)

The UNICEF/MICS Support and Coordination System UNICEF MICS Consultant / UNICEF Focal Point Regional Experts: In household surveys, sampling, data processing Regional Office: MICS Coordinator/Specialist, M&E Sections UNICEF New York: MICS Team, Statistics and Monitoring Section

Review Process Participation in the Global MICS Programme A continuous review process is in place with communication lines open [Technical Assistance Framework Presentation] No one should (or can) do a survey alone In-country assistance by UNICEF Review of and feedback to all survey documents, tools, data, and results to ensure quality, timeliness, acceptance, and effective dissemination

Reporting Domains Typically national, urban/rural and regional Other domains may be defined or desired – special population groups, subnational estimation Careful assessment of pros and cons needed

Country Survey Plan The Country Survey Plan and Budget is the operationalisation of the MICS It is the key document that describes all aspects of planning and implementation It is an effective document of agreement and is often used for fund-raising It is considered a “live” document that evolves through the planning and implementation

CSP should answer…. Implementing agency, MICS/Survey Team and Governance Structure Questionnaires –Which modules will be retained, which modules will not be used? –Any issues regarding specific questions or question groups?

CSP should answer…. Sampling –What will be the sample size? [calculator] –Sample frame? Updating necessary? –What will be the general design characteristics of the sample? –What will be the domains?

CSP should answer…. Survey instruments needed –Equipment and supply needs [Supply document available] Training of Fieldwork Staff –Timing, participants, length [Calculator, model agenda, a.o. tools available] Fieldwork –Timing, constraints, team composition, duration, logistics, monitoring [Calculator available]

CSP should answer…. Data Entry and Data Processing –Timing, data entry team, duration, monitoring Data Analysis and Reporting –Timing of production of Summary Findings Report and Final Report Archiving and Dissemination –Plans for printing, launch and dissemination of the final Main Report and public sharing of SPSS data –The SPSS data and survey documents will be archived using the DDI Metadata Editor (Nesstar Publisher)

CSP should answer…. Budget –How much will the major budget line items cost? –Available funding and funding needs [Budget template available] Timetable –What is the timetable envisaged for MICS activities? [Timetable template available]

CSP should answer…. Technical Guidance and Support –Detailing information on collaboration with the Global MICS Programme Major Challenges and Support from UNICEF –What are the major challenges? –What kind of technical support will be needed? In what areas? Who from?

Timetable and budget Golden rules: –Basic measures adhered to (e.g. simultaneous data entry) –Good planning is key to timeliness of production of results –Ensure that all activities are fully funded –A good survey plan of a necessary survey, with national ownership and international support, always attracts funding and is always funded

Major timetable components Preparation, planning, governance, personnel Adaptation of survey instruments (presentation) Pre-test of survey instruments (presentation) Sampling and Listing (presentation) Main training (presentation) Fieldwork (presentation) Data processing* (presentation, workshop) Data analysis and report preparation (presentation) Dissemination of results (presentation, workshop) Archiving and further analysis (workshop)

Budget and Timetable …A look at the templates

Computer – Assisted Interviewing (CAPI) MICS started implementing surveys using computer-assisted personal interviewing (CAPI) in 2011 The CAPI approach uses Tablet Personal Computers, Pocket PC personal digital assistants (PDAs) or smartphones as data collection tools

Potential Benefits Positive improvements, but only with good quality control in place and without underestimating how much work it takes to properly implement a survey. Checking of data in real time during interview Considerably reduced levels of missing data Shorter interview durations Data ready almost immediately Survey indicators monitored early in fieldwork

CAPI Specific Survey Planning, Design and Implementation Process Equipment procurement and cost Needed for all interviewers and supervisors planned for fieldwork, plus IT staff Extra tablets/PDAs in case of problems/loss in the field Basic PDAs/Tablet cost approx $450 with no accessories (plus approx $100 in needed accessories)

CAPI Specific Survey Planning, Design and Implementation Process Technical assistance, Preparation and Pretesting Significant extra programming compared with classic data entry Full pretest required to properly test all aspects of the system Programs must be ready and thoroughly tested before training begins Difficulties in making revisions to the programming once data collection has started

CAPI Specific Survey Planning, Design and Implementation Process Training Paper questionnaires still important Train on paper first, then Tablet / PDA Extensive practice during training is needed Data transfer and logistics of transferring data to central office Field staff: computer literate and comfortable with applications Balance between good interview experience and good computer knowledge

Country Survey Plan Country teams planning or considering a MICS will be required to draft and present a ‘Country Survey Plan’ on the last day of the workshop

Country Survey Plan 5-10 minutes presentations Followed by short general discussion There will be preparatory discussions and group work

Country Survey Plan Power point presentation is encouraged Handouts are optional Please copy all power point presentations to the workshop laptop

Country Survey Plan Between now and the time of your presentation, facilitators will work with groups of countries both during workshop hours and after the workshop hours, if needed Bilateral meetings will also be organized to meet with country teams, if needed Please take advantage of facilitators. That is why we are here