Data Management Seminar, 8-11th July 2008, Hamburg 1 ICCS 2009 – On-line Data Collection in the Main Survey

Data Management Seminar, 8-11th July 2008, Hamburg 2 Content Field Trial Experience Integration with Main Survey Operations Guidelines Restrictions Main Survey Time Planning

Data Management Seminar, 8-11th July 2008, Hamburg 3 Field Trial Experience In general, Field Trial analysis showed that on-line data collection can be used as an optional mode for the main survey. During the NRC meeting in Windsor, individual feedback was given including recommendations for the main survey.

Data Management Seminar, 8-11th July 2008, Hamburg 4 Instrument Preparation Sampling and Preparing Survey Administration Data Entry and Verification SURVEYSURVEY Codebook Adaptation Receiving Material Data Sendout Scoring Done by SurveySystem. Convert for ODC after process is complete for paper instruments. Determine and track administration mode. Monitor participation on-line and follow- up. No data to enter, just participation status. Send “cover letters” instead of questionnaires. No data to send. It’s at the IEA DPC already.

Data Management Seminar, 8-11th July 2008, Hamburg 5 Integration – Instruments Instrument Preparation –International definition files prepared by IEA DPC. –National adaptations to structure within countries. –“Copy and Paste” conversion AFTER paper translation, within country and international translation verification, and layout verification.

Data Management Seminar, 8-11th July 2008, Hamburg 6 Integration – Instruments Instrument Verification –Within country content comparison and optical check (final paper versions vs. on-line) –NRCs send localized definitions to IEA DPC –Final optical / logical checks against paper and NAF –Approval or request for corrections from IEA DPC –IEA DPC sets up survey for the Internet Codebook Adaptation –WinDEM codebooks are automatically created!

Data Management Seminar, 8-11th July 2008, Hamburg 7 Integration – Sampling School sampling is not affected in terms of coverage and selection Within-School Sampling –WinW3S allows you to assign the questionnaire type (paper or on-line) per default for principals and/or teachers –WinW3S allows you to track the questionnaire type for individuals that require the other type (e.g., a school that was supposed to fill all questionnaires on-line, but in which some teachers requested paper questionnaires)

Data Management Seminar, 8-11th July 2008, Hamburg 8 Integration – Administration On-line respondents, usually entire schools, receive “cover letters” instead of questionnaires. Same WinW3S labels used. Credentials created with WinW3S are distributed to respondents together with a culture-specific and typo-safe login address, for instance: Administration period applies to both modes.

Data Management Seminar, 8-11th July 2008, Hamburg 9 Integration – Data Entry and Verification Trivial: No need to enter data! Track participation in WinW3S from downloadable ‘Monitor’ reports (“returned, data available” vs. “no data available”) and teacher tracking forms (for other participation indicators, like “left school permanently”). ‘On-line’ data merged with ‘paper’ data from WinDEM at IEA DPC. Verification: Done at the IEA DPC.

Data Management Seminar, 8-11th July 2008, Hamburg 10 Guidelines Overall goal: highest possible response rates, both on questionnaire as well as variable level. Decision and responsibility of usage lies with NRCs, who are generally free to decide about type and scope of implementation, i.e. as the default mode, for principals and/or teachers, or for specific schools or individuals.

Data Management Seminar, 8-11th July 2008, Hamburg 11 Guidelines Individual restrictions in terms of computer/Internet penetration, poor on-line response rates in previous surveys or other obstacles should be taken into account. Eventually, the results of the field trial in terms of response rate and drop-out should become the basis for decision.

Data Management Seminar, 8-11th July 2008, Hamburg 12 Restrictions On-line data collection is administered with the IEA SurveySystem ONLY. The IEA DPC can not manage the verification and data conversion for other systems.

Data Management Seminar, 8-11th July 2008, Hamburg 13 Restrictions On-line mode may become the default mode in your country for the MS, but it is MANDATORY to provide paper questionnaires to individuals/schools without Internet access, equipment, or that simply refuse. For the MS National Centers should therefore check with schools whether the on-line mode is acceptable for them BEFORE send-out of materials. In other words, school coordinators should not need to ask for paper questionnaires if they prefer them. Enquiries should take place during the first contact/notification of schools or together with the send-out of teacher listing forms.

Data Management Seminar, 8-11th July 2008, Hamburg 14 Main Survey Time Planning Documentation of software is part of the ICCS 2009 Survey Operations Procedures Manual Unit 3. International definition files distributes together with SurveySystem software on August 18, “Copy-and-Paste” conversion may start AFTER translation AND layout verification for paper questionnaires is completed.

Data Management Seminar, 8-11th July 2008, Hamburg 15 Main Survey Time Planning Submit zipped definition files to IEA DPC after layout verification approval, and latest two week before the first day of survey administration. Allow at least one week for review and paper/on-line comparison at IEA DPC. On-line administration at the same time as paper administration.

Data Management Seminar, 8-11th July 2008, Hamburg 16 Summary Participation is optional and should be guided by the level of confidence that participants have with the mode. Fall-back paper questionnaires are mandatory (ODC can not be the exclusive mode for the time being). More tasks going on in sequence and parallel in terms of preparation and monitoring. Challenge to assign and track the correct questionnaire mode (on-line or paper) to individuals in schools. Some may agree to participate on-line, but others will require paper versions. Advantages gained from reduced printing and shipping costs, less workload during data entry and preparation. Software and hosting will be provided by the IEA DPC.

Data Management Seminar, 8-11th July 2008, Hamburg 17 Thank you for your attention!