Data Management Seminar, 9-12th July 2007, Hamburg Data Entry Overview

Data Management Seminar, 9-12th July 2007, Hamburg 2 Table of Content 1. Organizing the Data Entry Process 2. Data Manager’s Responsibilities 3. Necessary Material for Data Entry 4. Control Receipts of Survey Material 5. Organize Material for Data Entry 6. Organize Material for Marking 7. The Process of (Reliability) Marking 8. Outlook: Data Entry with WinDEM

Data Management Seminar, 9-12th July 2007, Hamburg Data Documentation Data Organizing the Data Entry Process Instruments/ Forms International Codebooks WinW3S File Data Entry Rules... DE Unit Data Manager 3

Data Management Seminar, 9-12th July 2007, Hamburg Data Manager’s Responsibilities Data Entry Software Source of Information Hardware Software Adaptation of Codebooks Verification Documentation Instruments Tracking Forms Marking Material Organize:Training in: Organize: Perform: Send Out Materials Data Entry Staff Equipment Data Files Data Manager 4

Data Management Seminar, 9-12th July 2007, Hamburg From ICCS International Study Center International codebooks (20. August 2007) - Questionnaire codebooks - Achievement and reliability codebooks Survey Operations Procedures (Unit 3 FT) Windows Within-School Sampling Software (WinW3S) Windows Data Entry Manager (WinDEM) program IEA Survey System Necessary Material for Data Entry 5

Data Management Seminar, 9-12th July 2007, Hamburg Important documents for data entry –Survey Operations Procedures (Unit 3 FT) Copy relevant parts of documents for key punchers Copies of survey tracking forms should be available for key punchers Prepare Key Punchers for Data Entry 6

Data Management Seminar, 9-12th July 2007, Hamburg Planning for Required Resources Staff for marking and data entry depends on… –How much material? –How much time? –How much experience? –How much money? –How many persons available? Computer requirements –IBM-compatible (Pentium processor or higher) –OS: Windows 2000/NT 4.0/XP –Office 97 or higher Ask countries that participated in previous IEA studies for their experiences! 7

Data Management Seminar, 9-12th July 2007, Hamburg Amount of Materials Marking: Marker 6Achievement Booklets = ~ 700Student Achievement Test Booklets ~ 350Student Achievement Test Booklets for Reliability Marking ~ 25School Questionnaires ~ 250Teacher Questionnaires ~700Student Questionnaires ~ 700Regional Module Questionnaires ~ 700Student Achievement Test Booklets ~ 350Reliability Marking Sheets Data Entry: Key Punchers 8

Data Management Seminar, 9-12th July 2007, Hamburg Receiving Material Instrument Preparation Test Administration Codebook Adaptation Receiving Material Data Submission Sampling and Preparing Test Administration Marking Data Entry and Verification 9

Data Management Seminar, 9-12th July 2007, Hamburg Receiving Material 10

Data Management Seminar, 9-12th July 2007, Hamburg Control Receipts of Test Material Record return date on ICCS FT School Sampling Form Contact schools that have not returned testing material or where the material is incomplete! 11

Data Management Seminar, 9-12th July 2007, Hamburg Control Receipts of Test Material Check that all survey tracking forms have been returned and completed correctly... participation status Check participation status on tracking forms! Contact schools that have not returned tracking forms or returned incomplete! 12

Data Management Seminar, 9-12th July 2007, Hamburg Control Receipts of Test Material Check that all booklets/ questionnaires have been returned identification codes Verify identification codes on instruments! Contact schools that have not returned all testing instruments! 13 European Module Student Questionnaire Teacher Questionnaire School Questionnaire Student Test Booklet

Data Management Seminar, 9-12th July 2007, Hamburg Organize Material for Data Entry Receiving Material 14

Data Management Seminar, 9-12th July 2007, Hamburg Separate questionnaires and tracking forms from booklets! Sort and forward… Organize Material for Data Entry and Marking Forms + Questionnaires to Data Entry Staff Booklets to Markers... 15

Data Management Seminar, 9-12th July 2007, Hamburg Forms + Questionnaires to Data Entry Staff... Sorting Forms and Questionnaires 16

Data Management Seminar, 9-12th July 2007, Hamburg Hierarchical ID System IDSCHOOL: 4 digits  school IDCLASS: 6 digits  school + class IDSTUD: 8 digits  school + class + student IDTEACH: 6 digits  school + teacher Unique identifier in ICG file Unique identifier in xSG/xSE/xSL, xSA, xSR file Unique identifier in ITG file

Data Management Seminar, 9-12th July 2007, Hamburg Arrange Instruments for Data Entry School Questionnaires Sort by IDSCHOOL... 18

Data Management Seminar, 9-12th July 2007, Hamburg Teacher Questionnaires Sort by IDTEACH... Arrange Instruments for Data Entry 19

Data Management Seminar, 9-12th July 2007, Hamburg... IDSCHOOL 8001 IDCLASS IDSCHOOL 8001 IDCLASS IDSCHOOL 8002 IDCLASS IDSCHOOL 8002 IDCLASS Sort by IDSTUD Student Questionnaires Arrange Instruments for Data Entry 20

Data Management Seminar, 9-12th July 2007, Hamburg... IDSCHOOL 8001 IDCLASS IDSCHOOL 8001 IDCLASS IDSCHOOL 8002 IDCLASS IDSCHOOL 8002 IDCLASS Sort by IDSTUD Regional Module Student Questionnaires Arrange Instruments for Data Entry 21

Data Management Seminar, 9-12th July 2007, Hamburg Organize Material for Data Entry Organize Material for Marking Receiving Material 22

Data Management Seminar, 9-12th July 2007, Hamburg Sorting Booklets for Marking Two Teams for Marking: A and B Booklets to Markers Marking Team AMarking Team B 23

Data Management Seminar, 9-12th July 2007, Hamburg... 1,2,...,5,6 Steps for Booklet Arrangement Step 1: Sort by Booklet Type 24

Data Management Seminar, 9-12th July 2007, Hamburg Arrange Test Booklets for Marking Step 1: Sort by Booklet Type Booklet 1 R Booklet 1 R Booklet 1 R Booklet 1 R Booklet 2 Booklet 2 Booklet 2 Booklet 2 R ~120 ~ 700 ~120 Booklet 1 Booklet 1 Booklet 1 Booklet 1 R Booklet 2 Booklet 2 Booklet 2 Booklet 2 R... 25

Data Management Seminar, 9-12th July 2007, Hamburg Steps for Booklet Arrangement Step 2: Divide into Two Sets (only Booklets 1-3)... Set A (IDSCHOOL is an odd number), Set B (IDSCHOOL is an even number)... Step 1: Sort by Booklet Type 26 Note: From now on the example for Reliability Marking will focus on booklet 1 only! 1,2,...,6

Data Management Seminar, 9-12th July 2007, Hamburg Arrange Test Booklets for Marking Step 2: Divide into Two Sets (only Booklets 1-3) Booklet 1 Booklet 1 Booklet 1 Booklet 1 R ~60 Booklet 1 R Set A Booklet 1 R Set A Booklet 1 R Set A Booklet 1 R Set A Booklet 1 Set B Booklet 1 Set B Booklet 1 Set B Booklet 1 R Set B ~120 Booklet 1 Booklet 1 Booklet 1 Booklet 1 R Marking Team A Marking Team B 50% 27

Data Management Seminar, 9-12th July 2007, Hamburg Steps for Booklet Arrangement Step 3: Divide into Batches Step 2: Divide into Two Sets Step 1: Sort by Booklet Type... 1,2,...,6... Set A, Set B... Batch 1, Batch 2,... 28

Data Management Seminar, 9-12th July 2007, Hamburg Arrange Test Booklets for Marking Step 3: Divide into Batches 60 Booklet 1R Set A Booklet 1 Set A Booklet 1R Set A Booklet 1 R Set A ~ 15 Booklet 1R Set A Booklet 1 R Set A Marking Team A ~ 15 Booklet 1R Set A Booklet 1 R Set A ~ 15 Booklet 1R Set A Booklet 1 R Set A ~ 15 Booklet 1R Set A Booklet 1 R Set A ~ 15 Booklet 1R Set B Booklet 1 R Set B ~ 15 Booklet 1R Set B Booklet 1 R Set B 60 Booklet 1R Set B Booklet 1 Set B Booklet 1R Set B Booklet 1 R Set B Marking Team B ~ 15 Booklet 1R Set B Booklet 1 R Set B ~ 15 Booklet 1R Set B Booklet 1 R Set B 29

Data Management Seminar, 9-12th July 2007, Hamburg Steps for Booklet Arrangement Step 3: Divide into Batches Step 2: Divide into Two Sets Step 1: Sort by Booklet Type... 1,2,...,6... Set A, Set B... Batch 1, Batch 2,... 30

Data Management Seminar, 9-12th July 2007, Hamburg Instrument Preparation Test Administration Codebook Adaptation Receiving Material Data Submission Data Entry and Verification Marking Sampling and Preparing Test Administration 31

Data Management Seminar, 9-12th July 2007, Hamburg Marking Instrument Preparation Test Administration Codebook Adaptation Receiving Material Marking Sampling and Preparing Test Administration 32 Data Submission Data Entry and Verification

Data Management Seminar, 9-12th July 2007, Hamburg Reliability Marking Main Marking 33

Data Management Seminar, 9-12th July 2007, Hamburg Step 5: Reliability Marking Steps for Booklet Arrangement Step 3: Divide into Batches Step 4: Main Marking (Booklets 4-6) Items Marking Sheet Step 6: Return “R” Booklets Step 7: Main Marking (Booklets 1-3) Step 2: Divide into Two Sets Step 1: Sort by Booklet Type... 1,2,...,6... Set A, Set B... Batch 1, Batch 2,... 34

Data Management Seminar, 9-12th July 2007, Hamburg Main Marking Step 4: Main Marking (Booklets 4-6) Booklet Booklet 5 Booklet 6 Booklet 4 Booklet 4 Booklet 5 Booklet 5 Booklet 6 Booklet 6 ~ 120 Note: Booklets 4-6 can be marked by any marker! But: Single items should not all be marked by the same marker.

Data Management Seminar, 9-12th July 2007, Hamburg Reliability Marking Step 5: Reliability Marking Marking Team A Booklet 1R Set A Items Marking Sheet Marking Team B ~ Booklet 1R Set A Items Marking Sheet ~ Booklet 1R Set B Booklet 1R Set B Items Marking Sheet Items Marking Sheet ~

Data Management Seminar, 9-12th July 2007, Hamburg Documenting Reliability Marker 101 Marking Team A Markes ONLY on Constructed Response Items Marking Sheet even number Booklet 1R Set B

Data Management Seminar, 9-12th July 2007, Hamburg Return „R“ Booklets for Main Marking Step 6: Return “R” Booklets Booklet 1R Set A ~ 8 Marking Team AMarking Team B ~ 8 Booklet 1 Set A 38 ~ 8 Booklet 1 Set A ~ 8 Booklet 1R Set A Booklet 1R Set A ~ 8 Booklet 1R Set A ~ 8 Booklet 1R Set B ~ 8 Booklet 1R Set B ~ 8 Booklet 1R Set B ~ 8 Booklet 1R Set B ~ 8 Booklet 1R Set B ~ 8 Booklet 1R Set B ~ 8

Data Management Seminar, 9-12th July 2007, Hamburg Main Marking Step 7: Main Marking (Booklets 1-3) ~ 8 Booklet 1R Set A Booklet 1R Set A Booklet 1R Set B Booklet 1R Set B Marking Team AMarking Team B

Data Management Seminar, 9-12th July 2007, Hamburg Documenting Main Marker Marking Team B 202 Booklet 1R Set B even number 101

Data Management Seminar, 9-12th July 2007, Hamburg Instrument Preparation Test Administration Codebook Adaptation Receiving Material Marking Sampling and Preparing Test Administration 41 Data Submission Data Entry and Verification

Data Management Seminar, 9-12th July 2007, Hamburg Data Entry and Verification Data Entry and Verification Instrument Preparation Test Administration Codebook Adaptation Receiving Material Marking Sampling and Preparing Test Administration 42 Data Submission

Data Management Seminar, 9-12th July 2007, Hamburg Complete Student Information Enter Questionnaire Data Enter Achievement and Reliability Data Data Entry and Verification 43

Data Management Seminar, 9-12th July 2007, Hamburg WinDEM – New Data File Create New Data file (For the First time) From a codebook 44

Data Management Seminar, 9-12th July 2007, Hamburg WinDEM – New Data File Extension *.sdb Codebook (*.sdb)Data File (*.dbf) 45

Data Management Seminar, 9-12th July 2007, Hamburg WinDEM – New Data File New Data file File READY for: Data Entry 46