1st NRC Meeting, October 2006, Amsterdam 1 ICCS Sampling Design

1st NRC Meeting, October 2006, Amsterdam 2 Overview Target Population Definitions: Students, Teachers Sampling design: purpose Sampling design: method Definition of sampling units: schools, classes Sample sizes (Field Test, Main Survey) Response rate requirements

1st NRC Meeting, October 2006, Amsterdam 3 Target Population Definitions: Students Target of investigation are......all students enrolled in the grade that represents eight years of schooling, counting from the first year of ISCED Level 1, providing the mean age at the time of testing is at least 13,5 years. This will be mostly grade 8.

1st NRC Meeting, October 2006, Amsterdam 4 Target Population Definitions: Students Background of the definition: The amount of schooling represented by the grade assessed should be comparable across countries and is based on an internationally accepted classification scheme. Based on the definition for students in CIVED. Some countries tested in CIVED 1999 grade 9, those countries are invited to study grade 8 and grade 9.

1st NRC Meeting, October 2006, Amsterdam 5 Target Population Definitions: Teachers Target of investigation are......all teachers teaching any subject in the grade of the target population of students (mostly grade 8) plus all teachers teaching any subject in one adjacent grade (preferably one grade below, mostly grade 7).

1st NRC Meeting, October 2006, Amsterdam 6 Target Population Definitions: Teachers This is a substantially different approach than in CIVED (civic-related subject teachers teaching in the sampled classroom were selected). Advantages: –Higher degree of comparability across countries –Analysis on teacher level is possible Disadvantages: –Trend study of teacher survey impossible –No direct student-teacher-linkage

1st NRC Meeting, October 2006, Amsterdam 7 Sampling Design: Purpose Unified approach as basis for cross-national comparisons Samples have to be representative for the population in each country Exact measuring of sampling errors Decreasing of sampling errors to a minimum

1st NRC Meeting, October 2006, Amsterdam 8 Sampling Design: Method 3-stage stratified cluster sample: Schools ClassesTeachers Students

1st NRC Meeting, October 2006, Amsterdam 9 Sampling Design: School Selection Definition School: A school is one whole administrative unit with a defined number of teachers and students, which can include different programs, tracks or shifts. (??) Selected from a comprehensive school list Selection probability depends on the size of the school (PPS – sample) Replacement schools: 1 (Field Test) 2 (Main Survey)

1st NRC Meeting, October 2006, Amsterdam 10 Sampling Design: Class Selection Definition Class: Classes are mutually exclusive and exhaustive groups of students in the same grade within schools. Each student in a school has to belong to one, and only one class. „Class“ can be defined differently in different countries, e.g. home room, math class... Selected from a list of classes from selected schools Simple random selection

1st NRC Meeting, October 2006, Amsterdam 11 Sampling Design: Student Selection Definition = Target Populations Students All students in selected classes will be tested

1st NRC Meeting, October 2006, Amsterdam 12 Sampling Design: Teacher Selection Definition = Target Population Teachers Selection from a list of teachers from selected schools Simple random selection No direct linkage to selected students

1st NRC Meeting, October 2006, Amsterdam 13 Sample Sizes IEA standard: sampling design has to provide precise and economical estimates of the population parameters. Therefore an effective sample size of 400 students per country is required. Design Effect: increases sample size significantly (e.g. in TIMSS: minimum 4000 tested students)

1st NRC Meeting, October 2006, Amsterdam 14 Sample Sizes: Main Survey Schools: Minimum: 150 schools per country Some countries have to select more schools (depending on MCS and IC) Classes: Minimum: 1 class per school 2 classes in larger schools in countries with average class sizes below 30 (tbd with WS)

1st NRC Meeting, October 2006, Amsterdam 15 Sample Sizes: Main Survey Students: Minimum: 4,500 sampled students per country Test all students in selected classes Teachers: Minimum: 15 sampled teachers per school Survey all eligible teachers, if there are not more than 20 (??)

1st NRC Meeting, October 2006, Amsterdam 16 Sample Sizes: Field Test Schools: Minimum: 25 schools per country Classes: Minimum: 1 class per school 2 classes in larger schools in countries with small average class size

1st NRC Meeting, October 2006, Amsterdam 17 Sample Sizes: Field Test Students: ,000 students per country (depending on number of test items respectively booklets, 200 responses per item) Test all students in selected classes Teachers: Survey all eligible teachers in selected schools

1st NRC Meeting, October 2006, Amsterdam 18 Sample Sizes: Field Test Students: ,000 students per country (depending on number of test items respectively booklets, 200 responses per item) Test all students in selected classes Teachers: Survey all eligible teachers in selected schools

1st NRC Meeting, October 2006, Amsterdam 19 Response Rate Requirements Schools: 85% participation before use of replacement schools Students: 85% participation within participating schools Teachers: 85% participation within participating schools Overall response rate: ??

1st NRC Meeting, October 2006, Amsterdam 20 Thank You!