Teacher Survey Design National Research Coordinators Meeting Amsterdam, October 2006

NRC Meeting Amsterdam Oct 2006 Issues to be discussed Purpose of teacher survey Target population and exclusions Sampling and administration procedures Sample sizes Response rate requirements

NRC Meeting Amsterdam Oct 2006 CIVED teacher survey Population: Teachers teaching in civic- related subjects Identification of “civic subjects” using cognitive test content One to three teachers per student Student-teacher link Use of student weights for reporting

NRC Meeting Amsterdam Oct 2006 New direction for ICCES Focus on “whole school approach” in C&C education and school climate Inclusion of teacher from all subjects Possibility of reporting results from teacher survey (with teacher weights) Greater emphasis on development of teacher instrument

NRC Meeting Amsterdam Oct 2006 Uses of teacher data Reporting at country level: General context of C&C education Reporting of school level aggregates: School implementation of C&C education Reporting of school level aggregates in student level or multi-level analyses

NRC Meeting Amsterdam Oct 2006 Target population Four potential options –Teachers teaching any subject in the international target grade 8. –Teachers teaching any subject in the international target grade 8 plus one adjacent grade. –Teachers teaching any subject in lower secondary study programmes. –Teachers teaching in any subject and any grade at the sampled school.

NRC Meeting Amsterdam Oct 2006 Issues regarding target population Comparability of populations –Best for teachers only teaching in grade 8 –Worst for ALL teachers at school Achieving representative sample of teaching staff at school –Grade 8 restricted to one particular grade –Lower Secondary teacher sample appropriate –Sample of ALL teachers may not reflect those influencing tested students (combined schools with different ISCED levels)

NRC Meeting Amsterdam Oct 2006 Possible teacher exclusion criteria Minimum teaching load? Status of teachers (specific subjects, casual staff) Temporary absence (replacement teachers instead?)

NRC Meeting Amsterdam Oct 2006 Sampling and Administration PPS sampling at school level (already done) List of teachers in target population provided by school coordinator Random selection from list Administration of teacher questionnaire through school coordinator

NRC Meeting Amsterdam Oct 2006 Teacher sample sizes Smaller sample sizes (< 5) only viable when aim is to report teacher sample at country level only School level aggregates should be based on at least 10 teachers per school Non-response needs to be taken into account when determining sample size

NRC Meeting Amsterdam Oct 2006 Response rate requirements The minimum response rate for IEA studies is 85 percent For CIVED, the response rates ranged between 70 and 100 percent Standards for reporting will have to be developed Further discussions with the IEA TEG about reporting of data from countries with response rates below 85 percent.

NRC Meeting Amsterdam Oct 2006 Conclusions Preference for target population definition that maximises comparability (grade 8 teachers) Teacher exclusion criteria to be established with NRCs Tentative within-school sample size set at 15 (to ensure an achieved size of 10 per school)

NRC Meeting Amsterdam Oct 2006 Questions or comments?