ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid 1 Sample Participation and Sampling Weights

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Overview Progress report Participation in ICCS Weights –What are weights? –Why use weights? –What weights are there in ICCS? –How were ICCS weights calculated?

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Progress report 38 educational systems participated in ICCS Sampling weights were calculated for all of them –For the student study and the teacher study –For six additional grades Variables for variance estimation were prepared Participation rates were calculated

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Participation categories Three categories for sampling participation were defined in ICCS for students and for teachers –Category 1: acceptable participation rate without the use of replacement schools –Category 2: acceptable participation rate when replacement schools are included –Category 3: participation rate not acceptable even when replacement schools are included Each country was grouped into a category during sampling adjudication Each country was informed about the decision

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Participation categories - students Category 1: 28 countries Category 2: 8 countries Category 3: 2 countries

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Participation categories - teachers Category 1: 22 countries Category 2: 5 countries Category 3: 9 countries Two countries could not be reported

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Weights. They are everywhere. Weights are in your data Weights are in the tables of the reports –All means and percentages in the reports will be calculated with weighted data

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid What are weights? Weights are values that are assigned to every sampling unit The weight of a sampled unit indicates the population that is represented by this sampled unit 2 200

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Why do we need weights? Weights allow drawing conclusions about the population based on information from the sample Weights allow unbiased estimates of population parameters Un-weighted data only allow conclusions about the sampled units

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Population  sample Example: in the population, 20% of the students are in private schools, 80% are in public schools The NRC decides to over-sample students from private schools In the sample, 50% of the students are from private schools, 50% are from public schools

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Sample  estimate To estimate the correct proportion of students in the population, we must assign different weights to the students in the sample

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid What weights are there in ICCS? Student weights –TOTWGTS Teacher weights –TOTWGTT School weights –TOTWGTC

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Why do ICCS weights differ? ICCS student weights differ, because the following five elements differ: Selection probabilities –1)... of the sampled schools –2)... of the classes within the schools Non-participation –3)... of sampled schools –4)... of sampled classes within the schools –5)... of students within the classes Each element corresponds to a weight factor

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid 1) School Base Weight WGTFAC1 The school base weight is the inverse of the selection probability of the school –Low selection probability  large weight –High selection probability  small weight 5 1

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid 1) School Base Weight: PPS Schools were sampled with probability proportional to size (PPS) Large schools had larger selection probabilities Therefore, they now have smaller school base weights Example: the large school is three times larger The small school has a three times larger school base weight ww x 3

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid 2) Class Base Weight WGTFAC2S The class base weight is the inverse of the selection probability of the class Different selection probabilities because of the different number of classes in a school 6 2

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid 3) School Weight Adjustment WGTADJ1 Adjusts for schools that did not participate Example: Schools from two different explicit strata – one blue school does not participate 5/4

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid 4) Class Weight Adjustment WGTADJ2S Adjusts for classes that did not participate In most countries, only one class was selected per school Almost always, this factor equals 1

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid 5) Student Weight Adjustment WGTADJ3S Adjusts for students that did not participate Example: Students from two different classes – one blue student does not participate 5/4

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid The final student weight We now can compute the TOTAL STUDENT WEIGHT TOTWGTS = x School Base Weight x Class Base Weight x School Adjustment Factor x Class Adjustment Factor x Student Adjustment Factor

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Teacher weights The TOTAL TEACHER WEIGHT is computed similarly TOTWGTT = x School Base Weight x Teacher Base Weight x School Adjustment Factor x Teacher Adjustment Factor x Teacher Multiplicity Factor –adjusts for the fact that some teachers teach in more than one school

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid More about weights: IDB Training on Thursday –how weights are used for data analysis –some examples of what can go wrong if weights are not used for data analysis Please join!

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Country sessions If you have any questions about participation rates or weights... Please sign up!

ICCS th NRC Meeting, February 15 th - 18 th 2010, Madrid Thank you for your attention!