Presentation on theme: "NTTS conference, February 18 – 20 2009 New Developments in Nonresponse Adjustment Methods Fannie Cobben Statistics Netherlands Department of Methodology."— Presentation transcript:
NTTS conference, February 18 – New Developments in Nonresponse Adjustment Methods Fannie Cobben Statistics Netherlands Department of Methodology and Quality
NTTS conference, February 18 – In this presentation… Response selection model Use of response propensities Application to POLS 2002 Discussion
NTTS conference, February 18 – The response selection model Consists of two equations: Response equation (dichotomous) Survey item equation (continuous) The error terms are allowed to be correlated The outcome is adjusted for a possible selection bias
NTTS conference, February 18 – Extensions to response selection model Multiple selection equations, i.e. different response types Contact equation Participation equation Survey item equation Contact and participation are dependent and can both introduce a bias for the survey item If desirable, equations for other response types Categorical survey items
NTTS conference, February 18 – Response selection model – contact and participation ςi*ςi* γi*γi* P i = 1 C i = 1 P i = 0 C i = 0 Sample Non-contactContact RefusalParticipation Y i * missing Y i * = Y i Y i * missing
NTTS conference, February 18 – Advantages Model relationship with both R and Y Efficient use of auxiliary variables; paradata Closely follow fieldwork process Disadvantages Model based; dependent on distributional assumptions Issues of identification; exclusion restriction
NTTS conference, February 18 – Response propensities (1) Assume: Sample is selected from a sampling frame by some random selection procedure Two groups: R=1 response R=0 non-response X auxiliary information, available for all elements. For instance from the sampling frame. ρ(X) = P(R=1| X) is the propensity score, for instance determined by a logistic regression, i.e.
NTTS conference, February 18 – Response propensities (2) We can use the response propensities to adjust for nonresponse bias: Directly Response propensity weighting Response propensity stratification Indirectly In combination with linear weighting
NTTS conference, February 18 – Direct use of response propensities Response propensity weighting, Särndal (1981) Response propensity stratification
NTTS conference, February 18 – Indirect use of response propensities GREG – estimator with adopted inclusion probabilities
NTTS conference, February 18 – POLS 2002 Integrated Survey on Household Living Conditions (in Dutch: Permanent Onderzoek LeefSituatie) Monthly persons are selected Questions on living conditions, safety, health Basic module for persons > 12 years Datafile aggregated over 2002: n = and n r = (57%) Survey variables: Employment, Education and Religion Numerous auxiliary variables, such as age, region, house value, social insurance, ethnicity, etc.
NTTS conference, February 18 – Analysis POLS 2002 Two models: Weighting model (relation Y and R; Schouten, 2004) Age 15 + Houseval 14 + % Non-natives 8 + Ethnicity 7 + Region 15 + Type hh 4 + Telephone 2 Response model (relation R; psuedo R 2 = 2,2%) Age 15 + Houseval 14 + Urbanicity 5 + Mar_staat 4 + Ethnicity 7 + Region 15 + Type hh 4 + Telephone 2 (1) (2) Aim: Compare different response propensity methods Compare regular GREG-estimator and other methods
NTTS conference, February 18 – Propensity stratification Propensity weighting Survey itemResponse average GREG (1) (2)(1)(2)Propensity GREG Employed >= 12 hours52,4 (0,35) 53,7 (0,18) 53,5 (0,38) 53,7 (0,38) 53,7 (0,34) 53,8 (0,34) 53,8 unemployed6,7 (0,18) 6,4 (0,11) 6,4 (0,17) 6,4 (0,17) 6,4 (0,17) 6,4 (0,18) 6,4 < 12 hours40,9 (0,35) 39,9 (0,17) 40,0 (0,35) 39,9 (0,37) 39,9 (0,34) 39,8 (0,33) 39,9 Results - Employment No significant differences between method; clear difference with response average GREG and propensity weighting with same model: same results Propensity weighting and propensity GREG highest estimate employed labour force
NTTS conference, February 18 – Conclusions Remarks: Almost the same variables in weighting model and response model Low pseudo R 2 response model Conclusions: 1.Small difference between methods 2.No reduction of variance direct response propensity methods; reduction of variance GREG-estimator 3.Propensity weighting and GREG-estimator with same model gives same results for employed and education
NTTS conference, February 18 – Thank you for your attention!
NTTS conference, February 18 – Probability based surveys (2) Under-coverage in telephone surveys Two groups: with telephone (C=1) without telephone (C=0) Non-response Two groups: Respondents (C=1) Non-respondents (C=0)
NTTS conference, February 18 – Discussion Which variables should be inserted in the different equations, and how should these variables be selected? How to construct the hierarchical structure of the model. For instance, which response types should be distinguished? Is it profitable to construct a model for every survey item? If so, how should we deal with this in practice? The response selection assumes a correlation structure between response types and survey items. This correlation has a model-based interpretation, but how should it be interpreted in practice?
NTTS conference, February 18 – Example relationship R and Y in the LFS Relation between the number of contact attempts and estimated size of the labour force 1 attempt 2 attempts 4 attempts all attempts Quarter size of the labour force