1. Chasing hard to get cases in panel surveys – is it worth it? Nicole Watson, University of Melbourne Mark Wooden, University of Melbourne

2. www.melbourneinstitute.com Acknowledgements This paper uses unit record data from the Household, Income and Labour Dynamics in Australia (HILDA) Survey. The HILDA Project was initiated and is funded by the Australian Government Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) and is managed by the Melbourne Institute of Applied Economic and Social Research (Melbourne Institute). The findings and view reported in this paper, however, are those of the authors and should not be attributed to either FaHCSIA or the Melbourne Institute.

3. www.melbourneinstitute.com Research Questions 1.Are hard-to-get cases (that are interviewed) noticeably different from other interviewed cases? 2.Do the cases that require a lot of effort in one survey wave require a lot of effort in all waves? 3.Are hard-to-get cases in one wave simply going to attrit at the next wave? 4.Is data quality inversely associated with effort?

4. www.melbourneinstitute.com Data: The HILDA Survey  National household panel survey –Nationally representative household sample (7682 hhs) –Started in 2001; annual interviewing –Face-to-face interviews (mostly) w all persons 15+ yrs –New household members added each wave  Response –W1 hh response rate = 66% –Re-interview rates: w2 = 87%, rising to 95%+ by w6  Sample size (unbalanced panel, 11 waves)* –N = 143,812; i =22,019

5. www.melbourneinstitute.com Defining “Hard-to-Get” Measure based on: Examples of previous researchHILDA measure 1.Call attempts Fitzgerald & Fuller (1982); Cottler et al. (1987); Lin & Schaeffer (1995); Lynn et al. (2002); Yan et al. (2007); Heerwegh et al (2007); Hall et al. (2011) (i) 13+ calls vs fewer (ii) 7+ calls vs fewer 2.Time to final outcome Yan et al. (2004); Haring et al. (2009) (i)Responded in initial FW phase vs Later (ii)Prior to end of year vs Post New Year 3.Initial refusal Robins (1963); Smith (1984); Lin & Schaeffer (1995); Cohen et al (2000); Lynn et al. (2002); Yan et al. (2004); Billiet et al. (2005); Kaminska et al (2010); Hall et al. (2011) Initial refusal vs No refusal 4.Respond’t cooperation Kaminska et al (2010) Ivwr assessed cooperation: Very poor / Poor / Fair vs Excellent / Good

6. www.melbourneinstitute.com How Many Cases are Hard-to-Get?

7. www.melbourneinstitute.com Who are Hard-to-Get Cases Most Like? Tests of joint significance from MNL predicting response type at time t (P) Characteristics at t-1 LateInitial refusal13+ calls EasyNREasyNREasyNR Age0.0620.000 0.0030.000 Female 0.1940.0470.0380.0090.000 LF status x Hours 0.0000.0100.0000.0290.000 Home ownership 0.2300.3990.8120.0970.0000.006 Country of birth 0.0000.0010.0000.0010.0000.043 Education 0.0640.000 0.0700.000 Marital status 0.0000.0560.6040.2830.0000.013 Region 0.0000.6110.0000.2440.000 # adults in hh 0.000 0.0220.0000.004 # children in hh 0.3680.0640.0280.0030.0890.072 Eq. hh income 0.000 0.002 0.0320.009 LT health condition 0.0480.0470.0270.0210.004 HH moved 0.0000.0070.0000.0630.0000.144

8. www.melbourneinstitute.com Are Hard-to-Get Cases Always Hard to Get? (I)

9. www.melbourneinstitute.com Are Hard-to-Get Cases Always Hard to Get? (II)

10. www.melbourneinstitute.com Do Hard-to-Get Cases Exit at Next Wave?

11. www.melbourneinstitute.com Impact of Interview Status at t-1 on Response Outcomes at t

12. www.melbourneinstitute.com Number of Interviews by Wave 1 Interview Status

13. www.melbourneinstitute.com Do Hard-to-Get Cases Deliver Lower Quality Data? LateInitial refusal13+calls EasyHardEasyHardEasyHard Response set bias: Satisfaction1.21.31.21.31.21.4 Response set bias: Job satisfaction2.52.9*2.53.0*2.52.8 Item NR: FY wages5.08.2**5.28.8**5.19.3** Item NR: FY pensions1.83.2**1.93.7**1.94.0** Rounding: FY wages (nearest $000)67.872.5**68.172.5**68.172.7** Rounding: FY pensions (nearest $000)10.3 10.214.1*10.217.3** Phone interview4.424.9**5.427.0**5.036.8** Returned SCQ If phone respondent68.553.8**66.748.4**69.244.9** If F2F respondent93.280.3**92.778.3**92.778.3**

14. www.melbourneinstitute.com Summary  Size of hard to get (H2G) group is definition dependent.  H2G are distinct from both easy-to-get cases and non-respondents.  Most H2G cases (P=70-73%) will be E2G at next survey wave.  H2G more likely to attrit (P=12-17%), but most don’t.  There may be some implications for data quality.