1. Design of the 2011 Census Coverage Survey Owen Abbott (ONS) James Brown (Institute of Education)

2. 2 Aim of the Presentation Role of the Census Coverage Survey (CCS) Review of the CCS design in 2001 Basic structure of the design in 2011 Evaluating the basic design decisions Issues to be resolved

3. 3 Role of the CCS Provides the main data for assessing census coverage To achieve this it must: operate ‘independently’ of the census have ‘complete’ coverage of the ‘population’ high response rate ~ 90% happen close to Census Day (probably 6/7 weeks in 2011 compared to 3/4 weeks in 2001) The design needs to capture variation in census coverage (geographic, demographic)

4. 4 2011 Coverage Assessment Overview Estimation 1) DSE 2) Ratio estimation Matching Adjustment 2011 Census OUTPUT: Population estimates with CIs Census Coverage Survey (CCS) OUTPUT: Census Database

5. 5 Structure of the 2001 CCS Design National ‘Hard-to-Count’ (HtC) index with three levels Formed Estimation Areas (EAs) by grouping contiguous LAs Stratification by HtC within each of 101 EAs Further stratification at design by 1991 age-sex structure (which was ignored in the estimation) No direct control of the individual LA samples Clustering based on selecting 1991 Enumeration Districts (EDs) and then fixed # postcodes per ED

6. 6 Evaluation of the 2001 CCS Design Based on rather ad-hoc data and intelligence from 1991 HOWEVER The CCS worked well in most situations Problems with Manchester relate to the CCS design information not reflecting change Have much more data on coverage patterns for use in the 2011 design

7. 7 Some key questions What should the sample unit be? How much clustering? What should the stratification structure be? How do we allocate the sample?

8. 8 Basic Sampling Unit CCS must be independent of the Census THEREFORE Implies postcodes must be the basic unit of the design No lists of households and/or addresses with complete coverage exists (independent of the Census) Problems identifying the boundaries but at least postcodes are known by householders…

9. 9 How Much Do We Cluster? Cost against efficiency -Clustering reduces statistical efficiency but can be offset by reduced fieldwork costs allowing a larger sample -Often makes fieldwork management easier as a single postcode is too small for a single interviewer Some choices for cluster unit and number of postcodes per cluster Part of the estimation strategy in 2001 used the clustered structure

10. 10 Stratification Structure (at design) Can be different at estimation (as long as we think about it at design) -How do we reflect LAs in the design? -In 2011 do we still need to use Estimation Areas at design? National verses local HtC index -A national index gives more flexibility at estimation -Allows LAs to be grouped

11. 11 Sample Allocation Need to be careful that we are not highly optimised for 2001 as 2011 could be quite different -Need to build on the likely patterns based on 2001 while being robust to change (sample everywhere) Driven by a ‘design variable’ to proxy for under-count -Trying different ones but the analysis in this presentation based on imputed households in 2001

12. 12 ANOVA Results ANOVA of the design variable across OAs Shows benefit of both HtC and geography Geography is most important

13. 13 Level of Clustering OAs are the obvious choice for PSUs Have 2001 Census data Some external data Consistent size, homogenous More consistent with postcodes Comparing with EDs and SOAs For fixed costs, OAs do well with one to three postcodes Likely to have some clustering - Helps DSE and field management

14. 14 Performance of Designs Using LAs

15. 15 Comparing LA designs with EA design

16. 16 Performance of Designs Using LAs LA level designs appear to be ‘best’ for the total population -Agrees with ANOVA results This suggests we should break the direct link between design and estimation -Best to design by LA -Then group LAs at estimation Can control LA sample size -Benefits for small area estimation -Easier to understand for users?

17. 17 Issues to be Resolved Collapsing strata Minimum sample sizes Grouping of LAs at estimation - Contiguous or non-contiguous

18. 18 Summary - ‘Likely’ Basic Structure Stratified cluster sample of postcodes LA and HtC stratification at design Clustering based on 2001 OAs EAs formed from the LAs for estimation Pre-specified (not necessarily geographic) but checked after Census fieldwork

19. 19 Summary CCS is key component of coverage assessment Design of the survey is critical Maintaining a robust approach -Guarding against moving too far from 2001 design -More weighted towards harder areas than in 2001 -But still sampling within every LA