1. 2007 Census Test – Analysis of Coverage Owen Abbott Methodology Directorate

2. Agenda Introduction Census Test Evaluation Survey Within household coverage -Undercount -Overcount Visitor coverage Characteristics of non-responding households Summary

3. Coverage Evaluation Purpose: -To see if coverage levels or patterns have changed since the 2001 Census -Undercount and Overcount -To evaluate whether the test treatments have any differential impact -To evaluation the coverage of the visitor information -To evaluate the reasons for non-response Feeds into census procedures, publicity, coverage assessment etc

4. The Census Test Evaluation Survey Sample -1223 responding households -247 non-responding households Questionnaire: -Captured census data preloaded -Established usual residents/visitors and probed where differences -Asked Non-responders for reasons and collected characteristics of HH

5. Within household undercount Used CTES to determine whether individuals missed from responding household If CTES identifies a resident not in the census then this is an undercount -Due to recall problems, likely to be an underestimate -Some matching problems (Name capture) -I form problems – not used in CTES Found 72 cases 34 of these in households smaller than 6 residents Undercount 1.3% (±0.4%)

6. Within household undercount In 2001 in the Test LAs the CCS measured undercount within households to be 1.8% This difference is not statistically significant This is an indication that the problem is not getting significantly worse -Further testing of instructions and listing grids -Need to check again in the rehearsal -Ensure the CCS measures and adjusts robustly

7. Within household overcount Used CTES to determine whether individuals should NOT have been counted in a responding household If a resident in the census is not in the CTES then this is overcount -Likely to overestimate due to recall issues More matching/processing errors -We think we managed to remove most of them Found 23 ‘genuine’ cases Overcount 0.86% (±0.35%)

8. Within household overcount In 2001 overall, we estimated 0.4% nationally (using Longitudinal Study and study of duplicates) Is this an indication that this is getting worse? -A good indication that people misclassified themselves when should have been visitors -Need to check again in the rehearsal -Need to measure (and adjust) in 2011

9. Within household coverage Neither undercount or overcount were significantly different between: -Postout and hand delivery -Income and Non-income questionnaires -(or the interaction of these) -ETC levels (a slightly surprising result)

10. Characteristics – Household size Undercount within households varied significantly by size Overcount didn’t HH SizeUndercountOvercount 10.23%0.69% 21.14%0.57% 31.32%0.44% 41.80%1.20% 53.20%2.40%

11. Visitor coverage Used CTES to determine coverage of visitors in responding households -Both undercount and overcount -Likely to be recall problems Found lots of duplication of visitors and residents 59 households declared a visitor (in Census or CTES) 8 undercounted visitors – 40% undercount 23 overcounted visitors – 115% overcount

12. Visitor coverage Lots of evidence of misunderstanding of visitors questions Placement at the beginning confused many residents Conclusion is that coverage of visitors in the test was poor -Additional testing of these question is ongoing -Review coverage again in the rehearsal

13. Characteristics of non-responders To check whether patterns had changed since 2001 and to check ETC Asked whether they had got a questionnaire: -Didn’t get one (18%) -Don’t remember (9%) Asked for reason for non-response -60% said they were too busy/weren’t interested/didn’t want to disclose information or didn’t think it was important

14. Characteristics of non-responders Ethnicity -comparison of NR distribution with responder distribution -Shows similar patterns to 2001 Household NRsCensus Test Ethnicityrespondents All white87.0%90.5% All asian3.1%3.1% All black3.7%1.6% All other6.2%4.8%

15. Characteristics of non-responders Household type - Shows similar patterns to 2001 Household NRsCensus Test Structure respondents Single 20 - 305.3%2.0% Single 30 - 407.1%3.9% Single 40 - 8013.6%24.8% Single age 80+10.7%5.9% Two, < 203.6%0.8% Two 20 - 306.5%4.1% Two 30 - 407.1%5.0% Two 40 - 808.9%22.5%

16. Summary Coverage within households: -Undercount not higher than in 2001 Census -Overcount higher -No difference between test treatments Visitor information from the test was of poor quality Characteristics of non-responding households similar to 2001 Apathy is main reason for non-response