1. 2011 Census 2007 Census Test – emerging findings Garnett Compton, ONS Updated 4 September 2007 BSPS – 12 September 2007

2. Session Aim Aim: To share emerging findings from the 2007 Census Test Outline plan for publishing 2007 Census Test evaluation Present a brief update on other 2011 Census developments

3. Contents Test background Results: Address register development Post-out/hand delivery Income question Key findings of 2007 Test Further evaluation and publication Did you know ….

4. 2007 Census Test - Objectives Test objectives: Assess the effect on response of: –Inclusion of an income question; and –the use of post-out to deliver questionnaires. Assess the feasibility of major innovations in proposed 2011 Census operational procedures: –Outsourcing of field staff recruitment, pay and training; –development of an operational intelligence system to enable individual questionnaires to be tracked; and, –development of an address list and address checking procedures.

5. 2007 Census Test - Sample sizes by LA and ETC

6. 2007 Census Test – High-level Design Address checking –Conducted in all Test areas during Sept and October –Split discretionary and full contact methods –Used controlled errors Delivery –50% Post-out, 50% hand delivery –50% questionnaires income, 50% no income –For hand delivery 3 attempts at contact over 2 week period Collection/Follow-up –Central post-back –23 May – 22 June –3 attempts everywhere –reminder letter to all outstanding addresses as at 31 May

7. 2007 Census Test – Key constraints Key constraints affecting evaluation of the Test: Voluntary –Relied on public’s good will to complete a return Publicity Sample – skewed to harder to enumerate areas Follow-up –Fixed number of follow-up attempts everywhere

8. Results

9. Household* return rates by LA * As at 15 July

10. Household* return rates by ETC * As at 15 July

11. Address register development Results

12. Key results Address checking (AC) found about 12% new addresses –Time consuming to validate and update AR 4.4% (4,400) of questionnaires undelivered –Of which 17% were added by address checkers –Reasons include vacant properties, non-existing addresses, duplicate addresses. 1,200 new addresses found during enumeration –Of which, over 50% found at follow-up. –Nearly 70% of new addresses were sub-premise addresses – suggest existed at time of AC.

13. Early conclusions Suggests we need to plan to do a 100% address check in 2011 no matter what the delivery design. Plan is for a rolling address check over a longer period of time (4-6 months). –enables better quality address checking; and, –enables more time to update the final address register with Address check findings. Consider a re-address check shortly before the Census in a small %’age of areas

14. Early conclusions cont … Main enumeration and controlled errors didn't identify all missing addresses – need to improve methods and guidance Issues around whether information with address suppliers can be shared; Criteria for deciding which address products to use as a base under consideration.

15. Delivery Method Results

16. Household return rates* by delivery method by ETC * As at 15 July

17. Success rates* at follow-up by delivery method by ETC * As at 15 July ETCPost-outHand delivery 138.7%39.9% 229.1%29.4% 324.2%26.1% 419.1%18.4% 514.9%16.3% Overall23.9%24.3%

18. Delivery method – estimated cost savings Estimated cost savings between 100% post- out and 100% hand delivery Initial return* rate difference (%’age points) Estimated savings 5£28m - £35m 6£25m – £33m 10£6m - £21m 15-£18m - £1m * At the start of follow-up – 23 May

19. Address register coverage New addresses found during the 2007 Census Test by delivery method During deliveryDuring follow-upTotal Delivery method No.%’ageNo.%’ageNo.%’age Post-out90.02%4780.94%4870.95% Hand delivery5401.06%1810.36%7211.42% Total5490.54%6590.65%1,2081.19%

20. Delivery method - conclusions Post-out has an impact on return rates, minimal impact which can be addressed through additional follow-up. A post-out methodology will provide significant savings (£25-£33m) to invest in targeted follow-up and community liaison. Improvements identified for the address register and follow-up procedures suggest that the levels of AR undercoverage will be small and manageable. Decision: In E&W, post-out will be the primary means (at least 85%) of delivering questionnaires in 2011.

21. Income – Setting the scene Including income depends on: results from the Test; and, consultation on other topics and relative priority of income in relation to other demands. Further analysis required as follows: Quality and accuracy of responses to income question Item imputation rates Public perception Other data sources

22. Household return rates* by income/no income question by ETC * As at 15 July

23. 2007 Census Test - Key findings Delivery method In E&W, post-out will be the primary method (at least 85%) of delivering questionnaires in 2011. Address register development Address checking will be required for 2011, currently planning 100% for E&W. Outsourcing recruitment, training and pay: Worked well in the Test and are considering outsourcing for 2011;

24. Other key findings: Recruitment: More difficult than expected in some areas, mostly delivery enumerators. Good MI to identify and manage problems early. Training: Some development issues with e-learning but overall well received/effective. Pay: Much simpler system defined thereby avoiding some of the problems experienced in 2001. Hourly pay worked well, provided good control and flexibility; travel expenses still considered cumbersome. Refinement for 2011 required.

25. Other key findings: LA Liaison: Principles and benefits of LA Liaison proven. Variable engagement across the LAs – some more engaged than others Going forward we need to consider: –methods for achieving more consistency across LAs; –Making it simpler for ONS and LAs –Achieving Chief Exec buy-in. Follow-up: Transfer of information held centrally to field staff worked well within the constraints of the Test. A good start to follow-up is imperative – need to review start dates. Organisation and management of field staff worked well, but more development required on doorstep interaction to convince respondent to respond.

26. 2007 Census Test Evaluation – planned publications Publication Statistical Evaluation of the 2007 Census Test 2007 Census Test - Evaluation of the delivery method 2007 Census Test - Evaluation of the Income question Evaluation of 2007 Census Test – Summary Report

27. Did you know? Some other key Census developments: Rehearsal - Spring 2009 Route A – contract to be let in January 2008 Route C – start procurement in January 2008 Finalising questionnaire – Spring 2008 White Paper – Autumn 2008

28. Questions ????

29. Household return rates* by delivery method by LA * As at 15 July

30. Household return rates* by income/no income question by LA * As at 15 July