User Workshop Manchester 26th November 2012 Beyond 2011

User Workshop Manchester 26th November 2012 Beyond 2011
Alistair Calder – Head of Stakeholder Engagement, Beyond 2011 ONS

Introduction The Beyond 2011 Programme What are the Options 1 hour Questions / Discussion 15 mins Questions & ideas Research – progress and plans 45 mins 12:30 LUNCH The options revisited 30 mins 30 mins Business case, benefits & quantification 2:15 TEA BREAK Break out session Identify the benefits Feedback 45 mins Questions & ideas Discussion & round-up Feedback & next steps 30 mins 3:45 END

Beyond 2011 (England & Wales)
Context - why change? Statistical options – what are the alternatives? Research plans & the (long) timescale Risks & Issues – what could possibly go wrong? Next Steps

Beyond 2011 The context and background Why bother to change ?

Context Census – every 10 years Uses: Resource allocation
Population statistics  Social conditions  Housing  Uses: Resource allocation Service planning Policy development, monitoring and review Social research EU regulations and duty to report to Parliament small areas + multivariate combinations

The Beyond 2011 Programme Why change? – Why look beyond 2011?
Rapidly changing society Evolving user requirements New opportunities – data sharing Traditional census – costly and infrequent?? UK Statistics Authority to Minister for Cabinet Office “As a Board we have been concerned about the increasing costs and difficulties of traditional Census-taking. We have therefore already instructed the ONS to work urgently on the alternatives, with the intention that the 2011 Census will be the last of its kind.”

Programme Purpose Identify the best way to provide small area population and socio-demographic statistics in future Provide a recommendation in September 2014 - underpinned by full cost-benefit analysis - & high level design for implementation (subject to agreement) Implement the recommendation

Beyond 2011 - Timeline - the key decision
2015 2016 2017 2018 2019 2020 2021 2022 2023 detailed design develop test rehearse run outputs CENSUS SOLUTION 2011 2012 2013 2014 research / definition initiation BEYOND 2011 ‘Phase 1’ Sept 2014 recommendation & decision point 2015 2016 2017 2018 2019 2020 2021 2022 2023 population estimates characteristics outputs detailed design procure / develop develop / test ALTERNATIVE SOLUTION

Beyond 2011 Statistical Options

Beyond 2011 : Statistical options
Full Census (long form to everyone) Rolling Census (over 5/10 year period) Census options Short Form (everyone), Long form (Sample) Short Form + Annual Survey (US model) Aggregate analysis Administrative data options Sample linkage e.g. 1% of postcodes 100% linkage Survey option(s) Address register + Survey

Beyond 2011 – statistical options
FRAME SOURCES DATA ESTIMATION OUTPUTS All National to Small Area Address Population Data Admin Source CENSUS Register Population estimates Admin Source Admin Source Adjusting for ?? missing data and error Coverage Assessment Quality measurement Population distribution provides weighting for attributes incl. under & over-coverage - by survey and admin data? Comm Source increasing later? Household structure etc Commercial sources? Adjusting for non response bias in survey (or sources) Surveys to fill gaps Socio demographic Household Attribute Data Socio demographic Attribute estimates Survey(s) Communal Establishments Longitudinal data Maintained national address gazetteer – provides frame for population data & surveys Interactional Analysis E.g. TTWA

Potential data sources
Population data NHS Patient Register DWP/HMRC Customer Information System Electoral roll (> 18 yrs) School Census (5-16 yrs) Higher Education Statistics Agency data (Students) Birth and Death registrations DVLA? Socio-demographic sources Surveys Commercial sources? Utilities? TV licensing?

Customer Information System
School Census UK Driving Licence Higher Education Students Coverage Of Main Administrative Sources Electoral Roll Patient Register Data Customer Information System Missing includes: Migrants not (yet) registered Newborn babies Some private only patients Missing includes: Non higher education students Independent University students Missing includes: Non-drivers Under 17’s Some foreign-licence holders Missing includes: Some migrant worker dependants Some international students Undocumented asylum seekers Missing includes: Under 17s Ineligible voters Non responders Missing includes: Non school aged people Independent school children Home schooled children HESA DVLA DVLA ER CIS SC SC Resident Population PRD ER Extras includes: Some duplicates International students on short-term courses Students ceased studying, not formally deregistered Extras includes: Short-term migrant children Extras includes: Some duplicates Some ex-pats Some deceased Short-term migrants Extras includes: Some ex-pats Some deceased Extras includes: Multiple registrations Some ex-pats Some deceased Short-term migrants Extras includes: Some ex-pats Some deceased Short-term migrants PRD CIS

DWP CIS population counts compared with ONS Mid Year population estimates
(for illustration only)

Patient Register population counts compared with ONS Mid Year population estimates

Electoral Roll population counts compared with ONS Mid Year population estimates

Beyond 2011 Timeline – Plans and the Future

Beyond 2011 - Timeline (non census solution)
2015 2016 2017 2018 2019 2020 2021 2022 2023 population estimates characteristics outputs detailed design procure / develop develop / test 2011 2012 2013 2014 research / definition initiation 2024 address register required on an ongoing basis – ideally the National Address Gazetteer – subject to confirmation of quality public sector admin sources & commercial ? developing over time coverage surveys testing continuous assessment supplemented by new targeted surveys as required test attribute surveys info from existing surveys – e.g. labour force survey, integrated household survey etc linkage increasing linkage over time modelling increasing modelling over time

Beyond 2011 - and into the future
2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 regular production of population and attribute estimates ongoing methodology refinement address register required on an ongoing basis and be added & develop over time administrative sources will change and disappear continuous coverage survey need for attribute surveys declines over time ? existing surveys increasing linkage over time increasing modelling over time

Improving quality & quantity
2013 2031 2021 accuracy of population estimates experimental statistics develop to become national statistics accuracy of characteristics estimates range of topics small area detail multivariate small area detail

Statistical benefit profile
Benefit Alternative method Census loss loss gain gain

Cost profile (real terms)
Cost Census ??? Alternative method

Beyond 2011 Key risks and issues What could possibly go wrong?

Key risks of non census alternatives
Public opinion Public acceptability research, open consultation & transparency of approach Technical challenge – cutting edge & a big change Building high quality team, talking to experts and strong external assurance Changes in administrative datasets Building resilient solutions – but may need legislation Ensuring decision making – getting political buy-in This is a big decision – needs real commitment Devolved issues / harmonisation of UK outputs Establishment of UK Beyond 2011 Committee + NS / RGs agreement

Beyond 2011 First Consultation Nov 2011 – Jan 2012

Respondent Analysis

Questionnaire Information about organisation and their use of data
User requirements for data - census consultation as baseline Current requirements Future requirements Impact of not having data Key quality trade-offs frequency/geography/accuracy Comments User requirements Wales Consultation/Beyond 2011

Emerging Themes

Changes in user requirements
Key drivers of change: Public policy The economic downturn Other societal trends

Accuracy, Frequency or Geography?
“What gives ONS data credibility over other sources is its accuracy and reliability, therefore this should not be sacrificed” City of Bradford MDC

Main messages Many users welcome the opportunity – but many like the census The expected range of views and priorities - we’re not going to be able to please everybody Strong genealogical interest Views vary by sector but most prefer small area geography and accuracy to high frequency - Census as a benchmark for other sources

Beyond 2011 Next Steps

Next Steps Recent publications
- consultation, progress, options, research Ongoing dialogue with users and other stakeholders Workshops – Cardiff (12th),Manchester (26th), London(28th November) – LA event Birmingham (5th) & More in new year Key focus : research, criteria and costs/benefits Further reports & first assessment in February 2013 Public consultation - Late 2013

Questions / Comments ?

Beyond 2011 The options revisited (hot off the press)

Criteria? Cost (both total cost and cost profile over time) & benefit
Fitness for purpose i.e. ability to meet user requirements Accuracy of the statistics produced Frequency of the outputs e.g. updated annually, or every 5 or 10 years Geographic level at which outputs can be produced e.g. local authorities, output areas Consistency and comparability of the outputs across geographic areas Etc etc etc Technical & legal feasibility Risk Public burden and public acceptability. etc etc etc

Findings from dry run (incomplete assessment – for illustration only) Many criteria were not discriminatory – giving the same results for many (or all) options Many criteria could not be assessed – too early or too abstract – so ranges were large Options could be designed to deliver varying levels of quality (eg. accuracy of attribute data is largely the result of survey size / frequency) The same functions were being assessed several times

New way forward Instead of defining the options, then trying to assess quality... ...define quality standards (of outputs) for population estimates population attributes Assess and shortlist options that can meet them .... (then chose from them in terms of business case & cost/benefit).

Quality standards (draft)
Population estimates (annually at ...) P1. Maximum quality of Census P2. Current variable (peak and trough) quality P3. Current average quality P4. Minimum quality in current system, Population attributes (of acceptable quality at ...) A1. MSOA data every 10 years LA data every 10 years A2. LA data annually MSOA data rolling 3 years LSOA data rolling 5 years A3. OA data every 10 years (& LSOAs MSOAs LAs ) (At least !!)

Full Census (long form to everyone) Rolling Census (over 5/10 year period) Census options Short Form (everyone), Long form (Sample) Short Form + Annual Survey (US model) Aggregate analysis Administrative data options Partial linkage e.g. 1% of postcodes 100% linkage Survey option(s) Address register + Survey

Full Census (long form to everyone) Rolling Census (over 5/10 year period) Administrative data options Census options Survey option(s) Short Form (everyone), Long form (Sample) Short Form + Annual Survey (US model) Aggregate analysis Partial linkage e.g. 1% of postcodes 100% linkage Address register + Survey

Admin data:Part Linkage Admin data:Full Linkage
Estimating the POPULATION Characteristics - ATTRIBUTES Full Census Short form 100% every 10 years Long form 100% every 10 years Rolling Census (France) Short form - rolling ≈ 4%-10% every year Long form - rolling ≈ 4%-10% every year (???) Short form, Long form (Canada) Short form 100% every 10 years Long form ≈ 10%+ every 10 years Short form + Annual survey (USA) Short form 100% every 10 years Long form - rolling ≈ 4%-10% every year (???) Aggregate Admin data:Aggregate every year Large Survey rolling or every 10 years Partial linkage Admin data:Part Linkage every year Large Survey rolling or every 10 years Full linkage Admin data:Full Linkage every year Large Survey rolling or every 10 years Address register + Survey Large Survey ≈ 4%-10% every year (???)

Admin data:Part Linkage Admin data:Full Linkage
Estimating the POPULATION Characteristics - ATTRIBUTES Full Census Short form 100% every 10 years Long form 100% every 10 years Rolling Census (France) Short form - rolling ≈ 4%-10% every year Long form - rolling ≈ 4%-10% every year (???) Short form, Long form (Canada) Short form 100% every 10 years Long form ≈ 10%+ every 10 years Short form + Annual survey (USA) Short form 100% every 10 years Long form - rolling ≈ 4%-10% every year (???) Aggregate Admin data:Aggregate every year Long 10% every 10y Long ≈ 4%+ rolling Long 40% every 10y OR OR Partial linkage Admin data:Part Linkage every year Long 10% every 10y Long ≈ 4%+ rolling Long 40% every 10y OR OR Full linkage Admin data:Full Linkage every year Long 10% every 10y Long ≈ 4%+ rolling Long 40% every 10y OR OR Address register + Survey Short form - rolling ≈ 4%-10% every year Long form - rolling ≈ 4%-10% every year (???)

estimates of POPULATION characteristics - ATTRIBUTES
Short 100% every 10y estimates of POPULATION rolling ≈ 4%-10% Admin Part Linkage Aggre-gate Full Linkage Long 40% every 10y ≈ 4%+ rolling 100% ≈ 10%+ every 10y characteristics - ATTRIBUTES

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y characteristics - ATTRIBUTES Long ≈ 4%+ rolling Long 40% every 10y Long 100% every 10y  We could do a census We know it is capable of delivering at least one of our quality standards

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y characteristics - ATTRIBUTES Long ≈ 4%+ rolling Long 40% every 10y Long 100% every 10y  x x x x We wouldn’t do these as well A short form at the same time is the only sensible combination

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Aggregate options don’t deliver accurate enough statistics They are interesting and but fail all of our quality targets (A1, A2, A3) Long ≈ 10%+ every 10y x characteristics - ATTRIBUTES Long ≈ 4%+ rolling Long 40% every 10y Long 100% every 10y  x x x x

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y x ? characteristics - ATTRIBUTES Long ≈ 4%+ rolling x Long 40% every 10y x Our current view is partial linkage is not good enough either. This will be further tested but currently considered unlikely to be an option. Long 100% every 10y  x x x x

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y x ? characteristics - ATTRIBUTES Long ≈ 4%+ rolling  x ? We could do a rolling census Evidence is that this would deliver at least one of our quality targets Long 40% every 10y x ? Long 100% every 10y  x x x x

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y x x ? But these options don’t make sense If we are running a rolling survey we might as well do the attribute survey at the same time characteristics - ATTRIBUTES Long ≈ 4%+ rolling  x ? Long 40% every 10y x ? Long 100% every 10y  x x x x

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y x x ? characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ? Short form every 10 years, Rolling survey The American model – does look like it would satisfy one of our quality targets – so is an option Long 40% every 10y x x ? Long 100% every 10y  x x x x

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y  x x ? Long form / Short form census An attribute survey of only 10% every 10 years Will deliver our minimum quality standard – an option – but probably a weak one (probably only minimum savings compared to a full census) characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ? Long 40% every 10y  x x ? Another Long form / Short form census This time an attribute survey of 40% of the population. Looks like it would satisfy our target quality (but not much saving over full census) – a possible option Long 100% every 10y  x x x x

Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y  x x ?  characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ? Long 40% every 10y  x x ? Full linkage of admin data We believe this can deliver quality for population estimates – the size of the attribute survey depends on proving the value of the small area data. Long 100% every 10y  x x x x

Which option to choose ? We know that these can produce good estimates We have yet to prove that these can estimates of POPULATION Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y  x x ?  characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ?  Long 40% every 10y  x x ?  Long 100% every 10y  x x x x No unit level record linkage Require us to link unit level data

estimates of POPULATION
Which option to choose ? We know that these can produce good estimates We have yet to prove that these can estimates of POPULATION Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Small area attribute data COST / BENEFIT of Long ≈ 10%+ every 10y  x x ?  characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ?  Long 40% every 10y  x x ?  Long 100% every 10y  x x x x No unit level record linkage Require us to link unit level data If we cannot prove that admin data works & is publically acceptable – we have to stick with a variation upon the Census Which option depends on the cost benefit case for the small area data

estimates of POPULATION
Which option to choose ? We know that these can produce good estimates We have yet to prove that these can estimates of POPULATION Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Small area attribute data COST / BENEFIT of Long ≈ 10%+ every 10y  x x ?  characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ?  Long 40% every 10y  x x ?  Long 100% every 10y  x x x x No unit level record linkage Require us to link unit level data If we can prove that admin data works & is publically acceptable – an admin data based solution looks very attractive The size and frequency of the attribute survey will depend on the quantified benefits of small area data

Which option to choose ? estimates of POPULATION Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y  x x ?  characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ?  Long 40% every 10y  x x ?  Long 100% every 10y  x x x x Admin Linkage + x

 x x ?    x ?   x x ?   x x x x Which option to choose ?
estimates of POPULATION Short 100% every 10y Short rolling ≈ 4%-10% Admin Aggre-gate Admin Part Linkage Admin Full Linkage Long ≈ 10%+ every 10y  x x ?  characteristics - ATTRIBUTES Long ≈ 4%+ rolling   x ?  Long 40% every 10y  x x ?  Long 100% every 10y  x x x x Can we produce good population estimates from administrative data ? What is the public perception of data linkage vs. census methods ? How strong is the case for detailed population attribute data ?

Admin data:Part Linkage
Estimating the POPULATION Characteristics - ATTRIBUTES Full Census Short form 100% every 10 years Long form 100% every 10 years Rolling Census (France) Short form - rolling ≈ 4%-10% every year Long form - rolling ≈ 4%-10% every year (???) Short form 100% every 10 years Long form ≈ 10%+ every 10 years Short form, Long form (Canada) Short form + Annual survey (USA) Short form 100% every 10 years Long form - rolling ≈ 4%-10% every year (???) Long 10% every 10y Admin data:Aggregate every year Aggregate OR ≈ 4%+ rolling 40% Admin data:Part Linkage every year Partial linkage Long 10% every 10y OR ≈ 4%+ rolling 40% Full linkage Admin data:Full Linkage every year Long 10% every 10y Long ≈ 4%+ rolling Long 40% every 10y OR OR Address register + Survey Short form - rolling ≈ 4%-10% every year Long form - rolling ≈ 4%-10% every year (???)

Questions / Comments ?

User Workshop Manchester 26th November 2012 Beyond 2011

Similar presentations

Presentation on theme: "User Workshop Manchester 26th November 2012 Beyond 2011"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

User Workshop Manchester 26th November 2012 Beyond 2011

Similar presentations

Presentation on theme: "User Workshop Manchester 26th November 2012 Beyond 2011"— Presentation transcript:

Similar presentations

About project

Feedback