1. Slide 1 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Agenda Item 8 Preliminary ideas for a 2011 census hub Giuseppe SINDONI Meeting of the CoRD Task Force “Collection of raw data” 4 June 2007

2. Slide 2 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Motivations Proposal for a regulation on population and housing censuses (Feb 2007) –Harmonisation of censuses –Output oriented –Reduction of burden on MS –Subsidiarity principle –Adoption of the appropriate technical format Need to collect and disseminate aggregates

3. Slide 3 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Data requirements Data on different topics –Population and Housing Data at different levels of detail –Hierarchical classifications (age groups, occupation sectors, …) –NUTS 1 to 3 Confidentiality issues –Secondary disclosure control –Different rules in different countries

4. Slide 4 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Organisational requirements Data to be transmitted in 27 months from the reference year Data may be transmitted by countries as soon as they are available Countries responsible for quality of their own data Eurostat to collect and disseminate

5. Slide 5 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Technical requirements A central dissemination site for country data Need to have a common transmission format Infrastructure to –Collect data asynchronously –Perform validation (which kind?) –Publish data on the Web

6. Slide 6 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Possible scenarios 1.Eurostat collects anonymised microdata and sets up a web- enabled data warehouse (ruled out) confidentiality issues aggregation issues maintenance issues 2.Eurostat collects and publishes predefined aggregate tables not very flexible for final users 3.Eurostat collects and publishes “safe” hypercubes transmission issues maintenance issues

7. Slide 7 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 The hub approach Data sharing –a group of partners agree on providing access to their data according to standard processes, formats and technologies. Data hub –a web-accessible system providing involved actors with the following functions: data providers can –notify the hub availability of new sets of data and corresponding metadata (measures, dimensions, code lists, etc.) –make data available directly from their systems through a querying system data users can –browse the hub to define a dataset of interest via the above structural metadata –retrieve the dataset from the interested NSIs.

8. Slide 8 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Notification Census hub MS n MS 2MS 1 … Notify availability of hypercubes Updates metadata

9. Slide 9 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 User interaction Census hub MS n MS 2MS 1 … Requests a dataset

10. Slide 10 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Querying Census hub MS n MS 2MS 1 … Query sent to MSs WSs

11. Slide 11 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Replies Census hub MS n MS 2MS 1 … Query processed and results sent

12. Slide 12 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Result presented Census hub MS n MS 2MS 1 … Dataset presented Replies put together

13. Slide 13 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Technological standards SDMX –XML format (SDMX-ML) hypercube structures and data files query reference metadata provision agreement –SDMX registry architecture back-end metadata repository –SDMX WS architecture interaction between systems Web feeds (RSS, Atom)

14. Slide 14 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Roadmap 1.2007Q3: selection of two or three volunteer NSIs for a pilot phase 2.2007Q4: requirement specifications for the pilot phase 3.2008Q2: pilot phase end Evaluation of pilot phase 1.2008Q4: start of hub project 2.2010Q3: hub ready for production

15. Slide 15 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Reference bodies European Commission (advised by SPC) –presents regulation EP and Council −adopts regulation Census TF (Eurostat + MSs) −technical work to implement regulation

16. Slide 16 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Issues Technological readiness of MSs –survey carried out in 2007Q1: all respondent countries (16) will most likely have a DW for Census data dissemination Organisational readiness of MSs –more than one department involved (Census, IT, Integration, …) Technical issues –performances: some data caching system could help improve response times to final users XML compression techniques to enable transmission of large hypercubes …