1. Census Hub in practice
Working Group "European Statistical Data Support"
Luxembourg, 29 April 2015

2. Contents The Census Hub: motivation and opportunities
Broad overview of the census data collection
The Graphic User Interface of the Census Hub
Feedback received from users

3. The EU programme for the 2011 census
Massive effort of data harmonization: first time that population and housing censuses are regulated at this level of detail
High level of geographical data detail
Possibility to cross-tabulate several social variables  focus on specific subgroups of the resident population
Huge volume of data involved  decentralized database

4. The Census Hub The Census Hub: an innovative dissemination tool
A distributed data warehouse based on agreed data structures
Data are physically resident in Member States and retrieved dynamically upon user request

5. The Census Hub
Central and national nodes can automatically exchange data among each other
Solution that can be reused (either for next census or for other data collections)
First full implementation of SDMX in the ESS

6. The Census Hub System designed to treat multiple data accesses
National Statistical Institute
SDMX-RI
National Statistical Institute
Eurostat Census
Hub
System designed to treat multiple data accesses
Peak of 1,000 users/hour
Very good performance level with fast response time
A data hub is a pull mode based architecture for common data sharing. Data is not previously collected and stored in a central repository but it is [A>] directly accessed from the data providers’ databases through a central hub upon [B> ]request of a data collector:[/A][/B]
[C>] The data collector browses the Hub to define the dataset of interest via its structural metadata.
The Hub converts the user’s request into an SDMX Query message and [D>] sends it to an National Statistical Institute’s Web Service. [/C] [/D]
[E>}The NSI Web Service converts the SDMX Query in a set of SQL queries, fetches the data from the NSI database, dynamically constructs the SDMX-ML file and [F>] sends it back to the hub.
If the request concerns data of several NSIs, the steps are executed simultaneously for each of them. [/E][/F]
[G>] The Hub assembles all the SDMX-ML files received from the NSIs and presents the result to the user in a readable format.[/G]
The workflow is:
Step 1: a “data user” browses the Hub to define a dataset of interest via structural metadata. He browses the dimensions and selects a dataset. Then he chooses the organization of the output layout specifying which dimension wll match X-axis and Y-axis and which dimension will vary item after item to generate new tables
Step 2: The Hub converts the user request into an SDMX Query and sends the SDMX Query to an interested NSI Web Service
Step 4: The NSI Web Service converts the SDMX Query in a set of SQL queries and sends them to the NSI data warehouse
Step 5: The NSI data warehouse sends the result to the NSI web service
Step 6: The NSI Web Service converts the result in a SDMX-ML Data message and sends it to the Hub
Step 7: The same steps are repeated if the user has requested data from different MSs
Step 8: the Hub puts together all the SDMX-ML data messages proceeding from the interested NSIs and presents the result to the “data user” in the web browser in readable format.
SDMX-RI 6 6 6

7. Topics covered by the census
Location
Usual residence
Place of work
Demographic characteristics
Sex
Age
Data on persons
Family & household characteristics
(Legal) marital status
Family status
Household status
Employment and education characteristics
Current activity status
Industry
Occupation
Status in employment
Educational attainment
Migration and mobility
Place of birth
Country of citizenship
Year of arrival
Residence one year before
List of topics collected on persons
Housing arrangements

8. Census datasets (hypercubes)
Distribution of census hypercubes by type of statistical unit
Distribution by type of statistical unit

9. Census datasets (hypercubes)
Distribution of census hypercubes by level of geographical detail
Distribution by level of geographical detail

10. Census datasets (hypercubes)
Distribution of census hypercubes by number of dimensions
Distribution by number of dimensions

11. Data selection in Eurostat database
Overview of the data selection process in Eurobase

12. Difficulty to implement "traditional" data selection with census hypercubes
Very high level of dimensionality: impossibility to find meaningful names to datasets
Two dataset might have 6 dimensions in common and differ only for the remaining 2
The same dataset can cover several domains (e.g. economic status, migration background, etc.)  difficulty to build a meaningful "data tree"

13. Data selection in the Census Hub
The data selection process in the Census Hub GUI

14. Data selection in the Census Hub
The data selection process in the Census Hub GUI

15. Data selection in the Census Hub
The data selection process in the Census Hub GUI

16. Data selection in the Census Hub
The data selection process in the Census Hub GUI

17. Data selection in the Census Hub
The data selection process in the Census Hub GUI

18. Data selection in the Census Hub18

19. Feedback from users
The data selection mechanism implemented in the Census Hub is very user-friendly
Data selection is completely user-oriented and not producer-oriented
Codelists and available breakdowns are immediately visible to users
The system is well performing, despite the high volume of data involved
 response time similar to Eurostat database

20. Thanks for your attention