1. Technical Coordination Group, Zagreb, Croatia, 26 January 2018
IT and technological solutions in different types of censuses in the region and their requirements Danilo Dolenc, MSc, Census Project Manager Statistical Office of the Republic of Slovenia
Technical Coordination Group, Zagreb, Croatia, 26 January 2018

2. Introduction Focus on Data collection methods
Data integration and principles of data editing
Traceability
Repeatability
Stage processing
Data warehouse

3. Paper vs. electronic Paper is out! Electronic data collection
Built-in data control – how far we can go?
To write, to mark, use of drop-down (pull-down) menus
Responsive design
Written answers
Taking into account all languages

4. Web vs. field Management information system
Coverage
Avoiding double enumeration
Coordination
More modes, more complexity
Security of data transmission
Online or offline

5. Identifiers
Common identifiers to link data from different input sources
Personal (PIN, tax number,…)
Geo-data (coordinates, addresses)
Linkage population and housing data
Dwelling numbers
Household numbers
Barcodes
Other specially designed unique codes

6. Data integration Two IT environments should be set up
Productional one for data integration and data editing
Final data warehouse
Data partly from productional part
Derived variables
Never ask what is possible to derive in the process
The most typical – family data

7. Principles of IT data production (1)
Repeatability
Change of any value at any variable in the record
New version of the record is created and inserted into database
Versions of the record properly denoted (sequel number)

8. Principles of IT data production (2)
Repeatability
You can start / repeat the statistical process from beginning or from some stage
Status 1 means input data after data integration but before data editing
Number of statuses depends on complexity of statistical editing
In 2015 the highest status was 18

9. Principles of IT data production (3)
Traceability
Metadata on method of change of value of every variable
Which productional phase
In combination with status of record
How the record has been changed
Automated correction
Manual correction
Imputation

10. Principles of IT data production (4)
IT process in stages
Important for register-based approach
Could be applied in any other census type
Not all sources available at the same time
No more changes of data after every stage
Final data could be disseminated much earlier
In Slovenia four stages process
Basic population data (T + 4 months)
Households, families (T + 9)
Socioeconomic characteristics + migration (T + 12)
Housing data (T + 18)

11. Productional environment (1)
Basic census tables (NAME)
Scheme of table – description of variables
Short name, format, length, classification
Auxiliary (‘‘shadow‘‘) tables for every basic table
Editable tables (NAME_EDI)
Status tables (NAME_EDI_S)

12. Productional environment (2)
Classifications (D_...) and codebooks (SIF_...) for every applicable variable
Primary data sources for data integration (VIR_...)
Variables with same name as basic table
Data in defined format
Secondary data sources (VIR_S_...)
Re-loaded new versions of data
Some variables only or limited number of records

13. Productional environment (3)
Views on tables – important for regular monitoring of data
All records (V_NAME_VSI)
6,4 million records in population table
Records with highest status only (V_NAME_ZST)
2,1 million records in population table

14. General advantages
Input data and final data available in the same table
Easy to calculate quality indicators
Controlled automated data editing in whole database
No partial processing
Possibility to apply manual editing if needed
Data from field should be coded in advance
No text data in basic census tables

15. Data warehouse (1)
At least one table per basic enumeration unit should be set up
Unique identifiers (not linkable to other identifiers) needed
Household ID
Family ID
Dwelling (building) ID
New - household table and family table
Not needed in productional environment

16. Data warehouse (2) Main advantages
More than one census data in warehouse
Longitudinal analyses
Most of topics same from census to census
Derived tables / variables allow easier tabulation and effective user support
Access to final warehouse by using standardized or self-developed tools
OLAP, Discoverer, SAS, Excel,…

17. Some examples - Slovenia
Table PERSONS
Only productional variables – 35
Only warehouse variables – 45
Productional variables directly transferred to warehouse table – 86
Derived table HOUSEHOLDS
86 derived variables
16 associated dwelling variables
16 territorial variables

18. Conclusion Every country must take into account their own situation
Model presented could be applied for every census type
Outsourcing of IT is feasible for data collection stage
Data processing should be developed in close cooperation by own IT and methodological staff