1. The bottom-up approach: Challenges in the production of statistical grid data
Rina Tammisto
European Forum for Geostatistics, Workshop 1- 3 October 2008 Bled, Slovenia

2. Challenges in... Production environment Production methods Quality

3. …from the point of view of Finland
... and relating to...
Spatiality of grids
Spatiality of statistical data
Metadata descriptions of geostatistical data
Use, instructions and distribution
…from the point of view of Finland
Georeferenced data and the production of grid data in Finland

4. 5. 6. 1. 2. 3. 4.

5. 1. Challenges relating to spatiality of grid data - handling the grid net
Co-ordinate systems, projections
Standards (national/international)
National practices
Grid cell sizes
User needs
Standards
Scalability!
Practical challenge:
Enormous file sizes, difficult to handle

6. Grid net... Essential qualifications of a grid net:
A grid net should be spatially unchanged over time.
The spatial characters should be equal to spatiality in the statistical data.
Regional classifications in a grid net:
Choice of the substantial classifications of all classifications (user needs!) .
Annual (or other) updating of regional classifications.

7. 2. Challenges relating to the spatiality of statistical data
The spatial attributes of statistical data should be equal to spatiality in the grid net.
The precision of the spatial information of statistical data.
Appropriate workflows handling the spatial information of data from preparation to storing phase.
Potential workflows in producing grid data from stored data:
From points to grids, always starting from the beginning.
Use of aggregated data
semi-finished data in the databases
ready-made grid data sets.

8. Statistical data... Scalability
Data protection directs the choice of scale (enough data for purpose of use).
User needs vs. data protection
conflict between the user needs and data protection
scalable threshold values (?)

9. 3. Challenges relating to metadata descriptions of geostatistical data
Metadata standards
Spatial data (ISO 19115)
Statistical data - ?
Combining these two - ?
Lack of specifications for describing geostatistical information
GeoSTAT-project, subproject 3!

10. 4. Challenges relating to use, instructions and distribution
In Finland, data sets are often expensive - depends on the value of grid data for the customer.
Fewer work phases might curb production costs.
Grid data are abstract. Flexible possibilities to combine them into other regions.
USER NEEDS vary a lot.
Direct use from servers (WFS, WMS)!
Still, a GIS application is needed for use (mostly).
Easy-to-use applications!

11. 5. Georeferenced data and the production of grid data in Finland
Data from registers:
Population register, Register of buildings and dwellings
Use of geodatabase (MS SQL Server + ArcSDE)
Use of data warehouse (MS SQL Server )
Buildings at the core
Grid data production:
Grid net updated annually
Intensive use of SQL, SAS, ESRI ArcGis

12. SHP (personal GDB, XML file…)
Geodatabase
Data warehouse
Selection, aggregation
Deduction
X,Y ID
Grid net 250 m
Database tables
aggregated tables
(250 m)
Joining
Creating a view
- selection
- data security
Spatial database view
Export
SHP (personal GDB, XML file…)

13. Challenges at Statistics Finland - production
Many applications involved (SQL, SAS, AG) in grid data production
Which of the production workflows is the most qualified?
How to correctly split the workflow between the applications in order to:
avoid errors (data at this level is so precise that even the tiniest errors are visible) - quality targets!
make production as effective as possible in every way
make production as homogenous as possible (repeatable)?

14. 6. Quality standard for spatial data (ISO 19113)
Quantative quality elements:
Completeness
Logical consistency
Positional accuracy
Temporal accuracy
Thematic accuracy - accuracy of descriptive attribute data
Non-quantative quality elements:
Purpose
Use
Lineage
history of a dataset

15. Defining the quality of statistical grid data
Quality from the point of view of users
Fitness to purposes of use
data content
spatial focusing of data
accuracy of data is sufficient for the use purposes
Quality from the point of view of producers
Quality=Data are produced “as well as possible” from the point of view of data qualifications
Quality =Data are produced strictly according to existing standards
Quality = Data are produced according to the defined target state

16. QUALITY Guide, define TARGET STATE adjust USER REQUIREMENTS
STANDARDS
LEGAL REGULATIONS AND RESTRICTIONS
TARGET STATE
adjust
The definition and description of the target state
USER REQUIREMENTS
guide, describe
Several parallel requirements:
-> intended use of data
QUALITY
instruct, describe
USE
PRODUCTION
Descriptions of data
Instructions for use
Description of the method

17. Finally.. Our forthcoming project should give us
a STRUCTURE for handling this never completed process.

18. How to respond to the challenges?
User needs define quality.
User needs are in focus when setting targets.
Standardized production
Predefined minimum standards for quality (beyond the changes in production environment or methods).
Instructions for use and descriptions of the purpose of use.
An effective data delivery and use possibilities (Google Map, Google Earth,…).

19. Thank You for Your attention!