1. Statistics Estonia's new system for statistical data activity processing (VAIS) ITDG Luxembourg 2010 Allan Randlepp

2. What it is? VAIS (pronounced as wise) is a collection of tools and technologies aimed at automating data processing (Phase 5 in GSBPM). In essence, the task of check, clean, and transforming statistical activity data can be identified as taking the raw data from one or more sources and transforming it to analytical system source data input data base structures. Statistics Estonia's new system for statistical data activity processing (VAIS) 19.10.2010

3. Framework for … Integrate data Cassify & code Review, validate and edit Impute Derive new variables & statistical units Calculate weights Calculate aggregates Finalize data files Statistics Estonia's new system for statistical data activity processing (VAIS) 19.10.2010

4. Metadata driven template based tool Template driven approach provides an universal solution for three main goals of the VAIS project: Create an easy to use statistical data processing tool requiring minimal programming skills for transformation package creation. Create a metadata driven process-oriented and automated statistical data processing tool. Create an extendable data transformation tool. Statistics Estonia's new system for statistical data activity processing (VAIS) 19.10.2010

5. Data processing with VAIS Statistics Estonia's new system for statistical data activity processing (VAIS) 19.10.2010

6. Balancing automation and manual intervention Statistics Estonia's new system for statistical data activity processing (VAIS) 19.10.2010 RAW data Metadata (validation and transformation rules) Automated data processing Manual data processing OK? Input data

7. Technical platform VAIS is built on open-sourced freely available technological components. XDTL (eXtensible Data Transformation Language – an XML based descriptional language designed for specifying data transformations, see http://xdtl.org) run-time engine (XDTL RT).http://xdtl.org MMX Metadata Repository, part of Metadata Framework (a MOF compliant metadata management environment designed with a wide variety of metadata-driven applications in mind, see http://mmframework.org). http://mmframework.org Apache Foundation's Velocity template engine (http://velocity.apache.org) is used as the template engine combining excellent template rendering functionality with very easy to use template language.http://velocity.apache.org The user applications are programmed in Java, based on Wicket MVC framework (http://wicket.apache.org)http://wicket.apache.org Quartz scheduling framework (http://www.quartz-scheduler.org) is used for execution scheduling.http://www.quartz-scheduler.org Statistics Estonia's new system for statistical data activity processing (VAIS) 19.10.2010