Automated Access to 100.000.000 Statistical Facts via Statline4 Web Services Olav ten Bosch Statistics Netherlands UN-ECE conference, Bratislava April.

Slides:

Advertisements

Similar presentations

National Institute of Statistics, Geography and Informatics (INEGI) Implementation of SDMX in Mexico.

Advertisements

17th February, 2000 by Maciej Korzeniowski (CERN-IT-IA-MI) 1 Oracle Discoverer Product Presentation  This is an ad hoc query and analysis tool for.

Slide 1 Eurostat Unit B3 – Statistical Information Technologies MSIS 2005 SDMX Open Data Interchange (SODI) by Leonhard Maqua and Georges Pongas Eurostat,

Stefania Bergamasco, Cecilia Colasanti An integrated approach to turn statistics into knowledge combining data warehouse, controlled vocabularies and advanced.

Looking Ahead Archive-It Partner Meeting November 12, 2013.

MEDIN Standards M. Charlesworth and the MEDIN Standards Working Group.

1 Introduction to XML. XML eXtensible implies that users define tag content Markup implies it is a coded document Language implies it is a metalanguage.

ITEC810 Project By: P. M. Mathindri Nilushika Pathiraja 1.

A Topic Specific Web Crawler and WIE*: An Automatic Web Information Extraction Technique using HPS Algorithm Dongwon Lee Database Systems Lab.

ReQuest (Validating Semantic Searches) Norman Piedade de Noronha 16 th July, 2004.

Slide 1 Eurostat Directorate B – Statistical methods and tools; dissemination Towards implementation of SDMX – 9/11 January 2007 SDMX Open Data Interchange.

Databases & Data Warehouses Chapter 3 Database Processing.

Linked Data Visualizations for Eurostat Linked Data Dr. Brand Niemann Director and Senior Data Scientist Semantic Community

Sys Prog & Scripting - HW Univ1 Systems Programming & Scripting Lecture 15: PHP Introduction.

Agenda Item 3.3 SDMX reference architecture for NSIs Francesco Rizzo 24 th Meeting of the STNE Working Group “Statistics, Telematic Network & EDI”

The implementation of the SDMX standards by the ECB and the European System of Central Banks Werner Bier (ECB) Gérard Salou (ECB) Sami Airo (Bank.

Augmenting search using a semantic visual graph Edwin de Jonge Olav ten Bosch Statistics Netherlands.

Classroom User Training June 29, 2005 Presented by:

Geolinking content Patrick H. Lauke / Institutional Web Management Workshop 2007 / York Experiments in connecting virtual and physical places.

Using XML technologies to implement complex tables in short- term statistics Francesco Rizzo

M i SMob i S Mob i Store - Mobile i nternet File Storage Platform Chetna Kaur.

Presenting Statistical Data Using XML Office for National Statistics, United Kingdom Rob Hawkins, Application Development.

CountryData Development Improving the collation, availability and dissemination of development indicators (including the MDGs) Nairobi, 27 November 2013.

Restricted Daejeon, April An SDMX based unified data catalogue (UDC) MSIS – Meeting on the Management of Statistical Information Systems 1.

HOW WEB SERVER WORKS? By- PUSHPENDU MONDAL RAJAT CHAUHAN RAHUL YADAV RANJIT MEENA RAHUL TYAGI.

Open access & visibility Management Digital Preservation ORA: Purposes.

Design of a Search Engine for Metadata Search Based on Metalogy Ing-Xiang Chen, Che-Min Chen,and Cheng-Zen Yang Dept. of Computer Engineering and Science.

McLean HIGHER COMPUTER NETWORKING Lesson 7 Search engines Description of search engine methods.

Francesco Rizzo (ISTAT - Italy) SDMX ISTAT FRAMEWORK GENEVE May 2007 OECD SDMX Expert Group.

IS 325 Notes for Wednesday August 28, Data is the Core of the Enterprise.

Implementation Experiences METIS – April 2006 Russell Penlington & Lars Thygesen - OECD v 1.0.

United Nations Economic Commission for Europe Statistical Division The Importance of Databases in the Dissemination Process Steven Vale, UNECE.

Database Systems: Design, Implementation, and Management Eighth Edition Chapter 14 Database Connectivity and Web Technologies.

1 Exchange of data between OECD and Statistics Norway Marianne Vik Dysterud, Statistics Norway Meeting of the OECD Expert Group on Statistical.

Slide 1 Eurostat Unit B3 – Statistical Information Technologies CoRD Meeting – 4 June 2007 Agenda Item 8 Preliminary ideas for a 2011 census hub Giuseppe.

The CERA2 Data Base Data input – Data output Hans Luthardt Model & Data/MPI-M, Hamburg Services and Facilities of DKRZ and Model & Data Hamburg,

1 3 Computing System Fundamentals 3.4 Networked Computer Systems.

Mercury – A Service Oriented Web-based system for finding and retrieving Biogeochemical, Ecological and other land- based data National Aeronautics and.

Olav ten Bosch and Edwin de Jonge Statistics Netherlands UNECE - Meeting on the Management of Statistical Information Systems (MSIS) Luxembourg, 7-9 April,

Eurostat SDMX and Global Standardisation Marco Pellegrino Eurostat, Statistical Office of the European Union Bangkok,

Seminar on Dynamic Graphics for presenting Statistical Indicators 5-6 March 2007, Rome Eurostat approach to graphical representation of statistical data.

From institutional repositories to personal collections of learning resources Julià Minguillón 1,2, Jordi Conesa 1 1 Computer Science, Multimedia and Telecommunication.

Eurostat November 2015 Eurostat Unit B3 – IT and standards for data and metadata exchange Jean-Francois LEBLANC Christian SEBASTIAN SDMX IT Tools SDMX.

CENSUS OUTPUTS Dissemination Plans Chris Ashford 2011 Census Outputs : Technical Delivery.

Jens Hartmann York Sure Raphael Volz Rudi Studer The OntoWeb Portal.

OECD Expert Group on Statistical Data and Metadata Exchange (Geneva, May 2007) Update on technical standards, guidelines and tools Metadata Common.

Dissemination Statline tool and organisation André de Boer.

Institute for the Protection and Security of the Citizen HAZAS – Hazard Assessment ECCAIRS Technical Course Provided by the Joint Research Centre - Ispra.

Slide 1 Eurostat Unit B6 – Dissemination IT Directors Group 24 – 25 October 2005 Cooperation with Member States on Internet Gunter Schäfer Eurostat B6:

Web Design Terminology Unit 2 STEM. 1. Accessibility – a web page or site that address the users limitations or disabilities 2. Active server page (ASP)

ETERE NUNZIO The ultimate end-to-end solution for your NewsRoom.

Eurostat May 2016 Eurostat, Unit B3 – IT solutions for statistical production Test Client Jean-Francois LEBLANC Christian SEBASTIAN.

1 Copyright © 2008, Oracle. All rights reserved. Repository Basics.

Structural and reference metadata in the European Statistical System

Web Engineering.

Progress Update MSIS: Bratislava, April 2005

Web scraping tools, an introduction

Eurostat internet tools Point 17: DWG 15-16th April 2010

What is a Search Engine EIT, Author Gay Robertson, 2017.

International Marketing and Output Database Conference 2005

Eurostat Internet tools Unit D4

Report on Dissemination B.Le Goff Unit D4

Dissemination of statistics … on the web

Prepared by Peter Boško, Luxembourg June 2012

Query Interface using Django

Dynamically Updated Publications

Annegrete Wulff Statistics Denmark

Standardizing and industrializing a business process – the dissemination use case Alessio Cardacino - ESTP Course “Information standards.

SDMX IT Tools SDMX Registry

Presentation transcript:

Automated Access to Statistical Facts via Statline4 Web Services Olav ten Bosch Statistics Netherlands UN-ECE conference, Bratislava April 2005

Contents - StatLine in a Nutshell - Automated Access to Statistics: Why? - Design of StatLine4 Web Services - The SN SODI Pilot - Conclusions

StatLine in a Nutshell (1) - StatLine: -a statistical output database accessible via Internet -about 100 Million statistical facts -Multidimensional cubes with hierarchical dimensions organized by theme -Search facility -Tables, charts and maps -DUAL = refer to any table, map or chart in StatLine within one URL -(example: give the 10 most recent statistical facts on subject x)

StatLine in a Nutshell (2)

StatLine in a Nutshell (3) StatLine4: -Mechanisms for standardization an coordination of metadata -Facilities to handle changes in metadata. Multiple versions of statistical facts -complete redesign in.NET -“smart” user interface -Automated access through Web Services

Automated Access Why? (1) - The Web evolves into a loosely coupled web of data repositories: statistics should be part of that….. - We see clients linking deeply into our statistical database (deeplinking): we should make this easy….. - Clients tend to query for certain specific statistical figures regularly: we should help them retrieve this content automatically….. - NSI´s and others institutes should be able to interchange statistical data automatically…..

StatLine4 Web Services (1) Browsers html, DUAL StatLine4 100M. Facts Clients, NSI’s Data stores Automated access XML Web Services: –Third parties may automatically connect to the latest statistical data via the web –Service based on Industry standards & proven technology –Web service is self-describing: “what can I get here and how should I ask it” ASPX

StatLine4 Web Services (2) - Search Web Services -Inspired by Google API and uses Lucene -Integrate StatLine search results into your own web page or application ( - Update Web Services -RSS feeds on three levels: - Statistical Database as a whole (100 M. facts) - Within a specific Theme - Within a specific Cube - Data Web Services -Facility to actually retrieve specific data sets -or just one statistical fact -Backwards compatible with current hyperlink mechanism (DUAL)

The SN SODI Pilot (1) - EuroStat SODI (SDMX Open Data Interchange) Task Force - Pilot for: - Quarterly GDP ( Quarterly Gross Domestic Product) - Monthly IPI (Industrial Production Index) - SN added dedicated Web Services: -Return results in SDMX-ML format -RSS feed for updates -Technical: yet another interface to StatLine

The SN SODI Pilot (2) SODI front-end EuroStat apps … SDMX-ML, rss GDP IPI Browsers Clients, NSI’s Data stores Automated access html, DUAL ASPX

The SN SODI Pilot (3)

The SN SODI Pilot (4)

The SN SODI Pilot (5)

Conclusions Designing a Statistical Web Service: -Start simple; use examples -Keep Web Services efficient and effective -Support services on a detailed and a general level (example: information on updates) SL4 approach versus SODI approach: -general web services (all statistics) versus dedicated web services (web service per domain) -Easy general access versus complete and exact communication -Both are necessary Web Service technology does not solve the metadata matching problem: -Better metadata standards or semantic techniques

Further Reading - Test page StatLine4 - -SODI page of Statistics Netherlands: - -SDMX page: -