Statistics 2 go (simplePXsql, PX2go and API) PC-Axis reference group, Tirana 2013Lars Pedersen.

Slides:



Advertisements
Similar presentations
Easily retrieve data from the Baan database
Advertisements

Tuning in Action Dennis Shasha. Application Context Each record, when inserted, undergoes several tests to see where and how it should be inserted. Programmer.
Demo Overview Opening HANA Admin Accessing HANA Loading data to HANA with Data Services Exploring HANA contentExploring HANA Results Creating New HANA.
London & Zurich Plc User Guide. Service Benefits Full on-line management of client accounts Paperless direct debit – no signatures required Standing orders.
User Guide. Service Benefits  Full on-line management of client accounts  Paperless direct debit – no signatures required  Standing orders fixed not.
1 1 StatBank Moldova - The newest member of the PC-Axis Family Mr. Per Olav Løvbak Head of Division, Statistics Norway International seminar on dissemination.
News in PX-Edit Copenhagen, 2 October Kööpenhamina PX-Edit: The Mission PX-Edit is an expert tool for specialists.
Query Manager. QM is a collection of tools you can use to obtain information from the AS/400 database Used to –select, arrange, and analyze information.
Today: Run SAS programs on Saturn (UNIX tutorial) Runs SAS programs on the PC.
Data Migration Process
Serial benchmarks in Mysql, Oracle and PostgreSQL -Test objectives -Test setup -Test result and highlights Carlos Jesus
HORIZONT 1 TWS/Audit The Audit System for TWS z/OS HORIZONT Software for Datacenters Garmischer Str. 8 D München Tel ++49(0)89 /
AGENDA Tools used in SQL Server 2000 Graphical BOL Enterprise Manager Service Manager CLI Query Analyzer OSQL BCP.
1 Chapter Overview Transferring and Transforming Data Introducing Microsoft Data Transformation Services (DTS) Transferring and Transforming Data with.
Copying, Managing, and Transforming Data With DTS.
Overview of Mini-Edit and other Tools Access DB Oracle DB You Need to Send Entries From Your Std To the Registry You Need to Get Back Updated Entries From.
PI Data Access via OLE DB/SQL
ApexSQL Log Recover accidentally truncated, dropped or deleted data with or without transaction log or its backup. Recover data from un-attachable/corrupted.
CFT Offline Monitoring Michael Friedman. Contents Procedure  About the executable  Notes on how to run Results  What output there is and how to access.
International Conference on Agricultural Statistics (ICAS III) Cancun, Mexico 1- 3 November 2004 Topic: Methodology and New Techniques for Statistical.
The CountrySTAT Philippines Maura S. Lizarondo Asst. Director Jing B. Jalisan Webmaster Bureau of Agricultural Statistics Department of Agriculture Philippines.
SQLCMD – A Better DBACCESS? Utility by Jonathan Leffler Presented by Joe Kennedy ( )
OracleAS Reports Services. Problem Statement To simplify the process of managing, creating and execution of Oracle Reports.
1 Country report 2014 – Statistics Norway PC-Axis Reference Group meeting
Publishing Metadata with Data - XML based dissemination process of statistical information (CoSSI) Harri Lehtinen
Informix IDS Administration with the New Server Studio 4.0 By Lester Knutsen My experience with the beta of Server Studio and the new Informix database.
ABC Insurance Co. Paul Barry Steve Randolph Jing Zhou CSC8490 Database Systems & File Management Dr. Goelman Villanova University August 2, 2004.
Architecture Rajesh. Components of Database Engine.
Introduction to MySQL Lab no. 10 Advance Database Management System.
Polling System Part 1 Creating PHP & MySQL Files CIS 254.
United Nations Economic Commission for Europe Statistical Division Software Approaches for the Dissemination of Statistical Information UNECE Training.
EDIMatrix Ltd 1997 DMX Overview SEMA Infodata Nov 1997.
Chapter 2: SQL – The Basics Objectives: 1.The SQL execution environment 2.SELECT statement 3.SQL Developer & SQL*Plus.
PX development Mikael Nordberg & Petros Likidis Statistics Sweden.
Introduction to Enterprise Guide Jennifer Schmidt Rhonda Ellis Cassandra Hall.
The story of PC-Axis Thesis on multidimensional matrices by Dr. Bo Sundgren (70´s) Axis (80´s) PC-Axis (90´s) –Swedish Population Census (1990) Relational.
Wishlist 2013 Statistics Sweden 22nd PC-Axis Reference Group Meeting - Tirana.
Grid Deployment Enabling Grids for E-sciencE BDII 2171 LDAP 2172 LDAP 2173 LDAP 2170 Port Fwd Update DB & Modify DB 2170 Port.
WINDOWS AZURE MOBILE SERVICES AN INTRODUCTION Bret Stateham Technical Evangelist
The PC-Axis family A consortium for output databases Presented by: Lars Nordbäck, Statistics Sweden IT Directors Group, Luxembourg
Welcome! PC-Axis Reference Group Meeting 2015 Stockholm Statistics Sweden.
MySQL and GRID status Gabriele Carcassi 9 September 2002.
JDS5 Training Guide. On Start Up you will see this screen click the OK button Click OK.
Greenlandic Summer 2015 PC-Axis International, Stockholm 2015Lars Pedersen.
PX-API and development of new console Norwegian experiences Jan Bruusgaard Pc-Axis Reference Group Meeting 2015.
© 2007 by Prentice Hall2-1 Introduction to Oracle 10g Chapter 2 Overview of SQL and SQL*Plus James Perry and Gerald Post.
PX-Web 2014 Maria Gunnarsson, Statistics Sweden Petros Likidis, Statistics Sweden.
Maria Gunnarsson SCB Mikael Nordberg SCB PX-Web 2015 December.
Creating Simple and Parallel Data Loads With DTS.
Aggregator Stage : Definition : Aggregator classifies data rows from a single input link into groups and calculates totals or other aggregate functions.
CAA Database Overview Sinéad McCaffrey. Metadata ObservatoryExperiment Instrument Mission Dataset File.
>> Dissemination databases and PC-Axis in Statistics Denmark Lars Knudsen Statistics Denmark.
INFO 1103 Database System – Group Project. Project evaluation Deliverables – 10% Presentation – 40% Project Report – 50%
Update 2015 Nordic Web, Stockholm 2015 Lars Pedersen.
Eurostat main tables and graphs
Maria Gunnarsson Statistic Sweden Mikael Nordberg Statistic Sweden
Easily retrieve data from the Baan database
Intro to Ethical Hacking
National updates Statistics Denmark
MS Access Database Connection
Welcome day 2.
PX-Web 2017 v1 and The situation at Statistics Sweden right now
Intro to Ethical Hacking
PX-Web developments plan
Maps and Charts In Statistics Denmark.
PX-Web 2019 and more… Mikael Nordberg Developer Statistics Sweden.
Computer Science Projects Database Theory / Prototypes
Grocery Store Outline csc242 – web programming.
Data Access Layer (Con’t) (Overview)
Final Project Geog 375 Daniel Hewitt.
Presentation transcript:

Statistics 2 go (simplePXsql, PX2go and API) PC-Axis reference group, Tirana 2013Lars Pedersen

How many persons are living in Greenland ? Look to: Question:

Answer by a pc:

Answer by an iPad :

Answer by an iPhone:

Motorola Storno Sony Ericsson Nokia and px-web 2008 Think off:

How many persons are living in Greenland ? Question:

Answer by a pc:

Answer by an iPad:

Overview a little history from px-files to a database what is that good for? px2go api’s cliffhanger

a little history :

the family concept

We need to stay connected Meet, party and quarrel a road-map the family concept

PX-files DB Nordic PX-web PC-Axis PC-Axis/sql Denmark Norway Ireland … Create procedures Create procedures Create procedures Create procedures ?

Database specific settings: [System] sql_database=statbank2013 sql_baseowner=.. sql_datestamp=Y-D-M h:m:00 sql_endline= sql_bulksave=1 from px-files to a database

All can be done batch by a few lines in PX-JOB from px-files to a off : Batch command line: : PX-Job.exe {job} [in] {out} {err} {def} {log} {-options} :PX-Job job=sql in=H:\database\test\ out=H:\database\test\ log=H:\database\test\pxjob1.txt err=H:\database\test\ -s -o1 -e1 PX-Job job=sql in=H:\database\test\ out=H:\database\test\ log=H:\database\test\pxjob1a.txt err=H:\database\test\ -s -o3 -e1

input approximately 650 px files normal tree structure

output with switch -s -o1 -e1 INSERT INTO statbank2013..PXTables VALUES('en','BEEST1','ANSI','2000','',null,' :00:00',' :00:00','','','Annually','',null,null,0,null,null,'','','BE','Population','','',null,'Population January 1st by residence, gender, age and place of birth ','','Population','Persons','',' :00:00','','','','','Lars Pedersen, Greenland','','',null,null,null,null,'','','','','','','','','',null,'','','',null,null,null) INSERT INTO statbank2013..PXVariables VALUES('en','BEEST1','time','1','R','','',null,'','',null,null,null,'','',null,'',null,null,'','') INSERT INTO statbank2013..PXValues VALUES('en','BEEST1','time','1','1977','1','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1978','2','',null,null,'','','','','',null,null,null,null,null,' ',''),('en','BEEST1','time','1','1979','3','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1980','4','',null,null,'','','','','',null,null,null,null,null,'',''),( 'en','BEEST1','time','1','1981','5','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1982','6','',null,null,'','','','','',null,null,null,null,null,'',''),('en', 'BEEST1','time','1','1983','7','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1984','8','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEE ST1','time','1','1985','9','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1986','10','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST 1','time','1','1987','11','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1988','12','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1989','13','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1990','14','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1',' time','1','1991','15','',null,null,'','','','','',null,null,null,null,null,'',''),

output with switch -s –o3 -e1 Converts our 574 Mb px-files to tab-separated csv-files in less than 1 hour

Empty database (drop if exsist) Create 3 tables (tables, variables and values) Loop though the px-files set metafile=objf.opentextfile(f.name) uttext = metafile.readall metafile.close adoConn.execute uttext set metafile=nothing Create table for data by use of newly added meta Create insert statement for data BULK INSERT statbank2013..PXData_NRE13 FROM 'G:\web\database\Greenland\National accounts\NRE13_data.sql' WITH (CODEPAGE = 'ACP',FIELDTERMINATOR='\t',ROWTERMINATOR='\n') transfer script (load all)

simplePXsql (data tables)

simplePXsql (tables)

simplePXsql (variables)

simplePXsql (values)

what is that good for : a dirty px-file datacollection Used for cleansing queries across tables translation services bridge to the nordic model ?

connections PX-files DB Nordic PX-web PC-Axis PC-Axis/sql Denmark Norway Ireland … Create procedures Create procedures Create procedures Create procedures simplePXsql PX-Edit

nordic db model

Find table – a link ( – category – search Table selected – known selection – unknown selection px2go

Greenland Tourism ( ) automatic updated from our statbank api s… happens

Graph Table Pyramid Bubles

matrix Json-stat getmeta getdefault showcube Graph Table explore generate API Json-stat csv re-doing px2go for json-stat

groupings production cliffhanger

Screenshots from live - demo