Presentation is loading. Please wait.

Presentation is loading. Please wait.

Challenges with distributed systems like WISE

Similar presentations


Presentation on theme: "Challenges with distributed systems like WISE"— Presentation transcript:

1 Challenges with distributed systems like WISE
Jan Bliki, EEA

2 Challenges Partnership is as important then technology choice.
Many technological solutions but they only fit specific purposes. Find clear use cases that explains WISE distribution needs. Different implemented systems will demand an open solution. WISE is a pioneer. Challenge data (Rombout, Keiran) Partnership is as important then technology Find a generic process to simplify development cost to all nodes. Are we willing to experiment and find best approach? Make it fit to any existing national system. Open standards that fit our needs. Challenge app Define clear use cases where we go for this approach. Partnership is as important then technology. Define level of quality we expect from those services. (Should all legends/layouts of those maps be the same?)

3 Distributed systems Application level (Human to system) View services
Identify services Analytical services Discovery services Data distribution (System to system) Aquire data and process (Aggrigate) on demand Near real time data flows Cashing data. Download and Transformation services WMS,WFS,WFST,OLAP

4 Application level (Human to system)
Made for a specifique purpose. WISE today has arround 15 thematic maps. Many more will come representing European views. How many national representations are currently existing today? Static representations : Orthophoto or Satelite images Harmonise vector data on all National levels so look and feel is the same -> costly If the data is send to European level -> make the presentation on EU level. If the data is not sent and it only used for viewing, then mapping services is a good choice. (Cadastral maps?!! As an example)

5 Application level Use National mapping services to deliver european maps. Static representations : Orthophoto or Satelite images Harmonise vector data on all National levels so look and feel is the same -> costly If the data is send to European level -> make the presentation on EU level. If the data is not sent and it only used for viewing, then mapping services is a good choice. (Cadastral maps?!! As an example)

6 Data distribution (System to system)
Tabular data or/and Spatial elements Full detail or/and Aggrigated levels Different systems and solutions Static representations : Orthophoto or Satelite images Harmonise vector data on all National levels so look and feel is the same -> costly If the data is send to European level -> make the presentation on EU level. If the data is not sent and it only used for viewing, then mapping services is a good choice. (Cadastral maps?!! As an example)

7 Types of data flows today
Existing data flows (Manual) Today : Obligation package, delivered on specefique time, once every 6months too once every three years. Today : 500 Such flows exists today in ReportNet Today: WISE has a couple of those WFD, Bathing Water, WaterBase, Manual why: Gap issues, Validation, Topology and scale corrections on European level, Additional relations to other EU data, Very slow updates. Other categories : Manual reporting directly on European level, very aggregated data. “How many station did exceeded the treshold”. Other categories : Fully automated on National level, reporting became a automated report. (We come back to that one later)

8 Traditional data flows
Why manual : Data is received by 25 countries, collected inside reportnet as individual files. Only 2% today is fully system readable. (Excel, PDF, Text files, etc) Why manual : EEA topic centre collects those data and generates a European dataset from this. (We call this an European working database) Why manual : Validation is done between countries. (Topology issues, Compare to previous delivery, etc…) Why manual : Delivered to EEA, where we prepare the datasets for distribution and application use. Takes about 1 or 2 years

9 Why pre-process in WISE?
Optimisation of data for visualisation on Web. Generalisation Spatial and attribute indexing as well as index tuning Data aggregation Internal mapping Summary statistics

10 EEA SDI pre-process flow for WISE

11 Life data flows Define life data: National system updated constantly, Has a team that managed this. Define life data: same for the European system setup. Define life data: constant updates are passed on and monitored. Small changes on the system happen all the time and are corrected instantly. Traditional flows will still be existing as well. EEA only has experience in tabular life data flows where spatial data still comes from the traditional flows. Life systems are more expensive then traditional data flows but not when already existing.

12 Types of data flows today
Existing data flows (Manual) Life data flows can easier provide more details data and should move instantly. Maintaining life systems on National and European level is expensive and no benefit would come out if we can’t break the barrier of once a year delivery. (RAW, Accepted reporting) Life systems

13 Data distribution There is no common solution on this day.
When/How will a national syst. sends data to European systems. ? When/How will European syst. Requests data from National systems. ? XML formats,WFS, WFST (OGC), SOAP Database mirroring technologies Sensor web SWE protocol (OGC) Static representations : Orthophoto or Satelite images Harmonise vector data on all National levels so look and feel is the same -> costly If the data is send to European level -> make the presentation on EU level. If the data is not sent and it only used for viewing, then mapping services is a good choice. (Cadastral maps?!! As an example) - Vector data from National level. How will we define and manage topology rules on a European level?

14 Challenges Harmonizing data to European level is a challenge. Harmanizing on application level is as hard if not harder.

15 Challenges Partnership is as important then technology choice.
Many technological solutions but only fit specific purposes. Find clear use cases that explains WISE distribution needs. Different implemented systems will demand an open solution, WISE is a pioneer. Challenge data Partnership is as important then technology Find a generic process to simplify development cost to all nodes. Are we willing to experiment and find best approach? Make it fit to any existing national system. Open standards that fit our needs. Challenge app Define clear use cases where we go for this approach. Partnership is as important then technology. Define level of quality we expect from those services. (Should all legends/layouts of those maps be the same?)

16 Where is all the data coming from?
EEA NRT Database Internet EEA website Data exchange HTTP Post HTTP Get FTP Put FTP Get Sampler logger sources Airbase Annual data transfer Public national database Compliance reporting QA/AC Regional database Internet Public transfer report


Download ppt "Challenges with distributed systems like WISE"

Similar presentations


Ads by Google