1. WP 9.4 Use Case (ESA-ESRIN, IPSL, KNMI)
Annalisa Terracina, ESA-ESRIN
Earth Observation work package (WP9)

2. Earth Observation use case
The processing, archiving and validation of GOME ozone profile products using two different algorithms and ground based measurements for validation form the basis of this use case.
This use case is an example of common EO data usage and EO science: the 'raw' data is processed into products, using different algorithms. The outcome is then validated using ground-based measurements.
One aspect to tackle is to find the proper GOME pixels that coincide with the ground based measurements, for this we are going to use a grid enabled database, to store geo- and time- references of GOME files/pixels
Goal: do this for one year of GOME data

3. Earth Observation DEMO
Raw satellite data
from the GOME instrument
Earth Observation DEMO
for PM 24
LIDAR
data
Processing of raw GOME
data to ozone profiles
With OPERA and NNO
Validate GOME ozone profiles
With Ground Based measurements
DataGRID
Visualization

4. GOME Use Case + KNMI ESA ESA Science Application End User IPSL
4724 files = 66 Gb
9,448,000 files = 108 Gb
KNMI L1 L2
ESA
ESA
RAW L1 L1 L2
Science
Application
End User L2 + L3
IPSL
VAL L2
VAL
Regulated Access to Grid processing power
Secure access to Grid-registered high-volume data storage

5. Scenario overview: PM24
ESA will provide GOME level 1 data on a GRID storage element (SE).
The meta information is stored in a GRID accessible database, Spitfire.
This data and meta data is used by KNMI to produce Ozone profile data using the Opera S/W, by running it on a Computing Element (CE).
The profile data (level 2) is stored on a GRID SE. The profile meta data is stored in a database accessible on the GRID via Spitfire.
ESA will also use the level 1 data and meta data by retrieving level 1 files from a SE, and producing Ozone profiles. A Neural Network algorithm will do the processing.
ESA will store the profile data and meta data on the GRID via a SE and Spitfire.
The IPSL application will perform a validation of the produced ozone profiles. The profiles are validated against in situ LIDAR ozone profile measurements.
IPSL will retrieve profile data (OPERA and NNO) from the GRID, querying the provided meta data to find data which coincides with their measurements. The LIDAR data will be stored on a SE on the GRID.

6. File numbers for one year of GOME data :
Number of files
(to be stored and replicated):
File Size
Level 1:
4,724
15 Mb
NNO:
9,448,000
10 kb
Opera :
12 kb
Lidar 12
2.5 Mb
Total:
18,900,736
267 Gbyte
Gome has a data set of 5 years
Gome is relatively small (in both size and number of files)

7. Use case flow for profile processing:

8. Questions:
Is the amount of files a problem? (for replication and meta data storage)
Does the use case contain enough detail for middleware?
Other questions?