1. CNR experience within CIOP: porting the SBAS-InSAR software into the cloud F. Casu, I. Zinno, S. Elefante, P. Imperatore, M. Manunta CNR-IREA Via Diocleziano 328, 80124, Napoli

2. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Outlook The SBAS-InSAR algorithm: – scope and main application Why the cloud? Porting SBAS-InSAR into cloud environment – Sequential algorithm – Barely parallel solution – CIOP adaptation – Performances Lesson learnt and future expectation

3. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it SBAS-InSAR Processing ~150 Satellite raw (L0) data per scene: 40GB Sequential Processing Time: 300h CNR Earthquakes Volcanoes Oil & Gas Water Resources

4. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it SBAS-InSAR Application L’Aquila earthquake Civil Protection Risk Management Insurances

5. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it SBAS-InSAR Application Mt. Etna volcano ERS-1/2 and ENVISAT data (1992-2010) Eruption Civil Protection Risk Management Natural Hazards

6. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it SBAS-InSAR Application ERS-1/2 Descending data (1995-2002) Los Angeles >1<- 1 cm/year 0 Oil & Gas Field Oil extraction

7. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it SBAS-InSAR Application ERS-1/2 Descending data (1995-2002) Los Angeles >1<- 1 cm/year 0 Agriculture Sustainable and social development Water Resources SBAS-InSAR Application

8. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Building and infrastructures analysis >1<- 1 cm/year 0 Damage assessment Insurances ESA - ESRIN YOU ARE HERE (and safe)! SBAS-InSAR Application Rome

9. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Past, present and future SAR Satellite Constellations Time

10. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Too many data Too many users Why the cloud?

11. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Why the cloud? Wide application scenarios Process large archived (ERS, ENVISAT, …) or forthcoming (CSK, TSX, Sentinel-1) amount of SAR data High resource demand during emergency phases Processing time must be “reasonable” Make the processed data/achieved results/processing tools available to the community: implement the Supersites concept

12. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it CIOP Application Platform Data & Catalogue User (CNR) Cloud Controller API OCCI Web Interface Sandbox private public

13. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it User (CNR) Web Interface Sandbox CIOP Application Platform

14. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration RefinementSLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series

15. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Sequential test case 7 ASAR-ENVISAT acquisitions 1 core 8 GB RAM 500GB of storage (NFS)

16. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration RefinementSLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series 5 min 33 min 190 min 3 min 0.3 min N/A Negligible 75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min N/A ~14.8 hours 247 min

17. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration RefinementSLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series Orbits 33 min 190 min 3 min 0.3 min N/A Negligible N/A 75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min 5 min 247 min ~14.8 hours

18. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Barely parallel test case 7 ASAR-ENVISAT acquisitions 2 cores 8 GB RAM per core 500GB of storage (NFS)

19. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Sequential SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration Refinement SLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series Orbits 33 min 190 min 3 min 0.3 min N/A Negligible N/A 75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min 5 min 247 min ~14.8 hours

20. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it ~10 hours Parallel SBAS Flow Diagram SLC Scaling selection SLC cut DEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration Refinement SLC Scaling SLC Registration Interferometric Pairs Selection DEM Shift Calculation Interferogram Computation Displacement time series Orbits 33 min 190 min 3 min 0.3 min N/A Negligible N/A 5 min75 min 70 min 66 min 5 min 111 min 7.3 min 24 min 56 min 247 min 2.5 min 100 min 1.8 min 39 min 34 min 56 min 3.8 min 30 min 177 min

21. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Pros and Cons Scalable (on Images & Interferograms) Easy to implement and debug No job scheduling and balancing Not fully portable

22. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it SBAS Flow Diagram within CIOP SLC cutDEM to SAR conversion Range File computation DEM Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area 2nd step Registration Registration Refinement SLC Scaling SLC RegistrationInterferometric Pairs Selection Collec t

23. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Interferogram Selection Coherent point Selection Tie Point Refinement Unwrapping Time Series Generation Residual Phase Computation Orbital Ramp Estimation Filtered Interferograms Unwrapping Tie Point Refinement Time Series Generation Residual Phase Computation Atmospheric Filtering Orbit Correction DEM Shift Calculation Interferogram Computation Collec t SBAS Flow Diagram within CIOP Collec t yes no End

24. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it IREA cluster vs. CIOP Sandbox comparison 7 ASAR-ENVISAT acquisitions 2 cores 8 GB RAM per core 500GB of storage IREACIOP ~10 hours~13 hours

25. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it IREACIOP 2.53 3318 10063 -12 Negligible Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area SLC Scaling Collec t IREA vs. CIOP Sandbox Comparison

26. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Focusing Master Focusing Parameters Computation Data Deformatting OrbitsSAR Data Shift Computation Common Area SLC Scaling Collec t SLC cutDEM to SAR conversion Range File computation DEM 2nd step Registration Registration Refinement SLC RegistrationInterferometric Pairs Selection Collec t IREA vs. CIOP Sandbox Comparison

27. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it SLC cutDEM to SAR conversion Range File computation DEM 2nd step Registration Registration Refinement SLC RegistrationInterferometric Pairs Selection Collec t IREA vs. CIOP Sandbox Comparison IREACIOP 0.34 7574 3944 -40 3486 588 -30 5644 -25 3.820

28. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Unwrapping Preparation Unwrapping Time Series Generation Orbital Ramp Estimation Final processing and Atmospheric Filtering Orbit Correction DEM Shift Calculation Interferogram Computation Collec t SBAS Flow Diagram within CIOP Collec t yes no Collec t IREACIOP 2448 5676 10 3550 4150 5662

29. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it >1<-1 cm/year Analyzed time interval: 2002-2010 Real case study (Benchmark) IREACIOP ~24 hours~48 hours 55 ASAR-ENVISAT 8 cores 8 GB RAM per core 1TB of storage Napoli Bay area

30. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it CIOP Pros & Cons Scalable (on Images & Interferograms) Relatively easy to port scientific algorithms, after a preparatory learning phase on the environment Collect steps represent a bottle neck! (large data transfer overhead) Portable, but built on Open Nebula which could be not supported by the cloud provider (need of additional driver)

31. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Final considerations CIOP has been conceived to be the most general and portable as possible but, at the present stage/configuration, seems to be not the optimal solution for an application which is not characterized by independent jobs For SBAS purposes, different environment solution (cloud resources as grid computing, single virtual workstation on the cloud) could be explored

32. CNR experience within CIOP: porting the SBAS-InSAR software into the cloudcasu.f@irea.cnr.it Final considerations The presented activities have been carried out within the Helix Nebula framework, which aims at providing a federated European science cloud The ESA SSEP flagship, aimed at creating a common scientific ecosystem for sharing data, querying tools, processors, and added value results, depicts an ambitious goal, which found in Helix Nebula its natural growth environment The large debate within Helix Nebula for the generation of a unified driver for federated cloud implementation should be carefully taken into account by ESA