1. S P A Z I O A Finmeccanica Company IMAS-D Project 1 IMAS-D Project Overview - Quadrics Ltd. IMAS-D Image Management System Demonstrator Project Overview (extracts from presentations given at the official project meetings) M. Verola massimo.verola@roma.quadrics.com

2. S P A Z I O A Finmeccanica Company IMAS-D Project 2 IMAS-D Project Overview - Quadrics Ltd.  Anatomy of the Project

3. S P A Z I O A Finmeccanica Company IMAS-D Project 3 IMAS-D Project Overview - Quadrics Ltd. Application Scenario Remote Sensing (RS) data and related products have proven they can be exploited not only in scientific and academic communities, but also in governmental institutions, in national defense bodies and in commercial enterprises, thanks to the continuous increasing of spatial and spectral data resolution. In particular SAR, panchromatic and hyperspectral sensors are becoming more and more a source of information of paramount importance for a variety of critical applications such as environmental protection, urban and regional planning and monitoring, agricultural census, military tactical and strategic reconnaissance and surveillance.  Anatomy of the Project

4. S P A Z I O A Finmeccanica Company IMAS-D Project 4 IMAS-D Project Overview - Quadrics Ltd. Need for HPC As higher resolution ( 1 meter) data become available and the provision rate of data increases, due to the availability of satellite constellation with high revisit frequency, an enormous amount of data will have to be processed and stored. The most effective way to face to this huge amount of information is to manage them by using high performance computing facilities and appropriate parallel algorithms.  Anatomy of the Project

5. S P A Z I O A Finmeccanica Company IMAS-D Project 5 IMAS-D Project Overview - Quadrics Ltd. Project Objectives Main Goal: validate and accelerate the application of High Performance Computing (HPC) to command, control, communication and intelligence systems, in particular in the field of Image Management and Processing systems. Technical Objective: realize an Imagery Management and Processing System able to demonstrate its relevance by means of: - the provision of advanced image processing functionality, and, in particular, of parallel algorithms - the management (storage, retrieval, exploitation) of imagery data, linearly scalable to a potential unlimited size without performance degradation - the integration of a multi-tier software system based on both commercial packages and specialized newly developed modules, where a powerful parallel processing engine, a state-of-the-art image visualization environment, an innovative DBMS and a Web-based product dissemination gateway cooperate together for building up an end-to-end RS data management system  Anatomy of the Project

6. S P A Z I O A Finmeccanica Company IMAS-D Project 6 IMAS-D Project Overview - Quadrics Ltd. Project Exploitation The main benefit of combining the power of cluster computing with the advanced functionality of state-of-the-art commercial software packages is the opportunity for the end user to greatly expand the set of problems that can be solved, both in terms of computational domain size than in terms of the quality of the implemented algorithms and techniques. The prospected product resulting from the evolution of IMAS-D will match the ever increasing computing power and memory resource demands that new-generation RS sensors, especially in the hyperspectral field, will require for an effective exploitation of the produced raw data.  Anatomy of the Project

7. S P A Z I O A Finmeccanica Company IMAS-D Project 7 IMAS-D Project Overview - Quadrics Ltd. Technical Risks "The challenge of the IMAS-D project is to create a framework where stable commercial components live together with innovative components, thus several technical risks are implicit in the design of the system." SW integration complexity HPC system using DBMS services HPC system providing multiuser access via Web Bulky data management (DBMS, RAM requirements) Synchronization and cooperation of different programming models and environments (OO programming, ODBMS, message passing, Web technologies, Java) Effective deployment of new SW environments (ENVI+IDL, FastObjects)  Anatomy of the Project

8. S P A Z I O A Finmeccanica Company IMAS-D Project 8 IMAS-D Project Overview - Quadrics Ltd.  System Architecture

9. S P A Z I O A Finmeccanica Company IMAS-D Project 9 IMAS-D Project Overview - Quadrics Ltd. System Architecture (1) The system architecture has been designed in order to meet the following goals: fast and scalable processing performance application-oriented DBMS services, optimized for handling bulky data user-friendly and reliable end-to-end operations The final result is an integrated HW and SW solution, built around a Quadrics Linux Cluster and organized into three major subsystems: Image Processing Subsystem Storage Management Subsystem Collection and Dissemination Subsystem  System Architecture

10. S P A Z I O A Finmeccanica Company IMAS-D Project 10 IMAS-D Project Overview - Quadrics Ltd. System Architecture (2) T he Image Processing Subsystem extends the single-node computing capabilities of ENVI (by Research System), the most advanced remote sensing software available on the market, to the scalable cluster computing solution. The ENVI GUI has been enriched with hooks and handles to activate parallel functions, which will dramatically speed-up the image processing tasks. Furthermore the Storage Management Subsystem integrates the innovative object-oriented DBMS, FastObjects (by Poet Software), into the input/output ENVI menu, providing sophisticated imagery data query and management functions. The Collection and Dissemination Subsystem makes IMAS-D a real end-to-end system, suitable for the automatic collection and archiving of incoming satellite images and for delivering via Web the processed products.  System Architecture

11. S P A Z I O A Finmeccanica Company IMAS-D Project 11 IMAS-D Project Overview - Quadrics Ltd. HW Architecture RAID Storage SCSI QsNet links Ethernet Hub Dell 2 PIII node 0 Dell 2 PIII node 1 Dell 2 PIII node 2 Dell 2 PIII node 3 Dell 2 PIII node 4 Dell 2 PIII node 5 Dell 2 PIII node 6 Dell 2 PIII node 7 QsNet Switch Intranet User PC Color Printer Firewall Internet  System Architecture

12. S P A Z I O A Finmeccanica Company IMAS-D Project 12 IMAS-D Project Overview - Quadrics Ltd. Real HW Picture Elite Switch RAID Storage Terminal Concentrator Ethernet Hub Quadrics Linux Cluster Another QLC ! (  -based) Dual PIII computing nodes  System Architecture IMAS-D System

13. S P A Z I O A Finmeccanica Company IMAS-D Project 13 IMAS-D Project Overview - Quadrics Ltd. System Operation Definition FastObjects OODBMS API Lev. 1 RAW DATA in ENVI format Lev. 2 PRE-PROCESSED DATA Lev. 3 PROCESSED DATA OFF IMASD - GUI HPC Data Collection Pre-processing Processing RAW IMAGERY DATA standard sequential image processing IMASD-SHARED MEMORY LIBRARY SHARED MEMORY C + MPI Proc 1Proc 2Proc N ON Standard Dissemination Offline products delivered via traditional channels HPC ON Internet Dissemination WEB Site  System Architecture

14. S P A Z I O A Finmeccanica Company IMAS-D Project 14 IMAS-D Project Overview - Quadrics Ltd. DELL LIB (based on Linux shmem) IMASD-DB Client Classes DBAPI LIB (based on JNI) Intranet Local Web Client IMAS-D GUI SW Actors IDL LIB ENVI LIB ENVI GUI Library Executable Java Appl. Java Archive IDL Library Proj. Partner 3rd Party SW Quadrics Legenda IDL LIB ENVI LIB IMAS-D DB Server Image Transfer Server Fast Objects Classes IMAS-D DB Server Classes Fast Objects Server IMAS-D DB Imagery Data and Metadata Internet Remote Web Client Firewall IMAS-D Web Server IQLC LIB (based on Quadrics MPI) AIPB Server (parallel proc. server)  System Architecture

15. S P A Z I O A Finmeccanica Company IMAS-D Project 15 IMAS-D Project Overview - Quadrics Ltd. Data Flow between SW Modules DELL Data Exchange Local Library PGIV Parallel GUI for Image Visualization AIPB Advanced Image Processing Broker IQLC Imagery Qualified Library for Cluster ODBMS API C/Java API for FastObjects WEB SERVER Dissemination Gateway AADC Advanced Automatic Data Collection proc. request + image data + ancillary data processed image + response info processed image data blocks image data blocks + ancillary data insert/query command search statistics + retrieved image raw image data START / STOP proc. request + image data + ancillary data processed image + response info query request retrieved images  System Architecture

16. S P A Z I O A Finmeccanica Company IMAS-D Project 16 IMAS-D Project Overview - Quadrics Ltd. SW Module Dependencies WEB SERVER Dissemination Gateway Responsibilities -- provide Web services for product dissemination Data Base Parallel Processing Visualization Web Server RT Data Collection ODBMS API C/Java API for FastObjects Responsibilities -- API for accessing IMASD imagery DB IQLC Imagery Qualified Library for Cluster Responsibilities -- provide parallel image processing algorithms AIPB Advanced Image Processing Broker Responsibilities -- handle requests to execute parallel image processing algorithms DELL Data Exchange Local Library Responsibilities -- handle the communication between the sequential interactive IMASD GUI (PGIV) and the parallel processing engine PGIV Parallel GUI for Image Visualization Responsibilities -- extend the functionality and performance of ENVI GUI with hooks and handles to parallel processing functions AADC Advanced Automatic Data Collection Responsibilities -- simulate the automatic data collection procedure -- provide a communication library for transferring imagery data files  System Architecture

17. S P A Z I O A Finmeccanica Company IMAS-D Project 17 IMAS-D Project Overview - Quadrics Ltd. Parallel Processing Interaction Diagram [7] return data subset result send image subset [4] ENVI+ IDL library ENVI+ IDL library ENVI widgets IDL sequential code shared mem C communication library [10] visualize results activate processing [1] [9] receive response send request [2] IDL to C interface DELL communication library  System Architecture C+MPI parallel programming [8] send response receive request [3] [7] return data subset result [4] send image subset MPI library C+MPI parallel programming C+MPI parallel programming C to IDL interfaces [6] return call [5] [5] call return [6]

18. S P A Z I O A Finmeccanica Company IMAS-D Project 18 IMAS-D Project Overview - Quadrics Ltd. Development tools Visualization and processing Linux Red HatLinux distribution, installed on QLC ENVIGUI + library for remote sensing applications IDLinterpreted programming languages for image processing C/C++ compilercompilers for sequential/parallel C/C++ programs MPIMessage Passing Interface library, optimized for QsNet makesupport tool for SW compilation CVSConcurrent Version System, used for SW development  System Architecture

19. S P A Z I O A Finmeccanica Company IMAS-D Project 19 IMAS-D Project Overview - Quadrics Ltd.  Performance Results

20. S P A Z I O A Finmeccanica Company IMAS-D Project 20 IMAS-D Project Overview - Quadrics Ltd.  WP6 Progress Status Parallel Performance

21. S P A Z I O A Finmeccanica Company IMAS-D Project 21 IMAS-D Project Overview - Quadrics Ltd.  WP6 Progress Status Parallel Performance

22. S P A Z I O A Finmeccanica Company IMAS-D Project 22 IMAS-D Project Overview - Quadrics Ltd.  WP6 Progress Status Parallel Performance

23. S P A Z I O A Finmeccanica Company IMAS-D Project 23 IMAS-D Project Overview - Quadrics Ltd.  WP6 Progress Status Parallel Performance

24. S P A Z I O A Finmeccanica Company IMAS-D Project 24 IMAS-D Project Overview - Quadrics Ltd.  WP6 Progress Status Parallel Performance

25. S P A Z I O A Finmeccanica Company IMAS-D Project 25 IMAS-D Project Overview - Quadrics Ltd.  Demo Session

26. S P A Z I O A Finmeccanica Company IMAS-D Project 26 IMAS-D Project Overview - Quadrics Ltd. IMASD

27. S P A Z I O A Finmeccanica Company IMAS-D Project 27 IMAS-D Project Overview - Quadrics Ltd. Main Menu Tools for IMASD Cluster Management Highlights operations related to the IMASD Cluster

28. S P A Z I O A Finmeccanica Company IMAS-D Project 28 IMAS-D Project Overview - Quadrics Ltd. Overview of IMASD Tools Data Collection & Pre-Processing Tools Interactive use of Processing functionalities Cluster Management Data Base Interface Help

29. S P A Z I O A Finmeccanica Company IMAS-D Project 29 IMAS-D Project Overview - Quadrics Ltd. Cluster Management Select the number of CPUs to be allocated for parallel computing Start and stop the parallel computing server Show CPUs allocation Check the Cluster status

30. S P A Z I O A Finmeccanica Company IMAS-D Project 30 IMAS-D Project Overview - Quadrics Ltd. Cluster Control Cluster Status Reserve a set of CPUs to be used for parallel computing Graphical display of CPUs status Check the status of the Cluster partitions Running 8 CPUsRunning 16 CPUs Start the parallel server