2. Cluster Programming Technology and its Application in Meteorology Computer and Automation Research Institute Hungarian Academy of Sciences Hungarian Meteorological Service Silicon Computers Ltd.

3. Background Hungarian Meteorological Service developed MEANDER (MEsoscale Analysis Nowcasting and DEcision Routines) –crucial task in the protection of life and property (storm warning at Lake Balaton, weather warnings for aviation...) –based on incoming meteorology information and computational intensive methods MTA SZTAKI developed P-GRADE parallel programming environment –efficient, graphical support for the entire life cycle of parallel program development Cluster programming technology and its applications in meteorology –joint project of MTA SZTAKI, Hungarian Meteorological Service and Silicon Computers Ltd. –supported by Research & Development Division, Ministry of Education

4.  Goal: Analysis of all the available meteorology information  producing parameters on a high resolution regular mesh (10km--> 1km)  ultra-short range forecast (up to 6 hours)  Application: Forecasting dangerous weather situations (storms, fog, etc.)  Meteorology information: surface level measurements, high- altitude measurements, radar, satellite, lightning, results of previous computed models, etc.  Basic parameters: pressure, temperature, humidity, wind, …  Derived parameters: type of clouds, visibility, … MEANDER Program Package

5. Structure of MEANDER First guess data ALADDIN SYNOP data Satellite data Radar data CANARI DELTA analysis Basic fields: pressure, temperature, humidity, wind Derived fields: Type of clouds, visibility, etc. BASIC GRID Radar to grid Satellite to grid “Present ” weather Type of clouds Overcas t Visibilit y Rainfall phase Visualization For meteorologist :HAWK For users: GIF Lightning decode

6. 36000 km Satellite raw image Receiving image Transformation & Interpolation & Processing Processing: altitude of clouds Transformation Interpolation to basic GRID Processing of satellite images

7. THE PROBLEM MEANDER Sequential code C, C++ Fortran MEANDER Parallel version PC claster SGI Origin 2000 SUN E10000 Parallelisation? Debugging? Performance?

8. GRAPNELGRP2C GRAPNEL graphical language / GRP2C pre- compiler Unix-like operating systems & C, C++, Fortran programming libraries PVM or MPI message passing library DIWIDE distributed debugger GRED graphical editor GRM GRM monitor distributed checkpointing tool GRP_CHKPT GRP_MM migration module GRP_LB load balancer PROVE performance visualization tool TLC TLC model checker Macrostep debugger P-GRADE The solution: P-GRADE development environment

9. Results: temperature and wind at 850 hPa level MEANDER: 3D FIELDS 2D analysis Radar data Satellite data Delta 3D analysis...computes the basic meteorological fields: pressure, temperature, humidity, wind velocity and direction for a high resolution 3D mesh ( 10km -1km)

10. Implementation of DELTA analysis in P-GRADE Fortran seq. code

11. P-GRADE version of MEANDER for clusters & supercomputers 25 x 4 x 10 x 25 x 20 x

12. Live demo (5 th DataGRID conference) 34 Mbit Shared PERL-GRID CONDOR-PVM job 11/5 Mbit Dedicated P-GRADE PERL-GRID job ftp.met.hu netCDF 512 kbit Shared netCDF input Parallel execution and GRM GRM TRACE & Results GRM TRACE & Results

13. PROVE performance visualization

14. Edit, debugging Performance- analysis Testing, Execution Resource requirements of P-GRADE

15. Advantages of P-GRADE environment  Efficient support for each stage of parallel program development  Fast parallelisation of existing algorithms  Reusability of sequential code  Hiding of low level communication functions  Unified and integrated graphical concept  Predefined communication templates  Support for hierarchical design  Even non-professional programmers can use it (steep learning curve)  Portability from supercomputers to PC clusters

16. Support for cluster computing More information: www.lpds.sztaki.hu New facilities in P-GRADE for long-running parallel applications: Distributed checkpointing Process migration Dynamic load balancing Fault-tolerance execution I. Execution without load balancing III. Load balancing & migration IV. Result: Faster execution II. Checkpointing