Weather Research and Forecasting (WRF) Portal Elias Rodriguez School of Computer Engineering Computer Science and Electrical Engineering Polytechnic University of Puerto Rico Seychelles Martinez School of Computing and Information Sciences Florida International University REU Summer Program 11/12/2018

Outline Introduction Motivation Previous work Description of the project Expectations Project Plan Questions 11/12/2018

Introduction What is WRF? What is Ensemble Forecasting? How does the system work? What is the scope of what we are doing? 11/12/2018

Introduction What is WRF? Weather Research and Forecasting Developed by National Center for Atmospheric Research (NCAR) Widely accepted in meteorology The Weather Research and Forecasting (WRF) code is the latest numerical model developed by the national Center for Atmospheric Research (NCAR) for both operational forecating and atmospheric research. A measrue of its success is that WRF has been adopted by meteorological services and researchers worldwide. 11/12/2018

Introduction What is Ensemble Forecasting? Purpose: account for errors in forecasting system due to chaotic nature of atmospheric dynamics Different models created in parallel with different initial conditions. Models, then, combined to produce a probabilistic forecast Ensemble forecasting is a method used by modern operational forecast centers to account for uncertainties and errors in the forecasting system which are crucial for the prediction errors due to the chaotic nature of the atmospheric dynamics (sensitive dependency on initial conditions). Many different models are created in parallel with slightly different initial conditions or configurations. These models are then combined to produce a forecast that can be fully probabilistic or derive some deterministic products such as the ensemble mean. 11/12/2018

Introduction How does the system work? Main objective of the overall project: Grid: enables rapid computation of parallel tasks. WRF does not currently work on a Grid. This is the main objective of this project. The current version of WRF has been designed to run either on a single machine or on a cluster of homogeneous nodes. The high resource requirements of WRF for fine resolution and ensemble forecasting demand a larger number of computing nodes with substantial memory and disk sotrage connected throgh a high speed network. The solution is greed computing 11/12/2018

Architectural Overview of the Overall Project Meteorologist Meteorologist BSC FIU 1 2 3 4 5 6 7 Web-Based Portal Web-Based Portal Peer-to-peer Protocols 1 2 3 5 7 1 4 6 Job-Flow Manager Job-Flow Manager Meta-Scheduler Meta-Scheduler 1 2 3 5 7 1 4 6 Local scheduler Local scheduler Resource Policies Resource Policies Local scheduler Local scheduler Local Resources Local Resources Local Resources Local Resources 7

Introduction What is the scope of what we are doing? The WRF Web Portal Front End Ease of use for meteorologists 11/12/2018

Layered Architecture Aggregate 3 phases: diamond – 5 -6 Provide interface for meteorologist w/ list for diff option how to create perturbed input file Define ensemble members Perturbation algorithm Agregation algorithm 2 goal: interface w/ the job-flow manager define API to interact w/ job flow manager (receive a work flow document response return result of execution) Give job flow manager XML that contains pertubation and aggregation algorithm and # of ensemble members 9

Motivation Help meteorologist focus on their science Make visualization easier Speed up Ensemble Forecasting The motivation for creating this portal is to provide a comprehensive, easy to use, high-level tool for meteorologist to simulate and perform ensemble runs using the WRF forecasting model. Likewise the idea is to help scientists visualize the output statically or dynamically via MPEG/GIF or Google maps with relevant information where one can zoom in and out, respectively.   This will help the meteorologists, who are currently occupied with the particulars of the computer science aspect of weather forecasting, concentrate on their own science. With a readily available tool created for them, a meteorologist does not have to be bothered by the business logic of the program; much in the same way a person who uses a telephone does not need to be aware of the circuitry inside to use it to his or her advantage. In this way the science of meteorology will not be hindered by the cumbersome task of creating tools for themselves, but rather can concentrate on their own science and achieve research and other scientific goals quicker and more efficiently. 11/12/2018

Previous work Transparent Grid Enablement of Weather Research and Forecasting S. Masoud Sadjadi, Liana Fong, Rosa M. Badia, Javier Figueroa, Javier Delgado, Xabriel J. Collazo-Mojica, Khalid Saleem, Raju Rangaswami, Shu Shimizu, Hector A. Duran Limon, Pat Welsh, Sandeep Pattnaik,`Anthony Praino, David Villegas, Selim Kalayci, Gargi Dasgupta, Onyeka Ezenwoye, Juan Carlos Martinez, Ivan Rodero, Shuyi Chen, Javier Muñoz, Diego Lopez, Julita Corbalan, Hugh Willoughby, Michael McFail,Christine Lisetti, and Malek Adjouadi1 Towards a Self-Configurable Weather Research and Forecasting System By Khalid Saleem, S. Masoud Sadjadi, Shu-Ching Chen Add other papers 1 goal: Portal provide appropirate intrefaces for metoriologist to be able to specify what I s the ensemble that he would like to run How many esnsemble members Ensemble members: what input they should receive (how it hould be perturen bed) How output should be aggregated Goal: web based portal to compose workflow that encapsulates the Aggregate 3 phases: diamond – 5 -6 Provide interface for meteorologist w/ list for diff option how to create perturbed input file Define ensemble members Perturbation algorithm Agregation algorithm 2 goal: interface w/ the job-flow manager define API to interact w/ job flow manager (receive a work flow document response return result of execution) Give job flow manager XML that contains pertubation and aggregation algorithm and # of ensemble members Generate worklfow pass it to job flow mananger Ask job flow mangaer status of worklfow Get result of document 3 goal: to be able to keep the profile of user w/ resepct to what are the ensemble user fiends When they were run (history) User logs out, system should still be running keep results of past results 11/12/2018

Description of the Project Grid WRF Portal (Web based portal ) Ensemble Hurricane Forecast using WRF - Specify WRF model configurations - Run simulation for the domains defined. Hurricane Turbulence Study 11/12/2018

Description of the Project Grid WRF Portal Definition Ensembles Upload a configuration file Define quantity of members What input should be perturbed, and how they should be perturbed How the output should be aggregated 11/12/2018

Description of the Project Grid WRF Portal Allow Definition of New Ensembles Change Configuration Parameters Simulate Weather Forecast Results Visualize Weather Forecast Results 11/12/2018

WRF conf Netcdf file Netcdf file WRF conf WPS or WRF SI 1 2 3 4 5 6 7 Web Based Portal WRF conf 1 2 3 4 5 6 7 Job-Flow Manager Netcdf file Meta-Scheduler Netcdf file Walk through idea (literal) GCB 11/12/2018 GCB Vis Mind FSU UNF WRF conf WPS or WRF SI

Current System Walk through idea (literal) 11/12/2018

Goals of the Project 1st Goal: Portal provides an appropriate interface for meteorologists to be able to specify what is in the ensemble that would like to be run Amount of ensemble members What input should the ensemble members receive (or, How should they be perturbed?) Perturbation algorithms: i.e. random, uniformly, normally, log-normally, etc. How the output should be aggregated Aggregation Algorithms: i.e. Mean, median, max, min, etc. 1 goal: Portal provide appropirate intrefaces for metoriologist to be able to specify what I s the ensemble that he would like to run How many esnsemble members Ensemble members: what input they should receive (how it hould be perturen bed) How output should be aggregated Goal: web based portal to compose workflow that encapsulates the Aggregate 3 phases: diamond – 5 -6 Provide interface for meteorologist w/ list for diff option how to create perturbed input file Define ensemble members Perturbation algorithm Agregation algorithm 2 goal: interface w/ the job-flow manager define API to interact w/ job flow manager (receive a work flow document response return result of execution) Give job flow manager XML that contains pertubation and aggregation algorithm and # of ensemble members Generate worklfow pass it to job flow mananger Ask job flow mangaer status of worklfow Get result of document 3 goal: to be able to keep the profile of user w/ resepct to what are the ensemble user fiends When they were run (history) User logs out, system should still be running keep results of past results 11/12/2018

Goals of the Project 2nd Goal: Define an API to interact with the job-flow manager. Action: Send a work flow document Response: Return result of the execution Generate workflow, pass it to the job-flow manager Ask job flow manager status of the workflow Get results of execution 1 goal: Portal provide appropirate intrefaces for metoriologist to be able to specify what I s the ensemble that he would like to run How many esnsemble members Ensemble members: what input they should receive (how it hould be perturen bed) How output should be aggregated Goal: web based portal to compose workflow that encapsulates the Aggregate 3 phases: diamond – 5 -6 Provide interface for meteorologist w/ list for diff option how to create perturbed input file Define ensemble members Perturbation algorithm Agregation algorithm 2 goal: interface w/ the job-flow manager define API to interact w/ job flow manager (receive a work flow document response return result of execution) Give job flow manager XML that contains pertubation and aggregation algorithm and # of ensemble members Generate worklfow pass it to job flow mananger Ask job flow mangaer status of worklfow Get result of document 3 goal: to be able to keep the profile of user w/ resepct to what are the ensemble user fiends When they were run (history) User logs out, system should still be running keep results of past results 11/12/2018

Goals of the Project 3rd Goal: To be able to keep the user’s profile with respect to: What are the user’s ensembles When the ensembles were run (history) Keep results of past runs A logged out user’s runs should still be running on the system 1 goal: Portal provide appropirate intrefaces for metoriologist to be able to specify what I s the ensemble that he would like to run How many esnsemble members Ensemble members: what input they should receive (how it hould be perturen bed) How output should be aggregated Goal: web based portal to compose workflow that encapsulates the Aggregate 3 phases: diamond – 5 -6 Provide interface for meteorologist w/ list for diff option how to create perturbed input file Define ensemble members Perturbation algorithm Agregation algorithm 2 goal: interface w/ the job-flow manager define API to interact w/ job flow manager (receive a work flow document response return result of execution) Give job flow manager XML that contains pertubation and aggregation algorithm and # of ensemble members Generate worklfow pass it to job flow mananger Ask job flow mangaer status of worklfow Get result of document 3 goal: to be able to keep the profile of user w/ resepct to what are the ensemble user fiends When they were run (history) User logs out, system should still be running keep results of past results 11/12/2018

Goals of the Project Applying Autonomic Computing Use of self-Configuration policies for dynamic configuration of WRF ensemble runs across the Grid. 11/12/2018

Expectations Develop a prototype which actually allows the meteorologist to run the simulation and visualizes the results. Implement self-configuration policies. The publication of a paper. 11/12/2018

Project Plan Required Analysis Document (Jun 2nd) System Design Document (Jun 23th) Object design Document (Jun 30th) Implementation Final Paper (Aug 4th) 11/12/2018

Questions ? 11/12/2018