NOAA’s NWS Telecommunication Gateway RTH Washington ICM MTN ET-OI, Sep 2008 Office of the Chief Information Officer NOAA’s National Weather Service Office of the Chief Information Officer NOAA’s National Weather Service

“Taking the pulse of the planet” Data Input to the NWSTG  MPLS  IP / Sockets  X.25  Asynchronous  FTP –  –  Web -  MPLS  IP / Sockets  X.25  Asynchronous  FTP –  –  Web -

“Taking the pulse of the planet” Replacement & Backup RTHs  Full functional replacement of existing capabilities  Expanded capacity and capability –Input data growth from 200 to 800 GB/day –Output data growth from 800 to 2400 GB/day  Transition to new technology –Middleware for internal transport –Network-centric systems interconnectivity –Central switching engine with relational database –SAN and NAS storage solutions  Highly scaleable architecture  Hardware refresh  Full functional replacement of existing capabilities  Expanded capacity and capability –Input data growth from 200 to 800 GB/day –Output data growth from 800 to 2400 GB/day  Transition to new technology –Middleware for internal transport –Network-centric systems interconnectivity –Central switching engine with relational database –SAN and NAS storage solutions  Highly scaleable architecture  Hardware refresh

“Taking the pulse of the planet” RTG/BTG Performance Performance Metric Threshold System Availability (averaged monthly) 99.90% Warning Message Latency (averaged monthly) 10 seconds Routine Message Latency (averaged monthly) 60 seconds Daily Traffic Volume (averaged monthly) 1.2TB Typical Performance Results (averaged monthly) Performance Category Threshold Actual System Availability 99.90%100% Warning Message Latency10 seconds< 1 second Routine Message Latency60 seconds< 1 second Daily Traffic Volume1.2TB TB Performance Metric Threshold System Availability (averaged monthly) 99.90% Warning Message Latency (averaged monthly) 10 seconds Routine Message Latency (averaged monthly) 60 seconds Daily Traffic Volume (averaged monthly) 1.2TB Typical Performance Results (averaged monthly) Performance Category Threshold Actual System Availability 99.90%100% Warning Message Latency10 seconds< 1 second Routine Message Latency60 seconds< 1 second Daily Traffic Volume1.2TB TB

“Taking the pulse of the planet” RTH Washington  Can route more than 50 routine messages per second with reliability for all dissemination to all of its users of 99.9 percent  Latency for high priority traffic of 10 seconds or less and routinely disseminates 1.2 terabytes of information per day  On Main Trunk Network (MTN) of the WMO Global Telecommunication System (GTS) –GTS delivers tsunami data and warnings to connected MTN centers within two minutes  On AFTN network (KWBCYMYX) & ICAO OPMET Databank  Can route more than 50 routine messages per second with reliability for all dissemination to all of its users of 99.9 percent  Latency for high priority traffic of 10 seconds or less and routinely disseminates 1.2 terabytes of information per day  On Main Trunk Network (MTN) of the WMO Global Telecommunication System (GTS) –GTS delivers tsunami data and warnings to connected MTN centers within two minutes  On AFTN network (KWBCYMYX) & ICAO OPMET Databank

“Taking the pulse of the planet” NOAANet Logical Design

“Taking the pulse of the planet” Primary / Backup NWSTG Access

“Taking the pulse of the planet”

Major Software Components Core software components of the NWSTG replaced in 2006  Data switching software –Core software rewritten and implemented in IBM WebSphere Message Broker –Message Broker provides an application execution environment, threading, and fourth generation language support (ESQL), integrated with middleware services  Data transport infrastructure (middleware) –Shared data stores replaced with IBM MQSeries Middleware –Applications send data to each other without concern for target location –Data delivery guaranteed even during failures  Data storage infrastructure (relational database) –All data stores moved into Sybase relational database Core software components of the NWSTG replaced in 2006  Data switching software –Core software rewritten and implemented in IBM WebSphere Message Broker –Message Broker provides an application execution environment, threading, and fourth generation language support (ESQL), integrated with middleware services  Data transport infrastructure (middleware) –Shared data stores replaced with IBM MQSeries Middleware –Applications send data to each other without concern for target location –Data delivery guaranteed even during failures  Data storage infrastructure (relational database) –All data stores moved into Sybase relational database

“Taking the pulse of the planet” Internal Redundancy Internal redundancy implemented for critical processes  Active – Warm Standby –Standby system running at all times, system disks moved to standby system during failover –EXAMPLES: Individual front-end processors, application servers  Active – Hot Standby –Data replicated to Hot Standby system in realtime, automatic failover –EXAMPLE: Sybase relational database servers  Active - Active –Multiple systems active in parallel, data shared/routed between systems groups –EXAMPLES – Switching system, HTTP/FTP server groups Internal redundancy implemented for critical processes  Active – Warm Standby –Standby system running at all times, system disks moved to standby system during failover –EXAMPLES: Individual front-end processors, application servers  Active – Hot Standby –Data replicated to Hot Standby system in realtime, automatic failover –EXAMPLE: Sybase relational database servers  Active - Active –Multiple systems active in parallel, data shared/routed between systems groups –EXAMPLES – Switching system, HTTP/FTP server groups

“Taking the pulse of the planet” Architecture Overview

“Taking the pulse of the planet” Architecture Overview

“Taking the pulse of the planet” System Extensibility Virtualized server hardware platform selected for extensibility  Virtualization –System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.) available within hardware frame –Individual resources selected to create virtual servers –Major storage implemented in Storage Area Network –Virtual disks allocated to virtual servers  Extensibility –Unused resources can be allocated to virtual servers, as needed –Underused resources can be removed from a virtual server –Virtual disks can be extended or reduced as storage requirements change Virtualized server hardware platform selected for extensibility  Virtualization –System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.) available within hardware frame –Individual resources selected to create virtual servers –Major storage implemented in Storage Area Network –Virtual disks allocated to virtual servers  Extensibility –Unused resources can be allocated to virtual servers, as needed –Underused resources can be removed from a virtual server –Virtual disks can be extended or reduced as storage requirements change

“Taking the pulse of the planet” System Extensibility Virtualized server hardware platform selected for extensibility  Advanced Virtualization –Network and SAN I/O interfaces can be shared through virtual I/O –CPUs can be carved into fractional components down to 1/10 th of a CPU  Advanced Extensibility –Resources (CPU, Memory) included within system that were not purchased –If additional resources are needed, resources are “turned on” via software key after purchase and are immediately available Virtualized server hardware platform selected for extensibility  Advanced Virtualization –Network and SAN I/O interfaces can be shared through virtual I/O –CPUs can be carved into fractional components down to 1/10 th of a CPU  Advanced Extensibility –Resources (CPU, Memory) included within system that were not purchased –If additional resources are needed, resources are “turned on” via software key after purchase and are immediately available

“Taking the pulse of the planet” RTH Washington WIS approach RTG/BTG upgrades made with knowledge of coming WIS/GISC requirements –Production and Replicated core systems All functions not essential to mission critical operations (i.e. switching) execute against replicant –SOA Approach to design –Integral file, message and application switching –RDBS core design ~ 2000 destinations defined ~ 400K defined products ~ 10M entries (rows) in switching table –Many tables used for managing bulletin, file and report storage including dissemination and services metadata. RTG/BTG upgrades made with knowledge of coming WIS/GISC requirements –Production and Replicated core systems All functions not essential to mission critical operations (i.e. switching) execute against replicant –SOA Approach to design –Integral file, message and application switching –RDBS core design ~ 2000 destinations defined ~ 400K defined products ~ 10M entries (rows) in switching table –Many tables used for managing bulletin, file and report storage including dissemination and services metadata.

“Taking the pulse of the planet” RTH Washington WIS approach We view Data Access & Retrieval (DAR) service as the only new service provision –Plan to lay this service onto existing architecture –Will break up DAR functions DAR catalog creation & update discovery & access servicing –DAR will not execute on primary production system (at least in early stages of implementation We view Data Access & Retrieval (DAR) service as the only new service provision –Plan to lay this service onto existing architecture –Will break up DAR functions DAR catalog creation & update discovery & access servicing –DAR will not execute on primary production system (at least in early stages of implementation

“Taking the pulse of the planet” RTH Washington WIS approach Develop metadata catalog first in conjunction with national & regional partners –Will leverage knowledge expertise of partners –Will expand to all RTH (GISC) data holdings and holdings of appropriate Centers (RSMC/DCPC/NC) and willing partners. –Work will be made available to all Members –Will collaborate with WIS Project Team, WMO ETs and developers in other Regions –Regional WIGOS Development Project will meet WIS requirements RA-IV Integrated Atmosphere Observing System With initial emphasis on upper-air observations Develop metadata catalog first in conjunction with national & regional partners –Will leverage knowledge expertise of partners –Will expand to all RTH (GISC) data holdings and holdings of appropriate Centers (RSMC/DCPC/NC) and willing partners. –Work will be made available to all Members –Will collaborate with WIS Project Team, WMO ETs and developers in other Regions –Regional WIGOS Development Project will meet WIS requirements RA-IV Integrated Atmosphere Observing System With initial emphasis on upper-air observations

“Taking the pulse of the planet” Meteorological Assimilation Data Ingest System (MADIS) Transition to Operations SFC-LAND SFC-MARINE U/A-IN SITU U/A-REMOTE SENSING SATELLITE GRIDS METADATA MADIS Collection, QC, and Distribution Information Bases (QCed Datasets) Quality Control Information (Data QC Flags) And Metadata PM 12/12/05

“Taking the pulse of the planet” MADIS Ingests and Performs QC on Meteorological Observational Data  Meteorological Surface –METAR –Airways –Maritime –Modernized NWS Cooperative Observer –UrbaNet –Integrated Mesonet State DOT Mesonets AWS  RSAS Surface Grids  Meteorological Surface –METAR –Airways –Maritime –Modernized NWS Cooperative Observer –UrbaNet –Integrated Mesonet State DOT Mesonets AWS  RSAS Surface Grids  NOAA Profiler Network  Hydrological Surface  Automated Aircraft  Multi-Agency Profiler  Cooperative Agency Profiler  Radiosonde  Radiometer  Satellite Wind –NOAA GOES Products  Satellite Radiances and Soundings –NOAA POES  Snow

“Taking the pulse of the planet” TOC Environment Gateway MADIS-T FTP External ISP LDM FTP Storage (SAN) Automated QC processing Gateway Customers (NCDC, NCF, GTS etc) Data Processing LDM NOAA Customers Non-NOAA Observations LDM Integrated Datasets Web Server Internet Web Server ? ? ? Individual Data Products Tunnel

“Taking the pulse of the planet” RTH Washington Other Issues –Foundation of Programs is Observing Systems & Data Exchange –Transition of GTS-MTN Two existing clouds to ??? –Cloud 1 - frame relay –RMDCN – satellite MPLS –Addition of Guam/Hono to AP-RARS –Future operations locations Backup first then Primary –Foundation of Programs is Observing Systems & Data Exchange –Transition of GTS-MTN Two existing clouds to ??? –Cloud 1 - frame relay –RMDCN – satellite MPLS –Addition of Guam/Hono to AP-RARS –Future operations locations Backup first then Primary

© Crown copyright Met Office GTS Link with NOAA-TOC Washington Inbound flow may be inhibited due to the saturated Outbound flow being unable to return IP sockets ‘ack’ packets quickly enough.

“Taking the pulse of the planet” Additional Slides

“Taking the pulse of the planet” NWSTG Functional Overview  The NWSTG is:  the central communication facility of the NWS;  the primary acquisition and distribution center for NWS data and products;  the primary acquisition and distribution center for international data and products to meet WMO, ICAO and bi-laterally agreed US requirements;  A major data exchange hub for NOAA and other agency data and products.  The NWSTG is:  the central communication facility of the NWS;  the primary acquisition and distribution center for NWS data and products;  the primary acquisition and distribution center for international data and products to meet WMO, ICAO and bi-laterally agreed US requirements;  A major data exchange hub for NOAA and other agency data and products.

“Taking the pulse of the planet” NWSTG Functional Overview The NWSTG includes  WMO Regional Telecommunication Hub (RTH) Washington  ICAO OPMET Databank  the ASOS Operations and Monitoring Center Operational oversight of U.S. federal automated surface observing systems  the AWIPS Network Control Facility AWIPS is the main NWS’ system which supports the NWS forecast and warning mission requirement The NCF is the central communications hub and technical support center for AWIPS The NWSTG includes  WMO Regional Telecommunication Hub (RTH) Washington  ICAO OPMET Databank  the ASOS Operations and Monitoring Center Operational oversight of U.S. federal automated surface observing systems  the AWIPS Network Control Facility AWIPS is the main NWS’ system which supports the NWS forecast and warning mission requirement The NCF is the central communications hub and technical support center for AWIPS

“Taking the pulse of the planet” Dissemination Systems  GTS  ISCS  EMWIN  Internet  GMDSS  NWS Telecom. Gateway  GTS  ISCS  EMWIN  Internet  GMDSS  NWS Telecom. Gateway  NWWS  NWR  LDAD  Family Of Services (FOS)  Interagency Connections  AWIPS SBN / NOAAPORT International National

“Taking the pulse of the planet” Dissemination and Distribution NWSTG FOS SBN/NOAAPORT GOES NCEP Product Suite NWWS (10 sec) WAN NESDIS Internet-Based Dissemination EMWIN (> 1 min) Telecommunications Operations Center Satellite Legend: EMWIN NOAAPORT NWWS Commercial Satellite GOES NWR WSR-88D Field Offices NCEP Data Servers NCF Media & Other Customers Public Specialized Customers Other Agencies LDAD Designated Local Customers Commercial Weather Services Research Institutions FAA, etc. ISCS GTSEMWIN Private line Dissemination Observations Imagery WMO ICAO

“Taking the pulse of the planet”

Geographically separated backup system Replacement / Backup TG Description Worldwide users ICAO / WMO Govt. agencies Family of services Internet users Foreign countries Emergency mgrs NWS users Nat’l Centers for Environ. Prediction NWS Regional Offices Domestic/int’l observation & forecast offices AWIPS NWS Telecom Gateway Located in Silver Spring New message switching system allows future upgrades 2x upgradeability Much improved response time Redundancy ensures uninterrupted service Full configuration management 120GB/ GB 80GB/ GB 90GB/ GB 700GB/ GB Daily Throughput/# of circuits Legacy over Replacement

“Taking the pulse of the planet” System Extensibility

“Taking the pulse of the planet” Backup NWSTG

“Taking the pulse of the planet” Government Networking Requirements - NOAANet IP-based networking solution − Any-to-any connectivity − High degree of bandwidth scalability − Optimum redundancy and survivability − IP convergence (i.e., voice, video and data over IP) − High-end performance Network security remains paramount, particularly in light of today’s socio-political threats Segmentation from the public Internet Minimizes risk of security or privacy breaches

“Taking the pulse of the planet” Sprint Peerless IP Network Map Internet Transport Node OC48 (significant portions upgrading to OC192) Chicago Anaheim San Jose Pennsauken, NJ Kansas City Stockton Cheyenne Fort Worth New York Tacoma Atlanta Relay, MD DC