1. Advanced Weather Interactive Processing System (AWIPS II) Overview for Development Collaboration Workshop September 29, 2010
Ed Mandel, Steve Schotz, & Jim Calkins
NWS – Office of Science and Technology

2. AWIPS II Technology Infusion Scope and Milestones AWIPS II Migration
AGENDA
AWIPS II Technology Infusion Scope and Milestones
AWIPS II Migration
Approach, Status and Latest Schedule
AWIPS II Architecture Overview
Major Software Components
Communications Interfaces
Standards and Formats
AWIPS II Extended Projects
AWIPS II Software Development Strategy in the AWIPS II Era

3. AWIPS II Technology Infusion Scope
AWIPS II Technology Infusion (FY2005 – FY2015)
A long-term project which delivers a modern, robust software infrastructure that provides the foundation for future system level enhancements for the entire NWS enterprise
Phase 1: (FY2006-FY2011)
Migration of WFO/RFC AWIPS (AWIPS I) to a modern Service Oriented Architecture (SOA) infrastructure executed incrementally through a series of task orders
Phase II: (FY2009-FY2013) – AWIPS SOA Extension
Creation of a seamless weather enterprise spanning NWS operations
Migration of NAWIPS into the AWIPS ISOA
Delivery of thin client to support for the Weather Service Offices, Center Weather Support Units, Incident Meteorologists, (e.g., Fire Weather, backup support for RFCs and National Centers)
Integration of Weather Event Simulator)
CHPS Integration into AWIPS SOA
Phase III: (FY2009 – FY2015) – Enterprise Level Enhancements
Data delivery enhancements: “Smart push-smart pull” data access
Integrated visual collaboration
Information generation enhancements
Visualization enhancements

4. AWIPS II Migration Re-Architecture Approach
Perform “black-box” conversion
Preserve existing functionality, look and feel on top of new infrastructure
Thorough field validation and acceptance before deployment
No loss of functionality
Deployed system current with deployed AWIPS capability (i.e., OB9)
Use open source projects - No proprietary code
Objective is to make AWIPS II available for collaborative development

5. AWIPS II Migration Status
Major Task Orders
Delivery Date
Develop of AWIPS I SW Product Improvement Plan (TO1)
June 2006
Conduct Initial system analysis (TO2)
October 2006
Develop ADE/SDK (TO3-6)
July 2007
Plan baseline application migration (TO7)
October 2007
Migrate primarily D2D/Warngen capabilities migrate (TO8)
February 2008
Migrate primarily GFE capabilities (TO9)
September 2008
Migrate primarily hydrologic capabilities and infrastructure improvements (TO10)
February 2009
Complete AWIPS SOA Release 1.0 (TO11)
May 2010
Begin Operational Test and Evaluation (OTE)
Begin deployment
September 2011

6. AWIPS-II: Reference Architecture Based on Plug In Extensible services
Enterprise Service Bus (ESB) - Camel
<<Service Framework >>
EDEX
<<Visualization Framework >>
CAVE
Core Generic Services
+Eclipse RCP PlugIns
+Data Animation
+GIS Raster Rendering
+Data Editing
+GIS Vector Rendering
+Data Analysis
+GIS Point Rendering
+Drawing
+Ingest Service
+Persist Service
+Metadata Service
+Subscription Service
+Request/Transform Service
+Adapter Service
+Collaboration Service
+Disseminate
Meta
Data
Core Libraries
Data

7. <<Java>> <<Java>> <<abstract>>
SOA Architecture – Logical Layered View Layers Separated by Simple APIs
Client/Presentation Services
JMX
CAVE
External Programs
Mission Services Layer
ProductSrv
Ingest Services
NotifySrv
Hydro Models
AdapterSrv
GFE Services
TextDBSrv
UtilitySrv
PurgeSrv
LAPS
Hydro Services
Security Services /Demilitarized Zone (DMZ)
Enterprise Service Bus - Communication
FORTRAN/C/C++
Command Line
Programs
Data Access Layer
<<Java>>
HDF5DataStore
<<Java>>
DataLayer
<<abstract>>
BaseDao
HDF5 API
Hibernate
Synchronous
Interface
Platform Layer
Localization
Store
HDF5
PostgreSQL
Data Persistence
Store
Metadata
Index
Spatial
Index

8. AWIPS II Dependencies: Open Source Based Free of All Fee-Based COTS Dependencies
CAMEL + Spring
Enterprise service bus and dependency injection container for SOA services
Eclipse RCP
Plug In driven visualization framework
GeoTools
Enables GIS capabilities and map projection framework
JOGL
Java API to OpenGL enables Gaming level visualization performance
HDF5
High performance file persistence of large data sets such as satellite and radar
Python + numPY
Data transform scripting languages with high performance math library
jepp+
Java to cPython JNI adapter extended for numpy
JAVA + ANT
Primary programming language and software build framework
Apache qpid
AMQP standard messaging with JMS api
PostgreSQL
Relational database for storing Metadata from Data plug ins and spatially enables ingested data

9. AWIPS II Deployment Standards & Formats
Ingested Data Formats
GRIB1/GRIB2 – Gridded data
NetCDF3 – Support AWIPSI/II interoperability
BUFR – Observational Data, e.g., soundings
METAR, SHEF – Surface and hydrological data
GINI – Satellite Imagery
OPRG L3 – Radar Imagery
Text Messages – Text products
Data Store Formats
PostGres – Metadata and select data type store, e.g., text
HDF5 – Binary store for grids, imagery and select observations
Product Distribution
NetCDF3 – NDFD Grids
ASCII Text – Text products

10. AWIPS II Extended

11. AWIPS II Extended Projects
Phase
Project
Target Completion
Extend AWIPS II Infrastructure to Entire NWS Forecaster Enterprise
NAWIPS Migration Deploy
4Q11
Thin Client Deploy
3Q12
WES Bridge Deploy
WES Integration Deploy
4Q11 (Deployed with AWIPS II)
4Q12
Implement System-Wide Enhancements to Benefit Entire AWIPS Enterprise
Data Delivery – IOC
Data Delivery - FOC
2Q13
FY13-14
Collaboration – Phase I
Collaboration – Phase II
Collaboration - Phase III
FY14-15
Information Generation FOC
FY14
3D Visualization IOC

12. AWIPS II Extended - NAWIPS Migration
Objectives
Incorporate NAWIPS capabilities into AWIPS SOA
Key Benefits
Enables more cost-effective software development environment for common requirements across the NWS enterprise; Helps enable software agility
Lays ground work for more effective collaboration and seamless products across NWS enterprise (NCs, WFOs, RFCs, CWSUs) by providing common infrastructure
Strengthens NWS partnership with University Community by providing entire suite of AWIPS capabilities – Helps enable more efficient R to O and O to R
Improves support for OCONUS Regions by providing common software infrastructure for NAWIPS and AWIPS functionality
Potential Synergies with Other Projects
Provides opportunities to leverage/share NAWIPS capabilities with broader AWIPS user customer base
Grid/Ensemble diagnostics
Drawing/Product generation tools to support collaboration requirements

13. AWIPS II Extended - AWIPS II Thin Client
Objectives
Develop enterprise solution for remote access to AWIPS capabilities
Key Benefits
Incorporate FX-Net capabilities into baseline
Provides common solution to address remote access requirements to support Incident Meteorologists, e.g., Fire Wx, Weather Service Offices, etc.
Allows remote users to have access to latest set of AWIPS capabilities
Provides partial solution for COOP scenarios at NCs and RFCs
In combination with Data Delivery Project, allows us to begin to analyze future opportunities for utilizing cloud computing

14. AWIPS II Extended - Weather Event Simulator (WES)
Objectives
Develop enterprise solution to support NWS training requirements
Phase I: Provide bridge for current WES capability into AWIPS II. Also serves as a pathfinder for Phase II
Phase II: Develop integrated solution for AWIPS enterprise
Key Benefits
Provides robust, sustainable baseline solution to support NWS training requirements
Allows training users to have access to latest built set of AWIPS capabilities
Provides training solution for all AWIPS applications

15. AWIPS II Extended - Data Delivery
Objectives
Develop robust data delivery system within AWIPS II infrastructure that enables efficient access to high volume datasets
Develop operational robust infrastructure to support “intelligent” access to non-local datasets
User defined sub-setting by space, time, and parameter
Subscription or Ad-hoc access methods based on weather events
Key Benefits
Supports high impact based forecast and decision assistance processes by allowing users to access just the data they need by space, time, parameter; Enables more efficient data mining
Enables effective on-demand access to Weather Information Database
Enables synergy and interoperability with NextGen technologies, e.g., data discovery services, data access services and data providers.
Mitigates high growth in data volume e.g. ensembles, high-resolution models
Multi-Phase Implementation
IOC Focus – NWS data providers, e.g., NOMADS, MADIS, possibly with basic services only, discovery, sub-setting – Target FY13

16. AWIPS II Extended Data Delivery Overview -Continued
Synergies with NextGen (4-D Cube)
Data registry and discovery services
“Smart” push/pull technology
Sub-setting by user selectable space, time, and parameter
Complex retrievals, e.g., derived parameters, coordinate transformations, etc
Ad hoc and subscription services
Operationally robust – supports availability, latency and security requirements for operational users
Plan to leverage NextGen services including Reg/Rep, WCS, WFS
Potential Synergies with Other Projects
IRIS for data access/distribution
IHIS for data access/distribution
GOES-R and NPP/NDE – Data Provider and data access/distribution services
NWS Infrastructure Project – Central Data Server services

17. Concept of Operations Data Delivery Paradigm
Three Main Components: the Data Delivery Agent, the Registry/Repository and the Data Providers 17

18. Data Delivery High Level Architecture

19. AWIPS II Extended - Collaboration
Objectives
Develop capabilities to support real-time collaboration
Phase I: Develop/Enhance internal NWS collaboration capabilities
Phase II: Develop/Enhance IOC collaboration capabilities with external partners
Phase III: Improve collaboration capabilities with external partners
Key Benefits
Enables more effective collaboration across all levels of NWS promoting a more coordinated and seamless set of products and services – Fosters consistency of NWS products and services
Enables interoperability between NWS and decision makers, e.g., emergency managers to support Decision Support Services
Potential Synergies with Other Projects
IRIS for collaboration with external partners
IHIS for collaboration with external partners
Canadian Environment Services /NWS Marine Forecast Collaboration Project
Weather and Emerg. Management Decision Support (RENCI) Project for collaboration with North Carolina Emergency Managers

20. Phase I Collaboration ConOps
Data display sharing
This capability will allow participating NWS offices to view each other’s AWIPS data displays during collaboration sessions
Telestrator Functions
Simple line drawing capabilities, that will allow forecasters to draw and erase temporary lines and polygons on their AWIPS displays that can be viewed by participating collaborators.
Telestrator functions will help forecasters identify and discuss specific features of interest, e.g., satellite and radar storm signatures, etc.
Creation and editing of hydrometeorological objects
Creation, editing and exchange of geo-referenced objects and products such as fronts, areas of severe weather, troughs and ridges, etc. and graphical products such as watches and warnings.
Meteorological objects and products will be displayed and/or exchanged among the participating collaborators allowing participants to import these graphics and display them in AWIPS applications. The display and exchange of meteorological objects and products will allow sites to collaborate in real-time on object or product features such as location, shape, and intensity.
Object and product attributes such as front type, product type, etc; will also be saved so that they are available for forecaster access and editing.
An integrated text chat capability that will allow forecasters to chat during the collaboration session.

21. AWIPS II Extended – Information Generation
Objective
Develop infrastructure to support common set of information generation services and tools to support decision assistance
Key Benefits
Enables AWIPS tools and Decision Maker tools for decision support
Streamlines generation of products to support emerging industry standards, e.g., CAP and GIS
Reduces development time associated with the introduction of new products and services
Enables generation of NWS products from both local data bases and the 4-D weather Data Cube
Opportunities for synergy with IRIS project
Potential Synergies with Other Projects
IRIS vision is to provide a enterprise solution for information storage for NWS and decisions makers access
Initial Capabilities Encapsulated in IHIS Project

22. AWIPS II Extended - Advanced Visualization
Objective
Develop advanced visualizations techniques, e.g., 3-D visualization that can be utilized by AWIPS baseline applications
Key Benefits
Enables more effective forecaster visualization of datasets to support forecast and decision making processes.
Methods to improve situational awareness
Methods to improve identification of factors that affect high impact events

23. AWIPS II Software Development Strategy In the AWIPS II Era

24. AWIPS II Software Development Strategy
Vision –
Enable collaborative AWIPS II Environment to effectively incorporate new science and technology into AWIPS II operational baseline from NOAA development organizations, proving grounds and test beds, and academic and research communities.
AWIPS II becomes the premium “Open Source” software platform for the hydrometeorological community
Supporting Objectives
Enable effective user and developer collaboration to support new capability development and enhancements
Enable agile prototyping and development in the AWIPS Development Environment (ADE) to reduce necessary software development time and cost to “productionize” new science and technology into AWIPS baseline
Provide timely and effective feedback to AWIPS program to ensure AWIPS architecture refresh plans keeps pace with new science and technologies

25. Software Agility Strategy - Continued
Software Agility Requirements
Provide well- defined common libraries and toolkits to support data access, hydrometeorological calculations, visualization, and information generation
Provide well-documented AWIPS Development Environment (ADE) to development community
Provide effective training and developer support to development community
Develop streamlined governance processes balancing software agility needs with necessary software engineering discipline to ensure high quality software infusion into AWIPS baseline
Provide agile AWIPS test environment and methods including regression and automated test tools
Establish OST in-house AWIPS II software development expertise to provide effective guidance on all phases of the software development life cycle to AWIPS stakeholders

26. Software Development Strategy Near-Term Activities and Plans
Spin-Up OST AWIPS II development team using support contract mechanism
Team will be the kernel of OST AWIPS II software expertise
Government Team chartered - July 2010
Contractor Team Tasking – October 2010
Develop AWIPS II software developers documentation, training and support
RTS tasking planned – FY11
Improve AWIPS II testing environment
Regression Testing Task – on Contract
Automated Testing Task – FY11
Data Pump – FY11 26

27. SEC AWIPS Software Development Team
Mission - The AWIPS Software Development Team (ASDT) shall serve as the center of expertise for AWIPS II Software for entire AWIPS II enterprise.
The ASDT Roles and Responsibilities
Leading the evaluation and guidance for all phases of software development life cycle from requirements analysis to testing
Managing and/or developing AWIPS II software infrastructure changes and additions
Developing select AWIPS applications
Facilitating new science and technology infusion into AWIPS operations.
Staffing and Resource Plan
Government Team of five led by Jim Calkins
Contractor Team of five utilizing Chugach contract (Same contractor that supports NCEP)
Addition of 6 NCEP contractors ~January 2011

28. ASDT Near-Term Activities
Team Activities
Establishing AWIPS II development environment patterned after RTS Omaha and NCEP development environments
Subversion – CM/Version Control
TRAC – Task tracking and management
Hudson – Software build support
Drafting AWIPS II development life cycle processes
Initial focus on supporting local application migration

29. QUESTIONS ????