New Technologies for Storage and Display of Meteorological Data 10 May 2002 New Technologies for Storage and Display of Meteorological Data Dale Paynter Operations Management Group May, 2002 NUMUG Conference

Benefits of Technology? New Technologies for Storage and Display of Meteorological Data 10 May 2002 Benefits of Technology? Move to modern technology path that will be developed and supported Make initial installation, maintenance & upgrades easier Change is the name of the game External requirements & standards Infrastructure changes Organizational re-alignment Meet rising expectations Simple systems interfaces Capitalize on staff knowledge & skills Let Meteorologists manage, analyze and use the data 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 New Technologies What are they? Data Driven System Architecture Relational Databases Web Applications 10 May 2002 OMG

Data Driven Architecture New Technologies for Storage and Display of Meteorological Data 10 May 2002 Data Driven Architecture What is it? Write code to be independent of specifics Make system configurable to define: Input stream Parameters for internal calculations Reports Interfaces to other systems 10 May 2002 OMG

Data Driven Architecture New Technologies for Storage and Display of Meteorological Data 10 May 2002 Data Driven Architecture What are the benefits? Respond to new requirements and standards Reduce dependency on vendors & IT Get out of program change queue Reduce change orders & site visits Give users control through configuring system 10 May 2002 OMG

What does this let you do easily? New Technologies for Storage and Display of Meteorological Data 10 May 2002 What does this let you do easily? Add new sensors with no code changes Create new list reports Change corporate information Revise alarm / flag limits Add or decommission sampling locations Interface to different systems 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 So how do we do this? Parameters, parameters, parameters... Make data base ‘self-descriptive’ Store ‘what’ is being stored as well as ‘values’ Eliminate need to reorganize / reload data to accommodate changes Make variables for all ‘constants’ Tower names Time zones Daylight saving time Measurement codes Report titles & headings Data to display / order Report field formats Conversion formulae 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 Adding a new sensor Traditional “Programming” Approach User configures or reprograms datalogger Programmer adds new field to file formats. Then dumps existing data, reloads and verifies Programmer changes data loading to process new input layout & update new field Programmer changes updating to display / edit new field Programmer changes reporting to display new field System administrator installs final programs on all affected computers Data Driven Architecture Approach User configures datalogger User defines sensor in database User tests and verifies results System administrator copies configurations to production system 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 Relational Databases What are they? Data is stored in Tables made up of rows of fields (columns) Data is referenced through name of the field Subsets of data may be selected Tables may be joined to retrieve related records 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 Relational Databases Usage comparison example Sequential file - 30’ wind speed is in columns 26 to 31 (format F6.2) Typical relational database - 30’ wind speed is stored in a field named WS30 Relational database using data driven architecture - 30’ wind speed is stored in a field named Value where the field MeasCode contains ‘WS’ and Height contains ‘30’ 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 Relational Databases What are the the general benefits? Built-in security Supports simultaneous accesses Built-in data recovery techniques Built-in data retrieval algorithms & optimization Standard database interaction (SQL) In-house expertise can be leveraged 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 Relational Databases And... for meteorological data? Concurrent loading, updating and reporting Permits non-sequential loading for legacy data and data recovery Handles data retention and archiving requirements Ad hoc queries Time-stamping data changes Keep original values as well as edited values 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 Web Applications In general... User requests report and submits parameters through browser Web application generates report on server and sends to browser User views / prints report through the desktop browser Benefits? Removes need to run / maintain specific report programs on desktop Just ask your LAN Administrator… 10 May 2002 OMG

Web Server Configuration New Technologies for Storage and Display of Meteorological Data 10 May 2002 Web Server Configuration Meteorologist’s Workstation running Updater Meteorological Monitoring System Server MMS Database Web Server Reporter Meteorologist’s & Users’ Workstations running Web Browsers InterceptTM & Loader Zenos Sensors Zenos Interface Files Other Servers Zeno®s 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 Web Technology Internet or Intranet Server Data manipulation and report generation performed on server Only results sent to browser Automatic report refresh option Additional security levels possible 10 May 2002 OMG

New Technologies for Storage and Display of Meteorological Data 10 May 2002 How does it perform? Even running on a notebook: Current reports are instantaneous Historical reports take a few seconds Frequency Distribution produces a 1 year report in 12 seconds Data Capture Summary produces a 1 year report in 55 seconds 10 May 2002 OMG

Benefits of Technology New Technologies for Storage and Display of Meteorological Data 10 May 2002 Benefits of Technology Move to modern technology path that will be developed and supported Get out of the program change queue Flexibility to respond to change Let Meteorologists manage, analyze and use the data 10 May 2002 OMG