1. New Technologies for Storage and Display of Meteorological Data
10 May 2002
New Technologies for Storage and Display of Meteorological Data
Dale Paynter
Operations Management Group
May, 2002
NUMUG Conference

2. Benefits of Technology?
New Technologies for Storage and Display of Meteorological Data
10 May 2002
Benefits of Technology?
Move to modern technology path that will be developed and supported
Make initial installation, maintenance & upgrades easier
Change is the name of the game
External requirements & standards
Infrastructure changes
Organizational re-alignment
Meet rising expectations
Simple systems interfaces
Capitalize on staff knowledge & skills
Let Meteorologists manage, analyze and use the data
10 May 2002
OMG

3. New Technologies for Storage and Display of Meteorological Data
10 May 2002
New Technologies
What are they?
Data Driven System Architecture
Relational Databases
Web Applications
10 May 2002
OMG

4. Data Driven Architecture
New Technologies for Storage and Display of Meteorological Data
10 May 2002
Data Driven Architecture
What is it?
Write code to be independent of specifics
Make system configurable to define:
Input stream
Parameters for internal calculations
Reports
Interfaces to other systems
10 May 2002
OMG

5. Data Driven Architecture
New Technologies for Storage and Display of Meteorological Data
10 May 2002
Data Driven Architecture
What are the benefits?
Respond to new requirements and standards
Reduce dependency on vendors & IT
Get out of program change queue
Reduce change orders & site visits
Give users control through configuring system
10 May 2002
OMG

6. What does this let you do easily?
New Technologies for Storage and Display of Meteorological Data
10 May 2002
What does this let you do easily?
Add new sensors with no code changes
Create new list reports
Change corporate information
Revise alarm / flag limits
Add or decommission sampling locations
Interface to different systems
10 May 2002
OMG

7. New Technologies for Storage and Display of Meteorological Data
10 May 2002
So how do we do this?
Parameters, parameters, parameters...
Make data base ‘self-descriptive’
Store ‘what’ is being stored as well as ‘values’
Eliminate need to reorganize / reload data to accommodate changes
Make variables for all ‘constants’
Tower names
Time zones
Daylight saving time
Measurement codes
Report titles & headings
Data to display / order
Report field formats
Conversion formulae
10 May 2002
OMG

8. New Technologies for Storage and Display of Meteorological Data
10 May 2002
Adding a new sensor
Traditional “Programming” Approach
User configures or reprograms datalogger
Programmer adds new field to file formats. Then dumps existing data, reloads and verifies
Programmer changes data loading to process new input layout & update new field
Programmer changes updating to display / edit new field
Programmer changes reporting to display new field
System administrator installs final programs on all affected computers
Data Driven Architecture Approach
User configures datalogger
User defines sensor in database
User tests and verifies results
System administrator copies configurations to production system
10 May 2002
OMG

9. New Technologies for Storage and Display of Meteorological Data
10 May 2002
Relational Databases
What are they?
Data is stored in Tables made up of rows of fields (columns)
Data is referenced through name of the field
Subsets of data may be selected
Tables may be joined to retrieve related records
10 May 2002
OMG

10. New Technologies for Storage and Display of Meteorological Data
10 May 2002
Relational Databases
Usage comparison example
Sequential file - 30’ wind speed is in columns 26 to 31 (format F6.2)
Typical relational database - 30’ wind speed is stored in a field named WS30
Relational database using data driven architecture - 30’ wind speed is stored in a field named Value where the field MeasCode contains ‘WS’ and Height contains ‘30’
10 May 2002
OMG

11. New Technologies for Storage and Display of Meteorological Data
10 May 2002
Relational Databases
What are the the general benefits?
Built-in security
Supports simultaneous accesses
Built-in data recovery techniques
Built-in data retrieval algorithms & optimization
Standard database interaction (SQL)
In-house expertise can be leveraged
10 May 2002
OMG

12. New Technologies for Storage and Display of Meteorological Data
10 May 2002
Relational Databases
And... for meteorological data?
Concurrent loading, updating and reporting
Permits non-sequential loading for legacy data and data recovery
Handles data retention and archiving requirements
Ad hoc queries
Time-stamping data changes
Keep original values as well as edited values
10 May 2002
OMG

13. New Technologies for Storage and Display of Meteorological Data
10 May 2002
Web Applications
In general...
User requests report and submits parameters through browser
Web application generates report on server and sends to browser
User views / prints report through the desktop browser
Benefits?
Removes need to run / maintain specific report programs on desktop
Just ask your LAN Administrator…
10 May 2002
OMG

14. Web Server Configuration
New Technologies for Storage and Display of Meteorological Data
10 May 2002
Web Server Configuration
Meteorologist’s
Workstation
running
Updater
Meteorological Monitoring System
Server
MMS
Database
Web Server
Reporter
Meteorologist’s
& Users’
Workstations
running
Web Browsers
InterceptTM
&
Loader
Zenos
Sensors
Zenos
Interface
Files
Other
Servers
Zeno®s
10 May 2002
OMG

15. New Technologies for Storage and Display of Meteorological Data
10 May 2002
Web Technology
Internet or Intranet Server
Data manipulation and report generation performed on server
Only results sent to browser
Automatic report refresh option
Additional security levels possible
10 May 2002
OMG

16. New Technologies for Storage and Display of Meteorological Data
10 May 2002
How does it perform?
Even running on a notebook:
Current reports are instantaneous
Historical reports take a few seconds
Frequency Distribution produces a 1 year report in 12 seconds
Data Capture Summary produces a 1 year report in 55 seconds
10 May 2002
OMG

17. Benefits of Technology
New Technologies for Storage and Display of Meteorological Data
10 May 2002
Benefits of Technology
Move to modern technology path that will be developed and supported
Get out of the program change queue
Flexibility to respond to change
Let Meteorologists manage, analyze and use the data
10 May 2002
OMG