CHPS Era Archive/Data Backup John Halquist Development and Operations Officer National Operational Hydrologic Remote Sensing Center (NOHRSC) HIC Meeting – Feb 2010
Major categories for Archive use General Archive requirements Model development Unitgraph Routing Calibration Verification Analogs Studies Case studies Applied Research Modeling/Product improvement Hindcasting/Reanalysis
Current situation NWS IHFS (ops DB) NWSRFS ofs-de raw SHEF-dec. proc.SHEF-dec. RAX archive RDB flat files (grids) (field) data acquisition SHEF-dec. FEWS fc DB XSETS … products MPE NAVX ICP EVS SHEF
Current situation (FEWS archive) FEWS archive: all data related to forecast bundled in one zip file forecast and associated inputs can be retrieved in stand alone FEWS data not directly accessible Recommendation: intermediate solution only (as short as possible) FEWS fc DB meta data RDB zip files archive-server
Where do we plan to go?
import ExchangeGridPoint Could use SOAP or PI Possible NEW import JAVA class Enhanced or Additional FEWS Export functionality GG Calb Verif IWRSS
Key differences from existing Archive System Data flows “through” the operational system to feed the Archives eliminates Quality Control nightmare minimizes multiple “decode” pathways Provide for a more “Integrated” Archive 2 way data flow to support usage in operations and analysis Scalable, Flexible & Extensible architecture hardware & database schemas
Filling in the gaps …
Gaps in current requirements: Meta-data What do we really need? What do we want? What about current archived data with little meta-data? Modifiers in CHPS Implemented significantly different than NWSRFS Some decisions need to be made on how we want to handle modified time series
CHPS time series, example: original data timeseries modifiers time series used
Hardware modification: chps2 (rdms) Enclosure* (tuples) Expansion… Fileserver* Enclosure* (Flat files) Expansion… * Indicates New hardware
Schedule objectives: Finalize conceptual design - Done Determine Hardware requirements – Mar 2010 Data requirements finalization – May/Jun 2010 Build Prototype system – Sept 2010 Test system built at NOHRSC Operational field test at NWRFC Provide initial DB Schema for review – Nov 2010 Field Deployment - ~2012
fini...