TIGGE Data Archive at NCAR 8th GIFS-TIGGE Working Group World Meteorological Organization Geneva February, 2010 Doug Schuster Steven Worley Dave Stepaniak Others

Archive Summary Online Data  Period, most recent two weeks  ~ 5 TB, public products  ~ 1 TB, data preparation, subsetting  ~ 65 GB Tropical Cyclone Track Data (CXML) Offline Data  Full period of record  ~ 300 TB, NCAR MSS system

Provider Summary Archive Status Service Improvements Special Projects Metrics Future Service Additions

Data Delivery (02/2010) Data exchanges using UNIDATA LDM, HTTP and FTP 440 GB/day (ECMWF = 332 GB/day) 1.7+ million 2-D gridded fields/day

Archive Inhomogeneities

PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level VariableLvLECWFUKMOJMANCEPCMACMCBOMMetFKMACPTC Geopotential ZPL Specific HPL T U-velocityPL V-velocityPL Potential VorPT Potential TPV U-velocityPV V-VelocityPV U 10mSL V 10mSL CAPESL Conv. Inhib.SL Land-seaSL Mean SLPSL Orog.SL Skin TSL Snow D. H20SL Snow F. H20SL Parameter vs Provider Matrix

VariableLvLECWFUKMOJMANCEPCMACMCBOMMetFKMACPTC Soil Moist.SL Soil TSL Sunshine D.SL Surf. DPTSL Surf. ATmaxSL Surf. ATminSL Surf. ATSL Surf. PSL LW Rad. OutSL LH fluxSL Net RadSL Net Therm. RadSL Sensible Rad.SL Cloud CovSL Column WaterSL PrecipitationSL Wilt. PointSL Field Cap.SL PL = Pressure Level, PT = 320K θ Level, PV = ± 2 Potential Vorticity Level, SL = Single/Surface Level Parameter vs Provider Matrix

2009 Service Improvements  Improved Web Portal Performance  User data selection interface runs smoother.  Automated capability to stage offline data for download.  Added “Client for URLs” (cURL) based parameter subsetting capability.  Directly extract selected 2D grids from files across multiple data providers.  Downloads managed from user’s workstation.  Script templates are provided to drive “subscription style” parameter download capability.  Strengthened data ingest processing.  System monitoring (disk capacity, LDM status, etc)  Failover mechanisms for LDM downtime.  Data integrity checking.

2009/2010 Supported Special Projects  NCAR Research Applications Laboratory (RAL) Google African Meningitis Forecast Initiative  Dr. Thomas Hopson  Integrate weather and health data to manage African meningitis outbreaks (in Ghana as starting point).  Looking for 2-week lead time to position vaccines.  Processing ECMWF data from present backwards through the archive for model calibration.  Total Precipitation, 2m Temperature, 2m Dew Point.  Global Subarea 40 deg N to 40 Deg S.  0.5 x 0.5 degree resolution.  Assist RAL and Unidata in providing “Real-time” access to TIGGE based Multi-Model Ensemble Forecast data.  

NCAR TIGGE Full Model Data Archive Usage YEARGB Accessed GB Downloaded Files Downloaded Active Users / Unique Users / ,00010 / ,3001,10040,00010 / ,7001,90068,00010 / / 5 Total15,9853,925140,51033 / 80 cURL impact

NCAR TIGGE CXML Archive Usage YEAR GB Downloaded Files Downloaded # Active Users / # Unique Users / ,30017 / / 6 Total27.513,66027 / 81

Planned Service Improvements  Additional Online Storage  45 TB data online (3 months of most recent data).  15 TB for data processing request output.  Interface driven data processing and immediate download capability extended from 2 weeks to 3 months.  Improved “Offline” Storage.  Data being transitioned from current NCAR MSS system to new HPSS system.  Improved access speeds to offline data.  Add data processing capability to offline data?  Expanded/Updated Computing.  Improved LDM ingest reliability.  Faster “turn-around” time on user sub-set requests.  Potential for subscription services?

Potential Service Improvements  Model Validation Data Portal (Pending Funding)?  Wide variety of observational and analyzed data sources.  In Situ Observation data (NCEP operationally collected data).  Ocean based observations (ICOADS).  Global tropical cyclone position and intensity data.  Atmospheric Reanalysis (JRA, ERA, NNRR, CFSR).  Global PCP datasets.  Suggested TWG sets to support FDP’s.  Will need discussions to set priorities for development.  Data sub-set and output format selection capability.  Integrated With TIGGE model data portal  Data analysis tools and codes.  Model Evaluation Tools offered by the Developmental Testbed Center at NCAR 