1. NOAA’s National Weather Service NDFD File Dissemination Services Allan Darling Office of the CIO, Telecommunication Operations Center National Digital Forecast Database Technical Workshop November 2, 2006

2. NDFD Technical Workshop 2006 NDFD File Dissemination Services Where to find NDFD product files – Files are available via: ftp://tgftp.nws.noaa.gov http://weather.noaa.gov/pub ftp://ptgftp.nws.noaa.gov ftp://tgdata.nws.noaa.gov Access to ptgftp.nws.noaa.gov and tgdata.nws.noaa.gov are limited to SAS customers. See http://weather.gov/datamgmt/fos/fospage.html for more information about Server Access Services.http://weather.gov/datamgmt/fos/fospage.html

3. NDFD Technical Workshop 2006 NDFD File Dissemination Services Where to find NDFD product files – On each server, NDFD data is found under the paths: /SL.us008001/ST.opnl/DF.gr2/DC.ndfd/... or /SL.us008001/ST.expr/DF.gr2/DC.ndfd/... The path indicates that the data is on the public NWSTG FTP server, is operational or experimental, is encoded in WMO GRIB Edition 2 format, and is all related to the NDFD data category. See http://www.nws.noaa.gov/datamgmt/filstnd.html for more information about the directory and file naming standards used on the NWSTG FTP and HTTP Servershttp://www.nws.noaa.gov/datamgmt/filstnd.html

4. NDFD Technical Workshop 2006 NDFD File Dissemination Services How files are named – Following the data format and category directories, the last directories and the file name specify: the geographic area ( /AR.aaaaaaaa ) the valid period (/VP.ddd-ddd ) the data subcategory ( /ds.sssss ) A list of all the NDFD Sectors (geographic areas) defined by lat/lon and grid points is available at: http://www.nws.noaa.gov/datamgmt/doc/ndfdelem.xls http://www.nws.noaa.gov/datamgmt/doc/ndfdelem.xls A list of data subcategories is available at: http://www.weather.gov/ndfd/resources/NDFD_element_status.pdf http://www.weather.gov/ndfd/resources/NDFD_element_status.pdf

5. NDFD Technical Workshop 2006 NDFD File Dissemination Services How files are constructed – Files are comprised of: A flag field separator block (option 2) A super heading One or more GRIB products each comprised of – –A flag field separator block (option 2) – –A WMO Heading – –A GRIB product

6. NDFD Technical Workshop 2006 NDFD File Dissemination Services A flag field separator block (option 2) is a file delimiter that provides the length of the product that follows it. The format is: ****nnnnnnnnnn****lf Where nnnnnnnnnn is the length of the product following the seperator, in bytes and lf is an ASCII line feed. See http://weather.gov/tg/fstandrd.html for more details.http://weather.gov/tg/fstandrd.html

7. NDFD Technical Workshop 2006 NDFD File Dissemination Services What is a “super heading”? In order to aggregate forecast periods for one data subcategory and one sector, the “super heading” was developed. The super heading specifies the data subcategory in T 2 and the geographic sector in A 1. A 2 and ii are fixed as “Z98” or “Z97”: NDFD Super Heading: YT 2 A 1 Z98 KWBN

8. NDFD Technical Workshop 2006 NDFD File Dissemination Services How product headings are formed – Generic NDFD Heading: YT 2 A 1 A 2 ii KWBN T 2 specified the data type (data subcategory) A 1 specifies the geographic designator (NDFD sector) A 2 and ii together specify the NDFD reference time (not represented in the directory or file names) The possible values for T 2, A 1, A 2, and ii and their meanings are specified on the NDFD WMO Heading Reference page at http://weather.gov/datamgmt/doc/ndfdref.html

9. NDFD Technical Workshop 2006 NDFD File Dissemination Services How files are constructed – ****0004911342**** YEUZ98 KWBN 301549 ****0000266234**** YEUB18 KWBN 301549 GRIB..................7777 ****0000018847**** YEUB21 KWBN 301549 GRIB.....................7777. ****0000018051**** YEUE00 KWBN 301549 GRIB.................7777

10. NDFD Technical Workshop 2006 NDFD File Dissemination Services Where to find NDFD product files – Files are available via: ftp://tgftp.nws.noaa.gov http://weather.noaa.gov/pub ftp://ptgftp.nws.noaa.gov ftp://tgdata.nws.noaa.gov Access to ptgftp.nws.noaa.gov and tgdata.nws.noaa.gov are limited to SAS customers. See http://weather.gov/datamgmt/fos/fospage.html for more information about Server Access Services.http://weather.gov/datamgmt/fos/fospage.html

11. NDFD Technical Workshop 2006 NDFD File Dissemination Services Legacy NWSTG file services: HTTP access via Internet – http://weather.noaa.gov (5 servers) FTP access via Internet – ftp://tgftp.nws.noaa.gov (4 servers) FTP access via dedicated communication paths – ftp://ptgftp.nws.noaa.gov (2) servers

12. NDFD Technical Workshop 2006 NDFD File Dissemination Services Significant system upgrades as part of the Replacement NWSTG implementation, including: Linux servers Redundant, highly available NFS servers Network-based load balancing Gigabit Ethernet internal network Improved network devices

13. NDFD Technical Workshop 2006 NDFD File Dissemination Services Replacement NWSTG file services: HTTP access via Internet – http://weather.noaa.gov (6 servers) FTP access via Internet – ftp://tgftp.nws.noaa.gov (5 servers) FTP and HTTP access via dedicated communication paths – ftp://tgdata.nws.noaa.gov (3 shared servers) http://tgdata.nws.noaa.gov (3 shared servers)

14. NDFD Technical Workshop 2006 NDFD File Dissemination Services tgdata.nws.noaa.gov available for testing in the Replacement NWSTG since August – –Supported operationally as of November 8, 2006 – –Four week overlap between startup of tgdata.nws.noaa.gov and shutdown of ptgftp.nws.noaa.gov weather.noaa.gov and tgftp.nws.noaa.gov still in Legacy NWSTG – –Service will move (transparently) to the Replacement NWSTG by December 1

15. NDFD Technical Workshop 2006 NDFD File Dissemination Services Questions Now? Questions Later? Allan.Darling@noaa.gov

16. NDFD Technical Workshop 2006 NDFD File Dissemination Services Additional Information

17. NDFD Technical Workshop 2006 NDFD File Dissemination Services ****0000149136**** YGCZ98 KWBN 031906 ****0000018449**** YGCB00 KWBN 031906 GRIB..................7777 ****0000018847**** YGCC00 KWBN 031906 GRIB.....................7777. ****0000018051**** YGCI00 KWBN 031906 GRIB.................7777 ****0000149136**** YGCZ98 KWBN 031906 149136 is the length, in bytes, of the super heading and all the following groups of flag field separators, WMO headings, and GRIB products. Super heading flag field separator -

18. NDFD Technical Workshop 2006 NDFD File Dissemination Services ****0000149136**** YGCZ98 KWBN 031906 ****0000018449**** YGCB00 KWBN 031906 GRIB..................7777 ****0000018847**** YGCC00 KWBN 031906 GRIB.....................7777. ****0000018051**** YGCI00 KWBN 031906 GRIB.................7777 ****0000018449**** YGCB00 KWBN 031906 GRIB.............7777 18449 is the length, in bytes, of the single heading and GRIB product. “Regular” flag field separator -

19. NDFD Technical Workshop 2006 NDFD File Dissemination Services Online References NWSTG Data Management NDFD Reference - http://weather.gov/datamgmt/doc/ndfdindex.html http://weather.gov/datamgmt/doc/ndfdindex.html Directory and File Naming Standards – http://weather.gov/datamgmt/filstnd.html Family of Services Main Page – http://weather.gov/datamgmt/fos/fosindex.html Gateway File Standards and Content Structures – http://weather.gov/tg/fstandrd.html

20. NDFD Technical Workshop 2006 NDFD File Dissemination Services Backup NWSTG file services - Data access service backup will be split – –Public Internet service will be provided at the Primary NWSTG and at NWS Central Region Headquarters – –Non-public service will be provided at the Primary NWSTG and the Backup NWSTG Back-end NFS servers will be populated at the active NWSTG – –Primary active: data copied to the backup NWSTG and CR HQ – –Backup active: data copied to CR HQ

21. NDFD Technical Workshop 2006 NDFD File Dissemination Services Access via Dedicated Communications

22. NDFD Technical Workshop 2006 NDFD File Dissemination Services Access via Internet