1. I-COADS Data and Products Steven J. Worley Scott D. Woodruff Richard W. Reynolds

2. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 2 Topics BackgroundBackground StatusStatus PlansPlans

3. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 3 Background Three-way collaboration since 1981Three-way collaboration since 1981 Partner organizations and PI’sPartner organizations and PI’s –NOAA/NCDC, (Dick Reynolds, Joe Elms) –NOAA/CDC, (Scott Woodruff) –NSF/NCAR, (Steven Worley) U.S. Government and Academic @ startU.S. Government and Academic @ start Now, International-COADS (I-COADS)Now, International-COADS (I-COADS)

4. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 4 Current Status, Release 2.0 DM Archive 1784-1997 RT Archive 1998-2002 185 Million Observation Records

5. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 5 Current Status, Release 2.0 WWI and WWII 100K per year, 1854-1885

6. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 6 Data Availability ObservationsObservations –NCAR 1784 - 2002 binary formatted archive is online1784 - 2002 binary formatted archive is online ASCII format available by online requestASCII format available by online request –Select region –Select time –Select variables –Select quality flags and options –Delayed mode delivery via pull FTP

7. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 7 Data Availability, Observations @ NCAR http://dss.ucar.edu/pub/coads/forms/

8. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 8 Data Availability ObservationsObservations –CDC Derived from NCEP GTSDerived from NCEP GTS 1991-current, online, ASCII1991-current, online, ASCII Updated monthlyUpdated monthly Only Basic VariablesOnly Basic Variables Primary UsagePrimary Usage –Extend I-COADS to near real-time http://www.cdc.noaa.gov/coads/nrt.html

9. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 9 Data Availability Monthly StatisticsMonthly Statistics –1°x1°, 1960 - 1997 –2°x2°, 1800 – 2002 –NCAR Access equivalent to observationsAccess equivalent to observations –Full binary format archive is online –ASCII format with subset selection capability Very popular user interfaceVery popular user interface

10. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 10 Data Availability, MSG @ NCAR http://dss.ucar.edu/pub/coads/forms/

11. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 11 Data Availability Monthly StatisticsMonthly Statistics –CDC I-COADS 1° and 2° netCDF format filesI-COADS 1° and 2° netCDF format files –Through 1997 now, to be updated through 2002 2° extension based on NCEP GTS2° extension based on NCEP GTS –ASCII 1998 - current –NetCDF 1991 - current http://www.cdc.noaa.gov/coads/coads_cdc_netcdf.shtml http://www.cdc.noaa.gov/cdc/data.nmc.marine.html

12. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 12 Data Availability Monthly StatisticsMonthly Statistics –CDC, Live Access Server (LAS), Public Debut Interactive interfaceInteractive interface Database – CDC I-COADS netCDF archiveDatabase – CDC I-COADS netCDF archive –To be updated through 2002 soon visualizevisualize subset variables, choose output file formatsubset variables, choose output file format compare (difference) data gridscompare (difference) data grids http://www.cdc.noaa.gov/coads-las/servlets/dataset

13. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 13 Data Availability, CDC LAS, Constraints Interface

14. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 14 Data Availability, CDC LAS, Output Options Choose Option 2 Graphic NetCDF 2 ASCII 2 Spreadsheet GIS

15. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 15 Data Availability, CDC LAS, 1° 01/1970 SST Median

16. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 16 Data Availability, CDC LAS, 1°, SST Median, Jan.1983 – Jan.1989, Niño – Niña difference

17. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 17 Data Availability Advanced Products and AnalysesAdvanced Products and Analyses –New Adaptive QC trimming Monthly SST, 2°x2°, 1800-1997Monthly SST, 2°x2°, 1800-1997 A few detailsA few details –Authors: Tom Smith (NCDC), Xiao-Wei Quan (CDC) –Observational outliers are trimmed according to a reference climatology that varies with the climate signal –See link for other details and publication references Access from NCAR, binary or ASCII (subsets)Access from NCAR, binary or ASCII (subsets) http://dss.ucar.edu/pub/coads/forms/msg/msga.form.html

18. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 18 Data Availability, Adaptive QC SST Original Trimming via fix climate Adaptive Trimming via varying climate Decembers 1970-1997, 8-2S, 80-88W, Red = Trimmed Observations 20 30 10 0 30 20 10 0 82-8397-98

19. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 19 Data Availability Advanced Products and AnalysesAdvanced Products and Analyses –Extended Reconstructed SST and SLP Authors Tom Smith and Dick Reynolds (NCDC)Authors Tom Smith and Dick Reynolds (NCDC) http://lwf.ncdc.noaa.gov/oa/climate/research/sst/sst.html http://www.ncdc.noaa.gov/oa/climate/research/slp/index.html Monthly SLP, 2°x2°, Global, 1854-1997 Monthly SLP, 2°x2°, Global, 1854-1997 –Land and Ocean Access from NCDC, ASCII format Access from NCDC, ASCII format Monthly SST, 2°x2°, Global, 1854-2002, continuing Monthly SST, 2°x2°, Global, 1854-2002, continuing

20. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 20 Plans Transitioning archive to IMMA formatTransitioning archive to IMMA format –International Marine Meteorological Archive (IMMA) –Why? IMMA is an ASCII format – easier to use?IMMA is an ASCII format – easier to use? Gaining acceptance by WMO – JCOMM Expert Team for Marine ClimatologyGaining acceptance by WMO – JCOMM Expert Team for Marine Climatology Encourage I-COADS data contributors to use itEncourage I-COADS data contributors to use it –Insures accuracy from data providers –Easier to grow I-COADS Well documented, and proven successfulWell documented, and proven successful –Project examples: CLIWOC, RT at NCDC, NCEP GTS

21. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 21 Plans, IMMA Basic format conceptsBasic format concepts Core segment – fixed lengthOriginal data attachments+ Historical data GTS data Special formats Aux. QC info. Working Principles Core segment  All commonly used variables  Fixed length ASCII (108 characters)  Alone will be sufficient for most users Original data attachments  Contains special fields not in Core  Can be used to improve, repair, and recreate Core Great format for: an irreplaceable data archive ensure easy software access (ascii based) http:/www.cdc.noaa.gov/coads/e-doc/imma/

22. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 22 Plans, Real-time data for I-COADS Observations from NCDCObservations from NCDC –Features Based on NCDC GTS data streamBased on NCDC GTS data stream Web and FTP accessWeb and FTP access Pilot activation date: September 2004Pilot activation date: September 2004 –Fast access b/c database system Will include the full historical archiveWill include the full historical archive –Preparation so far Cross-checks NCDC GTS and NCEP GTS streamsCross-checks NCDC GTS and NCEP GTS streams – show good agreement and mutual accuracy

23. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 23 Plans, Real-time I-COADS data Steps beyond real-time data provisionSteps beyond real-time data provision –Archive two GTS data streams –Compute Monthly Statistics Generate @ CDCGenerate @ CDC –Lag time ~ 1 month Benefit:Benefit: –More complete records –Standard QC –Standard format Currently not high priorityCurrently not high priority

24. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 24 Plans, Future data Inclusions Details given by Scott Woodruff in a presentation for 150 th CelebrationDetails given by Scott Woodruff in a presentation for 150 th Celebration Light overview hereLight overview here

25. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 25 Polish Baltic: 1961- 90; 360K US Navy Hourly: 1952-64; 3M UK Marine Data Bank: 1950-79 Japanese Whaling Data: 1946-84; 30K Japanese Kobe (2003 ed.): 1889-1940; 3.2M Arctic Drift Stations: 1952-76 U.S. Meteorological Journals: 1878-94;1.8M Arctic Norwegian Data: 1867-1912; 125K Fully digitized components to be added to I-COADS

26. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 26 Ukrainian Hydrometeorological: 1958-85 Russian R/V Marine: 1936-2002; 4-6M US Lightship Data: 1891- recent German Data: 1887-90; 30K CLIWOC: 1750-1850; 300K Partly digitized components to be added to I-COADS

27. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 27 German Maury Collection: 1845-67; 500-750K Undigitized component to be added to I-COADS

28. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 28 Plans, Future data Inclusions Early access to new data sourcesEarly access to new data sources –Availability before merge into I-COADS Termed “add on” datasetTermed “add on” dataset –Features Data are subjected to climate trimmingData are subjected to climate trimming In an I-COADS formatIn an I-COADS format Unique and important dataUnique and important data –E.g. Kobe Observatory and CLIWOC data From NCARFrom NCAR http://dss.ucar.edu/datasets/ds530.0/

29. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 29 Plans, Web Status and Developments New “Data and Metadata Page”New “Data and Metadata Page” –New look Organized by data productsOrganized by data products –Eliminated the CDC/NCAR/NCDC structure I-COADS metadataI-COADS metadata –Software and documentation –Graphics and figures to show I-COADS inventories –Ship metadata – Pub47 (1973-1999) Most products mentioned here are linked to this page.Most products mentioned here are linked to this page. http://www.cdc.noaa.gov/coads/products.html

30. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 30 Plans, Web Status and Developments “Related Data and Resources”“Related Data and Resources” –Links to Projects and Organizations VOSCLIMVOSCLIM CLIWOCCLIWOC JCOMMJCOMM –Links to Associated Data JISAO Climatologies (Mitchell UW)JISAO Climatologies (Mitchell UW) –1° global Extended Reconstructed Analyses (NCDC)Extended Reconstructed Analyses (NCDC) http://www.cdc.noaa.gov/coads/related.html Grow this area, link more projects and data

31. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 31 Conclusions I-COADS is international 1998-2002 update available now Many access points and formats New analyses are available New format (IMMA) will be future standard I-COADS real-time observations – coming next year

32. Brussels, Belgium 17-22 Nov. 2003CLIMAR II, I-COADS Data and Products 32 Thanks!