1. NCAR’s Reanalysis Effort : Past, Present, Futureby
Joey Comeaux
Steve Worley
Bob Dattore
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

2. Comeaux/Worley/Dattore - SCD/DSS
INTRODUCTION
Recap of NCAR’s Efforts
Observations Provided
Upper Air
Surface
Aircraft
Satellite
Data Processing QC
Station Histories/Libraries
Report Header
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

3. Comeaux/Worley/Dattore - SCD/DSS
INTRODUCTION
What’s New
In house at DSS / already provided
Potential sources
Experience/Suggestions
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

4. Comeaux/Worley/Dattore - SCD/DSS
DATA PREPARED by DSS
Upper Air
Raobs, Pibals, Dropsondes
Over 70,000,000 observations
More than 3,000 stations
About 50 different sources
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

5. Comeaux/Worley/Dattore - SCD/DSS
Upper Air Data
New Data
Updated
Unknown
Not DSS
Description
Period
NNR
ERA-40
C-CARDS Raobs
????
CARDS 542
China R & P
Countries R & P
French R & P
GATE R & P
1974
Line Island
1967
MIT Raobs
Navy Kunia
Navy Spot Soundings
NCEP ADP
1973-Cont ?
NCEP B3
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

6. Comeaux/Worley/Dattore - SCD/DSS
New Data
Updated
Unknown
Not DSS
Description
Period
NNR
ERA-40
Permanent Ship
????
Ptarmigan Drops
Russian NP
Russian Ship
Russian Upper Air
Scherhag Raobs
South African
TD390 Raobs
TD52 Pibals ?
TD53 Pibals
TD54 Raobs
TWERLE
US Control
1946-Cont
USAF Upper Air
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

7. Comeaux/Worley/Dattore - SCD/DSS
DATA PROVIDED by DSS
Surface Data
Land surface stations
~ 1 Billion observations
>10,000 stations
Ships
ICOADS
Update thru 2004 to be completed in 2005
185,000,000 observations
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

8. Comeaux/Worley/Dattore - SCD/DSS
SURFACE DATA
New Data
Updated
Unknown
Not DSS
Description
Period
NNR
ERA-40
TD13
TD14
1938-Cont
NCEP GTS
1975-Cont
????
USAF-Dick Davis
Russian Sfc Hrly
UW Antarctic
1980-Cont
ICOADS ?
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

9. Comeaux/Worley/Dattore - SCD/DSS
DATA PROVIDED by DSS
Aircraft
Provided data for
~190,000,000 observations
11 different sources
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

10. Comeaux/Worley/Dattore - SCD/DSS
AIRCRAFT DATA
New Data
Updated
Unknown
Not DSS
Description
Start
NNR
ERA-40
U.S.A.F. TD57
Sadler
NCEP B3 ?
NCEP ADP ACFT
1973-Cont ?
NCEP ACARS
1991-Cont ?
USAF GTS
????
Navy
New Zealand
Australia
GASP
NAVY SPOT
GATE
1974
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

11. Comeaux/Worley/Dattore - SCD/DSS
SATELLITE DATA
New Data
Updated
Unknown
Not DSS
Description
Start
NNR
ERA-40
VTPR
TOVS ?
SIRS ?
GOES Cloud Winds
B3 Cloud Winds
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

12. Comeaux/Worley/Dattore - SCD/DSS
Data Processing QC
Station History
Report Headers
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

13. Comeaux/Worley/Dattore - SCD/DSS
QUALITY CONTROL
QC/QA Checks
Duplicate Checks
Duplicate Elimination - some
Date/Time check
Range checks on Data
Location checks - some
Unit checks
Track checks
Cross-dataset compares
Corrected
Winds, Heights, Temps, Dew Points
Hydrostatic checks
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

14. Original Failure Rate
Final Failure Rate
Correction Rate

15. Comeaux/Worley/Dattore - SCD/DSS
Station Histories
Process for Creating Station Histories
Create History
Locations/elevations on observations
Calculate elevations from observations
Interpolate elevations from terrain data
Digitize Station Dictionaries
Use Atlas
Internal consistency checks
Compare to many other histories/libraries
Quite a few datasets with no locations/elevations
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

16. Sources for WMO Libraies
Station Histories
Sources for WMO Libraies
SOURCE
START
END
NCEP OBS
1962
2002
C-CARD
1948
1965
CARDS
1900
1999
GAFFEN
1995
WMO
Cont
NAVY
1964
TD54
1941
1967
USAF
1977
1997 (1999)
WBAN
DSS Elevations
Upper Air
WMO
WBAN (1900-Cont)
TD52 Library
Surface
WMO
WBAN Cont TD
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

17. No Elevations in B3
B3/ADP Change Over

18. Bad GTS Locs
Aug 20-21, 2002
GTS Elv = 2m

19. Comeaux/Worley/Dattore - SCD/DSS
DSS Report Header
Report Header
Added to all conventional obs
Contains info on
Source
Format
Type of obs
Type of ID
Station History - most recent lat,lon
New Version (8/2005)
Processing date
Station history name and version
Data provider
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

20. Comeaux/Worley/Dattore - SCD/DSS
NEW DATA
Reside at DSS / already provided
Potential sources
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

21. Comeaux/Worley/Dattore - SCD/DSS
NEW DATA at NCAR
UPPER AIR
Galapagos Isl
SHEBA
10/ /1998
Other Field Projects
????
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

22. Comeaux/Worley/Dattore - SCD/DSS
NEW DATA at NCAR
SURFACE DATA
Canadian Sfc Hrly
MDL U.S. & Can
1977-Cont
UW Antarctic Ship
????-Cont
Australian Sfc
Russian NP Ice Isl
NCDC ISH
1901-Cont
U.S. SAO
Brazil
Riverflow
????
Precipitation
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

25. Improvement in 1-Degree Box Coverage in ERA-40 for Jan, 1960

26. POTENTIAL SOURCES for NEW DATA
New Canadian data
SFC for later
U/A pre-1950’s
Brazillian u/a and sfc data
Japan u/a
South Africa aircraft for 1950’s
CDMP
Restoration for about 21 African u/a stations for 1950’s
DSS Restoration project w/ CDMP
Working 7 and 9 track tape drives
Univ of Alaska
Canada
ASLI
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

27. LESSONS LEARNED/SUGGESTIONS
QC & station history works takes 2x as long as expected
Merge process should allow for update of individual datasets
Insure updated versions of data overide older data
GTS
Datasets which have been corrected
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

28. LESSONS LEARNED/SUGGESTIONS
Data provided should contain
Original source of data
Data provider
Type of ID
Processing date
Station history identifier and version
This info should be carried into BUFR and Feedback
Improved communication
Better documentation
More info on data downloaded and merged
Improved feedback from model
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS

29. Archive Data Refresh Cycle
detail labels on all records (e.g. data and library version, ..)
inventories
first order data checks and systematic error fixes
Archive Center
Collections
Collection
Merging
create merged record format
convert units
QA/QC evaluations
Assimilation
Analysis
verify data receipt with archive center
feedback problems discovered in QA/QC
possible request to fix and re-supply the data
carry forward all detail record identification
insert metadata relative to observed data quality, etc
communicate back to archive center
Feedback
output
4/30/2019
Comeaux/Worley/Dattore - SCD/DSS