1. NOAA National Climatic Data Center http://www.ncdc.noaa.gov Dr. Karsten Shein Climatologist NOAA/NESDIS/NCDC 151 Patton Ave. Asheville, NC 28801 1.828.271.4223 Karsten.Shein@noaa.gov Interactive Quality assurance practices A look at some methods for evaluating COOP data at the NOAA National Climatic Data Center IIPS 6A.9

2. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 2 of 19 COOP Climate Data ~ 8500 stations reporting ~ 1,000,000 observations per month Manual observations and reporting –Most stations observe PRCP (SNOW) –Many also observe TMAX, TMIN, TOBS, SNWD –Few: DYSW, EVAP, WTEQ, WDMV, Soil T Daily data / Monthly processing Arrive at NCDC in electronic (daily) and paper (monthly) formats.

3. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 3 of 19 Sources of bias in COOP data Observations –Rounding, Instrument error, incorrect obs technique Recording –Transposition, Wrong column, Wrong units, Wrong sign, illegibility, non- entries, wrong resolution, wrong date, wrong time. Transcription (keying) –Most keying errors are due to bias already introduced by the recording step Transmission (file corruption) Validation / Quality Control –Failure to review forms prior to NCDC –Incorrect metadata –Inappropriate validation, flagging, estimation

4. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 4 of 19 Not all COOP observations are created equally

5. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 5 of 19

6. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 6 of 19 10° F or 12° F ? Error corrections or error creations? ° F ° F TMAX of 19° F with TOBS of 21 ° F ? 20 inches of snow or 2.0?Keyers must “key what they see.” TMIN and TOBS in wrong columns.

7. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 7 of 19 Automated COOP QC Primary checks on Temperature and Precipitation elements –Around 10 million values per year Internal consistency –Logical (e.g., TMAX ≥ TMIN) –Spikes, flatliners, outliers, excessive range, change points –Date shifting Spatial consistency Values not automatically invalidated unless logically or meteorologically impossible. –Suspect data are reviewed by operators Original values are NEVER changed or edited –unless they were keyed incorrectly

8. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 8 of 19 First stage Interactive QC “Quasi-Interactive” Applied to values deemed suspect and where a logical operation will not provide resolution (e.g., date shifting) Value compared to climatological neighbors and to computed grids (GEA, TempVAL, PrecipVAL) –See extended abstract or http://www.ncdc.noaa.gov/oa/hofn/coop/coop-pubs-doc.html for references and details.http://www.ncdc.noaa.gov/oa/hofn/coop/coop-pubs-doc.html Decision is made to accept/reject valid status assigned by automated QC Operator intervention is part of decision process

9. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 9 of 19 Fully-Interactive QC Health of the Network Datzilla

10. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 10 of 19 Health of the Network –Available once final QC has been completed –Output for: TMAX, TMIN, TOBS, PRCP, SNOW, SNWD –Accessible by anyone via the Internet –Track QC by station, state, WFO, NWS region, or RCC region. –Graphical and tabular reports to highlight quality issues. Web-based tool for viewing the results of NCDC QC processing. http://www.ncdc.noaa.gov/oa/hofn/index.html

11. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 11 of 19 Health of the Network Data Completeness (number of obs) Quality Assurance (invalid and missing) Missing Data / Non- reporting stations Data validity (% unflagged, non-missing) Watch list (change points detected w/o corresponding metadata)

12. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 12 of 19

13. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 13 of 19 Summary from HoN All TMAX, TMIN, TOBS, PRCP, SNOW, SNWD data subjected to automated QC. 96.72% of checked 2006 COOP data declared valid (no further checks). 3.28% declared invalid (331,777) –14.65% no estimate (48,602) –85.36% estimated (283,175) ◦31% of estimates supplied by TempVal (87,820) Thanks to Helen Frederick, HoN administrator, for supplying the numbers.

14. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 14 of 19 Datzilla Fully manual, interactive quality assurance Web-based tool to report and track errors in NOAA-held data, metadata or associated delivery systems. Developed and maintained by Kevin Robins at the SRCC http://datzilla.srcc.lsu.edu/datzilla/

15. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 15 of 19

16. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 16 of 19 Some possible reasons for a Datzilla ticket to NCDC Data issue (usually) –That TMAX of 74 should be a 47! –Your #$&!*@ QC clobbered my data! System issue –CDO inventory doesn’t match the data I got! Metadata issue –This station’s COOP number is wrong!

17. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 17 of 19 When NCDC receives a Datzilla ticket … Datzilla gatekeeper Initial determinations Reassignment Investigation Course of action Resolution Closure Your friendly Datzilla Gatekeeper

18. NOAA National Climatic Data Center http://www.ncdc.noaa.gov 18 of 19 Datzilla Summary Began operation early 2005 As of 1/15/08: –865 Datzilla entries (564 NCDC) –226 open (109 NCDC) ◦Receive about 15 new tickets per month –455 of the 564 resolved –317 verified errors with archive fix ◦Most are single values or metadata ◦Few involve many values or larger station issues Has resulted in improvement to the historical climate record. Data quality Data errors

19. NOAA National Climatic Data Center http://www.ncdc.noaa.gov Dr. Karsten Shein Climatologist NOAA/NESDIS/NCDC 151 Patton Ave. Asheville, NC 28801 1.828.271.4223 Karsten.Shein@noaa.gov Thank You