1 Archiving Requirements – Current Requirements A. Juliann Meyer Sr. Hydrologist – Data Systems Missouri Basin River Forecast Center and RAXUM Team Leader RFC Verification Workshop, 08/15/2007

2 What Will be Covered How to archive forecasts and observations for 4 different variables, these are: precipitation, temperature, height and discharge. How to archive forecasts from different scenarios, such as forecasts using actual QPF vs. forecasts using no QPF (zero precip), or actual forecasts vs. forecasts from raw model (without any MODs). RFC Archive DB quality control capabilities

3 NWS Directive Section 4.7 covers verification requirement Section 4.8 covers archiving requirement This directive can be found at

4 Flow Chart for SHEF Decoders

5 ob7.2 The verifcation software only looks for data in the following tables: pecrsep and pephsep for observed data pedfsep for forecast data

6 Everything from this point on is about ob8.2

7 Where Data Is Stored

8 “oops” table, peoosep Verification software does try to use this table when looking for observed data that does not fit into the other SHEF data value tables. This is the table the raw shefdecoder post SHEF Type code “R” data to that has duration and/or extremum codes that don’t fit the other raw data value tables. It is also where the raw shefdecoder post data with a SHEF Type codes of “H”, “M” and “P”.

9 Distinguishing Between Forecasts Use SHEF, primarily the Type/Source Codes The latest SHEF manual can be found at

10 RFC Archive DB Quality Control Capabilities Gross Range & Reasonable Range Checks can be done at posting time, uses info in db tables datalimits and locdatalimits sensok table, entries in this table are used by other applications rivercrit table, if using level 1 processors, allows the user to specify addition range checks

11 IVP and SHEF quality codes IVP uses the SHEF data value tables SHEF qualifier code fields during the pairing process. The observed data value's SHEF qualifier code is the first item checked when making a pair. IVP has a hardwired, ordered string that defines a quality code preference. The order is: G, M, P, V, S, Z, T, F, Q. IVP pairing will never use observed value that has SHEF qualifier code of B, E or R.

12 Raw shefdecoder and adjustfactor Table Only the raw shefdecoder must apply the adjustfactor. Forecast data (i.e. data for which the SHEF Type Code is 'F') is not affected, but all other SHEF Types Codes are. AFTER posting a value to the database, the decoder checks to see if the apps_defaults token adb_shef_raw_add_adjust is not set to OFF. If it is not OFF, then the decoder will search the adjustfactor table rows for one to use. If it finds one (i.e. matching the primary key and the date associated with the adjust precedes the obstime of the value to post), then it will find the best one (the date precedes and is closest to the obstime) and apply the adjust. It will then post the adjusted record with a SHEF Source Code of 9 (i.e. s = '9').

13 Raw decoder and adjustfactor Table continued 1.Create an appropriate record in the adjustfactor table. 2.Add an appropriate record to the ingestfilter table, where everything is the same as the existing applicable record, but the source is '9'. 3.Set the token adb_shef_raw_add_adjust to 'on‘ and stop and restart the raw shefdecoder. Need only do this step if the adjustfactor table has not previously been used. In order for the adjusted data to make it into the same table as the original data, an appropriate record must be added to the ingestfilter table with a source of '9'. In summary, the user must do the following:

14 …And that’s all there is!!!

15 SHEF Code Basics Physical Element Codes HG – height, river stage HT – elevation, project tail water stage HP – elevation, pool PP – precipitation TA – air temperature QT – discharge, computed total project outflow QI – discharge, inflow For full list, see SHEF Manual, Table 1.

16 SHEF Code Basics Duration Codes I – instantaneous (0000) D – 1 day (2001) H – 1 hour (1001) Q – 6 hour (1006) For full list, see SHEF Manual, Table 3.

17 SHEF Code Basics Type Codes C - Contingency F – Forecast P & 1 thru 9 – Processed R – Reading (observed data) For full list, see SHEF manual, Table 4.

18 SHEF Code Basics Extremum Codes Z – null character (filler) X – Maximum of day N – Minimum of day D – Maximum of 1 hour F – Minimum of 1 hour R – Maximum of 6 hours H – Minimum of 6 hours For full list, see SHEF Manual, Table 5.

19 SHEF Code Basics Data Qualifier Codes Used by IVP G - Good, Manual Edit M – Manual Edit P – Passed level 2, 3 V – verified level 1, level 2 S – screened level 1 Z – No QCD Performed T - triggered F – flagged by sensor or telemetry Q - questionable Never Used by IVP B – Bad, Manual QC E – Estimated R – rejected by level 1