1. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 General Philosophy of Table Driven Data Representation Forms Simon Elliott EUMETSAT simon.elliott@eumetsat.int

2. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Tuesday Late Morning Agenda  Self Description  Code Structures  BUFR and CREX Tables  Features Common to BUFR and CREX  Differences Between BUFR and CREX

3. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Data exchange and codes a code: “A system of words arbitrarily used for other words, to secure brevity and secrecy” a cipher: “A secret manner of writing … intelligible only to those possessing the key” to cipher: “To express by occult characters” Purpose is global exchange of data in an unambiguous and efficient way. WMO “codes” are data representation forms (DRFs).

4. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Problems Increasing volume and complexity of observations Changing requirements but fixed codes Change implies long time scale and costs: equipment manufacturers, training et c. Changes take about 2 years, if essential

5. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Table driven codes GRIB - Processed data in the form of grid- point values expressed in binary form. –Forecast, analysis, climatology, satellite images and products BUFR - Binary universal form for the representation of meteorological data. –Observations, satellite products CREX - Character form for the representation and exchange of data. –Like simple BUFR but human readable ASCII

6. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 General structure Identification: GRIB/BUFR/CREX Header: Date, time, originator, table versions... Optional section: Metadata (potentially XML), private data … Data description: What sort of data follows Actual data: here Closure: “7777” Table driven codes generally have this structure 

7. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Some features Data are self defining: type, content, edition, section lengths et c. Header data are at fixed offsets for quick access without decoding Binary data (GRIB and BUFR) are not really human readable Optional section can contain anything; use of XML for metadata under continuing discussion

8. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Consider BUFR Table B - Element descriptors –Start with ‘0’, temperature, pressure, et c. Table C - Operator descriptors –Start with ‘2’, change of scale, qc follows, et c. Table D - Sequence descriptors –Start with ‘3’, wind sequence, high accuracy location sequence, et c. Code and flag tables - –Type of cloud, channel combination, et c.

9. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Table B Example: 0-20-082 Amount of segment % 0 0 7 cloud free Data width (bits) Element name Element descriptor Unit Class 20, observed phenomena Reference value Scale Entry in class

10. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Scale, reference and width Encoding: value = (obs x [10^”scale”]) - “reference” in “width” bits e.g.: Let scale=1, reference=-50, width=7 then, if obs=1.3, encoded value is 63 allows obs from -5.0 to +7.6, steps of 0.1 Decoding: obs = (value + “reference”) ÷ [10^”scale”] e.g.: Let scale=-2, reference=5, width=4 then, if value=14, decoded value is 1900

11. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Table C Operator descriptors for more complex tasks, e.g. –Change width –Change scale –Change reference –Quality control follows –First order statistics follow –Define “data present” bit map –Re-use “data present” bit map –Cancel “data present” bit map

12. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Table D Not strictly required Saves space in Section 3 (list of descriptors) … about 1.5% for EUMETSAT winds One table D entry represents an agreed list of table B and/or C and/or D descriptors 3-01-011 = 0-04-001Year 0-04-002Month 0-04-003Day

13. WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, 17 - 20 April, 2007 Differences between BUFR and CREX BUFR is binary, better for telecoms and computers BUFR has a compression scheme BUFR mostly uses SI units (flight level is in meters) CREX is ASCII, easier for humans to read and write CREX has no explicit compression scheme CREX allows other units (temperature can be Celsius or Kelvin) In general, BUFR and CREX are quite similar, the difference being that CREX is easier for human encoders/decoders where computers are not available or telecoms are inadequate...