Presentation is loading. Please wait.

Presentation is loading. Please wait.

Slide 1 WMO BUFR training course Langen 17-20 April 2007 Slide 1 Ecmwf observations data handling Milan Dragosavac.

Published byCoral Fox Modified over 8 years ago

Similar presentations

Presentation on theme: "Slide 1 WMO BUFR training course Langen 17-20 April 2007 Slide 1 Ecmwf observations data handling Milan Dragosavac."— Presentation transcript:

1 Slide 1 WMO BUFR training course Langen 17-20 April 2007 Slide 1 Ecmwf observations data handling Milan Dragosavac

2 Slide 2 WMO BUFR training course Langen 17-20 April 2007 Slide 2 MDBRDBPDBFDB ODB IFS MARS General applications AnalysisForecast Display Print Plot Data transport Data transport Data acquisition

3 Slide 3 WMO BUFR training course Langen 17-20 April 2007 Slide 3 Data acquisition Acquire data Process data Archive data ( not MARS) Outgoing ( send data outside Ecmwf)

4 Slide 4 WMO BUFR training course Langen 17-20 April 2007 Slide 4 Acquisition process GTS data ftp Other data

5 Slide 5 WMO BUFR training course Langen 17-20 April 2007 Slide 5 Acquisition outgoing Qscat satellite data in BUFR form SSMI brightness temperatures TOVS 1B calibrated and converted into BUFR format

6 Slide 6 WMO BUFR training course Langen 17-20 April 2007 Slide 6 MEB feed Restores any old data Join segmented bulletins Write bulletins in MEB statistics

7 Slide 7 WMO BUFR training course Langen 17-20 April 2007 Slide 7 Message data base “STREAM”_YYYYMMDD AIRC_20070216 AIRS_20070216 AMAP_20070216 GRIB_20070216 75 streams in total One stream per day

8 Slide 8 WMO BUFR training course Langen 17-20 April 2007 Slide 8 Message data base attributes TIMEcharacter( 4,1) OBSTYPEcharacter( 4,1) TTAAiicharacter( 6,1) BBBcharacter( 3,1) CCCCcharacter( 4,1) _SOURCEcharacter(80,1) _CHECKSUMlonginteger _TIMESTAMPcharacter(20,1)

9 Slide 9 WMO BUFR training course Langen 17-20 April 2007 Slide 9 Message data base attributes continued _data_sizelonginteger _bulk_sizelonginteger _ownercharacter( 3,1) _last_accesstime(0) _access_numberlonginteger _compresscharacter( 1,1) _databulk(20,0,512,1)

10 Slide 10 WMO BUFR training course Langen 17-20 April 2007 Slide 10 Message data base indices UNIQUE I_timestamp_SYNO_20070215 on (_TIMESTAMP) UNIQUE I_comp_SYNO_20070215 on (TIME, TTAAii, BBB, _CHECKSUM) NORMAL (2,62) I_OBSTYPE_SYNO_20070215 on (OBSTYPE)

11 Slide 11 WMO BUFR training course Langen 17-20 April 2007 Slide 11 Message data base 100 Gbytes allocated + 150 Gbytes for METOP data Over 22 Gbytes / day Saved on ECFS according to OBSTYPE All data can be reprocessed if Data base is lost HP cluster 4 CPU 64 bit processors 4 operational machines + 1 for development Raid disks 1.2 Tbytes

12 Slide 12 WMO BUFR training course Langen 17-20 April 2007 Slide 12 MEB data arrival Monitoring arrival times of different data types Analysis cycle Average number of days

13 Slide 13 WMO BUFR training course Langen 17-20 April 2007 Slide 13 Message data base data availability Data availability according to synoptic hour

14 Slide 14 WMO BUFR training course Langen 17-20 April 2007 Slide 14 MEB x-interface Allows MEB search Find corresponding BUFR observation in REB Expand BUFR and GRIB data Write data into file

15 Slide 15 WMO BUFR training course Langen 17-20 April 2007 Slide 15 MEB x-interface REB BUFR observations corresponding to MEB bulletin

16 Slide 16 WMO BUFR training course Langen 17-20 April 2007 Slide 16 Preprocessing suite MEB – message data base feed task deqc – decoding, quality control and BUFR creation, writing into REB ex00 – extraction for analysis bc00 – extraction for boundary condition suite wmo – ftp WMO station list

17 Slide 17 WMO BUFR training course Langen 17-20 April 2007 Slide 17 Decoding Quality control BUFR creation Global grib fields creation Writing into REB

18 Slide 18 WMO BUFR training course Langen 17-20 April 2007 Slide 18 REB extraction for analysis 14 files created Conventional observations Satellite data Sea surface temperatures Ice analysis

19 Slide 19 WMO BUFR training course Langen 17-20 April 2007 Slide 19 REB – report data base One table per observation type per day AIRC_20070215 ATOV_20070215 B002_20070215 CREX_20070215 TEMP_20070215 SSBT_20070215 QSCA_20070215 statistic

20 Slide 20 WMO BUFR training course Langen 17-20 April 2007 Slide 20 REB attributes TIMEcharacter(6,1) IDENTcharacter(9,1) _BUFR_TYPEinteger _BUFR_SUBTYPEinteger _LATITUDE1float(6) _LONGITUDE1float(6) _COR1integer _QCinteger

21 Slide 21 WMO BUFR training course Langen 17-20 April 2007 Slide 21 REB attributes continued _TIMESTAMPcharacter(20,1) NR_SUBSETSinteger _data_sizelonginteger _bulk_sizelonginteger _ownercharacter(3,1) _last_accesstime(0) _access_numberlonginteger _databulk(20,0,512,1)

22 Slide 22 WMO BUFR training course Langen 17-20 April 2007 Slide 22 REB indices UNIQUE I_ts_SYNO_20070215 on (_TIMESTAMP) UNIQUE I_comp_SYNO_20070215 on (TIME, IDENT, _BUFR_SUBTYPE, _LATITUDE1, _LONGITUDE1 NORMAL (2,15) I_extract_SYNO_20070215 on (TIME, _BUFR_SUBTYPE, IDENT)

23 Slide 23 WMO BUFR training course Langen 17-20 April 2007 Slide 23 REB reports data base Just bellow 22 Gbytes / day 100 Gbytes allocated + 150 Gbytes for METOP data All data can be re-decoded at any time Data can be accessed through MARS request and Metview Archived in MARS

24 Slide 24 WMO BUFR training course Langen 17-20 April 2007 Slide 24 REB x-interface Search REB Find original bulletin in MEB Expand BUFR and GRIB data Write data to file Access WMO station list data base

25 Slide 25 WMO BUFR training course Langen 17-20 April 2007 Slide 25 BUFR x-interface Expands BUFR message Any subset can be viewed Filtering features

26 Slide 26 WMO BUFR training course Langen 17-20 April 2007 Slide 26 WMO Data Base WMO_a_master WMO_a_master_manual WMO_a_master_tape

27 Slide 27 WMO BUFR training course Langen 17-20 April 2007 Slide 27 WMO_a_master attributes Region_countryinteger not null Blocksmallint not null Station_indexinteger not null Station_subindexsmallint not null Principal _stationcharacter(1,1) Station_nametext(20,20,30,1) Latitudereal Longitudereal

28 Slide 28 WMO BUFR training course Langen 17-20 April 2007 Slide 28 WMO_a_master attributes continued Pressure_elevationinteger Pressure_elevation_flagcharacter(1,1) Ground_elevationinteger Ground_elevation_flagcharacter(1,1) Pressure_levelsmallint Surf_prog_00character(2,1) Surf_prog_03character(2,1) Surf_prog_06character(2,1) Surf_prog_09character(2,1) Surf_prog_12character(2,1)

29 Slide 29 WMO BUFR training course Langen 17-20 April 2007 Slide 29 WMO_a_master attributes continued Surf_prog_15character(2,1) Surf_prog_18character(2,1) Surf_prog_21character(2,1) Ua_type_00smallint Ua_type_06smallint Ua_type_12smallint Ua_type_18smallint Modify_flagcharacter(1,1) not null

30 Slide 30 WMO BUFR training course Langen 17-20 April 2007 Slide 30 WMO_a_master indices UNIQUE I_wmo_a_master ON (block,station_index, station_subindex)

31 Slide 31 WMO BUFR training course Langen 17-20 April 2007 Slide 31 WMO station list

32 Slide 32 WMO BUFR training course Langen 17-20 April 2007 Slide 32 ODB Observational data base Interface to analysis Data base contains unpacked bufr data and original bufr data Loading done in parallel with load balance achieved on supercomputer due to bufr features Analysis updates ODB Analysis bufr feedback files created Archiving of feedback files in MARS

33 Slide 33 WMO BUFR training course Langen 17-20 April 2007 Slide 33 Loading ODB with observations Tasks running in parallel 2-16 processors used per task Load balancing achieved

34 Slide 34 WMO BUFR training course Langen 17-20 April 2007 Slide 34 ODB to BUFR odb2bufr family

35 Slide 35 WMO BUFR training course Langen 17-20 April 2007 Slide 35 Odbviewer

36 Slide 36 WMO BUFR training course Langen 17-20 April 2007 Slide 36 Odbviewer

37 Slide 37 WMO BUFR training course Langen 17-20 April 2007 Slide 37 Odbviewer

38 Slide 38 WMO BUFR training course Langen 17-20 April 2007 Slide 38 Odbviewer

39 Slide 39 WMO BUFR training course Langen 17-20 April 2007 Slide 39 Odbviewer

40 Slide 40 WMO BUFR training course Langen 17-20 April 2007 Slide 40 Odbviewer

41 Slide 41 WMO BUFR training course Langen 17-20 April 2007 Slide 41 Odbviewer

42 Slide 42 WMO BUFR training course Langen 17-20 April 2007 Slide 42 Future Sharp increase in volume of observations expected IASI data with 8641 channels per observation Data bases have crucial role in observations handling Binary data representation allows easy handling of observations as bulk objects

Download ppt "Slide 1 WMO BUFR training course Langen 17-20 April 2007 Slide 1 Ecmwf observations data handling Milan Dragosavac."

Similar presentations

Interfacing BUFR to NMC Systems Jeff Ator NOAA National Weather Service United States of America WORLD METEOROLOGICAL ORGANIZATION RA.

Interfacing BUFR to NMC Systems Jeff Ator NOAA National Weather Service United States of America WORLD METEOROLOGICAL ORGANIZATION RA.
Templates and Regulations

Templates and Regulations
Web cross-cutting group o Attendance: ~12 o RAQ and AER were represented o ECMWF reported on their current capability to: Publish plots and data Produce.

Web cross-cutting group o Attendance: ~12 o RAQ and AER were represented o ECMWF reported on their current capability to: Publish plots and data Produce.
ECMWF 14-16 June 2006Slide 1 Access to ECMWF data for Research Manuel Fuentes Data and Services Section, ECMWF ECMWF Forecast Products User Meeting.

ECMWF June 2006Slide 1 Access to ECMWF data for Research Manuel Fuentes Data and Services Section, ECMWF ECMWF Forecast Products User Meeting.
BUFR decoding Dominique Lucas – Milan Dragosavac

BUFR decoding Dominique Lucas – Milan Dragosavac
FM 94 BUFR and FM 95 CREX Table Driven Code Forms (or “Self-descriptive” Code Forms). The term self-descriptive means that the form and content of the.

FM 94 BUFR and FM 95 CREX Table Driven Code Forms (or “Self-descriptive” Code Forms). The term "self-descriptive" means that the form and content of the.
RARS Implementation Group – Sixth meeting Exeter - 2012 RARS Implementation Group – Sixth Meeting Exeter – 2012 EARS-IASI and EARS-NPP concept Slide: 1.

RARS Implementation Group – Sixth meeting Exeter RARS Implementation Group – Sixth Meeting Exeter – 2012 EARS-IASI and EARS-NPP concept Slide: 1.
More on Processes Chapter 3. Process image _the physical representation of a process in the OS _an address space consisting of code, data and stack segments.

More on Processes Chapter 3. Process image _the physical representation of a process in the OS _an address space consisting of code, data and stack segments.
New Resources in the Research Data Archive Doug Schuster.

New Resources in the Research Data Archive Doug Schuster.
1 © Crown copyright 2003 GRAS SAF User Workshop Helsingør, Denmark, 11-13 June 2003 WMO BUFR Format for NRT Dissemination of RO Data Dave Offiler.

1 © Crown copyright 2003 GRAS SAF User Workshop Helsingør, Denmark, June 2003 WMO BUFR Format for NRT Dissemination of RO Data Dave Offiler.
Slide: 1 ROSA GRAS Meeting February 2009 Matera, Italy User Services EUMETSAT EUMETSAT Data Access & User Support.

Slide: 1 ROSA GRAS Meeting February 2009 Matera, Italy User Services EUMETSAT EUMETSAT Data Access & User Support.
DOSTAG Meeting #51, 3 rd May 2007 Access to ESA Earth Observation data (past and current missions)

DOSTAG Meeting #51, 3 rd May 2007 Access to ESA Earth Observation data (past and current missions)
MLP Lyrical Analysis ● % of Unique Words ● # of Unique Words ● Average Word Length ● # of Lyrics ● # of Characters Input Feature Vectors:

MLP Lyrical Analysis ● % of Unique Words ● # of Unique Words ● Average Word Length ● # of Lyrics ● # of Characters Input Feature Vectors:
Chapter 14: Object-Oriented Design

Chapter 14: Object-Oriented Design
AgWeatherNet Annual Note: Be sure you have already selected your station before choosing this product. Select AWN Reports Click on Annual.

AgWeatherNet Annual Note: Be sure you have already selected your station before choosing this product. Select AWN Reports Click on Annual.
APPLICATION OF K-MEANS CLUSTERING The Matlab function “kmeans()” was used for clustering The parameters to the function were : 1. The matrix of entire.

APPLICATION OF K-MEANS CLUSTERING The Matlab function “kmeans()” was used for clustering The parameters to the function were : 1. The matrix of entire.
Jeff Ator.  Convert incoming observational data of varying types and formats into BUFR for NCEP observational database in /dcom/us007003  49 different.

Jeff Ator.  Convert incoming observational data of varying types and formats into BUFR for NCEP observational database in /dcom/us  49 different.
TCP: Software for Reliable Communication. Spring 2002Computer Networks Applications Internet: a Collection of Disparate Networks Different goals: Speed,

TCP: Software for Reliable Communication. Spring 2002Computer Networks Applications Internet: a Collection of Disparate Networks Different goals: Speed,
Check Disk. Disk Defragmenter Using Disk Defragmenter Effectively Run Disk Defragmenter when the computer will receive the least usage. Educate users.

Check Disk. Disk Defragmenter Using Disk Defragmenter Effectively Run Disk Defragmenter when the computer will receive the least usage. Educate users.
Observations Preprocessing Carla Cardinali

Observations Preprocessing Carla Cardinali

Similar presentations

Ads by Google