FTP Server Structure Markus Heene, DWD.

Slides:



Advertisements
Similar presentations
Interval Heaps Complete binary tree. Each node (except possibly last one) has 2 elements. Last node has 1 or 2 elements. Let a and b be the elements in.
Advertisements

CCCC 8 CCCC CCCC 8 CCCC.
AMAZON S3 FOR SCIENCE GRIDS: A VIABLE SOLUTION? Mayur Palankar and Adriana Iamnitchi University of South Florida Matei Ripeanu University of British Columbia.
CCCC CCCC CCCC CCCC.
Lossless Compression of Meteorological Data in GRIB Format R. Lorentz Fraunhofer Institute for Scientific Computation and Algorithms (SCAI) Germany.
Agenda Duality (quickly) Piecewise linearity Start chapter 4.
Department of Computer Engineering University of California at Santa Cruz Data Compression (1) Hai Tao.
Tabular or Microdata? Both or Neither. Dissemination of data  Published census volumes  Electronic tabular data  Microdata 1960+
Активное распределенное хранилище для многомерных массивов Дмитрий Медведев ИКИ РАН.
FTP Server prepared by Mohammed Ibrahim Programmer Computer & Internet Center Mosul University Presentation.
WMO BUFR &CREX Gil Ross, UK Met Office
Training on Meteorological Telecommunications Alanya, Turkey, September 2010 General Philosophy of Table Driven Code Forms Simon Elliott, EUMETSAT.
M.Lautenschlager (WDCC, Hamburg) / / 1 Semantic Data Management for Organising Terabyte Data Archives Michael Lautenschlager World Data Center.
M.Lautenschlager (WDCC, Hamburg) / / 1 Semantic Data Management for Organising Terabyte Data Archives Michael Lautenschlager World Data Center.
Data Transfers in the Grid: Workload Analysis of Globus GridFTP Nicolas Kourtellis, Lydia Prieto, Gustavo Zarrate, Adriana Iamnitchi University of South.
Introduction to computers 103 學年度 上學期 Solution of Homework_ch1 授課教授:李錫智.
Data centre support for the IGS-RT PP W. Söhne, H. Habrich, G. Weber Federal Agency for Cartography and Geodesy, Frankfurt am Main, Germany.
NOAA’s National Weather Service NDFD Dissemination Services Allan Darling Office of the CIO, Telecommunication Operations Center National Digital Forecast.
1-2-3 February 2006 –Page 1 Mersea Integrated System How to improve Access/Downloading services ? How far do we go in terms of standardization ?
Look Ma! No Hands! 03/18/2003 Presented by Chris Smith Marshall County Information Services
Free Phone $25 a month 100 minutes then 35c per min Free Phone $25 a month 100 minutes then 35c per min Free Phone $20 a month 75 minutes then 25c per.
OPERATING SYSTEMS Frans Sanen.  Analyze a FAT file system manually  FAT12 first and simplest version  Still used on smaller disks (e.g. floppies) 
File Transfer Protocol (FTP) FTP host stores files Client logs into host Client program sends command to get a file FTP host downloads the file with error.
INIS Electronic Information Exchange INIS Training Seminar 7-11 October 2013, Vienna, Austria Serene Wong INIS Unit International Atomic Energy Agency.
TS7700 Performance and Capacity Daily charts only Enter Description, Month & Year in this Text Box.
1 Workload Analysis of Globus’ GridFTP Nicolas Kourtellis Joint Work with:Lydia Prieto, Gustavo Zarrate, Adriana Iamnitchi, Dan Fraser University of South.
WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 General.
TS7700 Performance and Capacity Daily and Hourly charts Enter Description, Month & Year in this Text Box.
Chris Bailey June Webinar: How to Transfer Large Volumes of Small Files at Accelerated Speeds.
M. Lautenschlager (M&D/MPIM)1 WDC on Climate as Part of the CERA 1 Database System Michael Lautenschlager Modelle und Daten Max-Planck-Institut.
Slide 1 WMO BUFR training course Langen April 2007 Slide 1 Ecmwf observations data handling Milan Dragosavac.
CGMS 44: EUM-WP-16.ppt (May 2016) Coordination Group for Meteorological Satellites - CGMS Action closure: MTG FCI and IRS Format Presented to CGMS-44 Working.
NetCDF4 Trade Study PO.DAAC Data are available compressed – GNUzip or Bzip2 Data compressor PO.DAAC OPeNDAP and THREDDS servers bottleneck: – Uncompressing.
File Characterization Assessment
Status of GISC Tokyo GISC Tokyo / JMA.
GISC Seoul Status TT-GISC, Melbourne April 2016
GISC Tehran Status Feb, 2016 TT-GISC 2016-Doc18.
Binary Universal Form Representation (BUFR) Paul Hamer November, 2009
Current Status of Implementation of MTN at RTH Toulouse
Status Report of GISC Beijing
Installation, setup and use - Ferro Backup – V1.1
Update on Plan for KISTI-GSDC
Markus Heene, DWD Status GISC Offenbach Markus Heene, DWD
Your great subtitle in this line
TT-GISC-2016 Doc 15 Agenda Item 2
CS 140 Lecture Notes: Technology and Operating Systems
Status of SREPS - Sept Full two runs a day from April 2006.
CS 140 Lecture Notes: Technology and Operating Systems
MATH UNIT #7 Measurement
GRIB2 and ecCodes in the COSMO-Model
GISC xxxx Status Report Submitted by: xxx XXX (Country) (Doc xx)
Toulouse GISC Status TT-GISC.
Kolb Anton Spectrum, Belarus
Toulouse GISC Status Doc 29
Operational Status of GIS Offenbach
GISC MOSCOW STATUS REPORT 2018
WIS Project Office WMO WIS Data Exchange WIS Project Office WMO
Highly Compressed 82MB 1 =---====""- ·-*i.
Monthly Director’s Report
A joined proposal by CMA, DWD and JMA
Status Reprot GISC SEOUL
Item 5.1 of the agenda Preliminary results of LUCAS 2009 Part III
GISC Offenbach – Status Report 02/2014 (Doc 19)
- Measurement of traffic volume -
Deutscher Wetterdienst – Open Data
GISC Offenbach Kai Wirt Task Team GISC TT-GISC2018
Valentine’s Day Adopters Party
Summer Ozone Data Exchange
WIS Project Office WMO Managing WIS WIS Project Office WMO
SEPTEMBER 2019 ISSUE/42 nd VOLUME.
Presentation transcript:

FTP Server Structure Markus Heene, DWD

Agenda Structure of FTP Server for Dissemination Phase Basic estimates of directories 24h Cache requirement

Dissemination Phase – Tree 1 Level 1: CCCC Level 2: Alpha / Binary Alpha: T1T2 – Data Type GRIB: A2 – Reference Time GRIB: ii – Level GRID: A1 – Geographical Area, A2 – Reference Time CREX: T1T2A1 YY: Day of the month GG: Hour of observation

Basic estimates of directories Level 1 – CCCC ~ 300 according to http://www.wmo.int/pages/prog/www/ois/Operational_Information/VolumeC1/CCCC_en.html Level 1 – CCCC ~ 1500 according to VolC1 published 17 September 2013 Level 1 – CCCC on FTP Server: 286 Idea: If CCCC ~ 1500, we can create an additional top level directory ISO 2-letter or ISO 3-letter country code to group the CCCCs Level 2 – alpha or binary Level 3 – GRIB – A2: 26 Reference times where 4 currently not assigned Level 3 – T1T2 in case of Alpha: ~ 174 Level 3 – BUFR T1T2A1: ~ 57 Level 3 – GRID: only 4 CCCC’s provide GRID data Level 3 – CREX: only 8 CCCC’s provide GRID data Level 3 – PIC: ~ 25 CCCC’s provide PIC data

Basic estimates of directories – Archive directory Archive: This directory contains 2-minute archives of all data received in this time interval Max archive size (zip): ~ 60.4 MB Max archive size (gz): ~ 58.2 MB Max archive size (bzip2): ~ 55.7 MB Max archive size (xz): ~ 51.8 MB One day (17th Oct. 2013: 5.4 GB [zip]) Suggestion: /RUMS_AMDCN: Directory with 2-minute archives of all data from GISC Moscow’s AMDCN Max archive size (xz): ~ 34 KB One day (17th Oct. 2013: 927 KB [xz]) Compressed archives reduce transfer data volume and we get an additional checksum for free