Presentation is loading. Please wait.

Presentation is loading. Please wait.

Exporting Raw/ESD data from Tier-0 Tier-1s Wrap-up.

Similar presentations

Presentation on theme: "Exporting Raw/ESD data from Tier-0 Tier-1s Wrap-up."— Presentation transcript:

1 Exporting Raw/ESD data from Tier-0 Tier-1s Wrap-up

2 The problem (Bernd) One copy of the LHC raw data for each of the LHC experiments is shared among the Tier-1s Full copies of the ESD data (1/2 of raw data size) Total ~10PB/year exported from CERN The full machinery for doing this automatically should be in place for full-scale tests in 2006

3 Tier-1 resources TRIUMF 2 machines purchased, 1Gbit(?) RAL Gbit link at present Parallel activities from ATLAS and CMS Not enough effort to dedicate for the moment More hardware in September FNAL Just finished CMS DC – very labor intensive Enough resources to sustain 2TB/day GridKA 1Gbit at present, expanding to 10Gbit in October/November Storage system is ready (dCache + TSM) BNL SRM service almost ready (in a month) One gridftp node OC12 connection, not much used NIKHEF/SARA 10Gbit since more than a year Running data challenges for experiments but mainly CPU intensive IN2P3/Lyon Not yet ready with interface to MSS 1Gbit

4 Agreed tests 1.Simple disk-to-disk, peer-to-peer 2.Simple disk-to-disk, one-to-many 3.MSS-to-MSS 4.In parallel? a)Transfer scheduling b)Replica catalogue & management

5 Test 1: Simple test disk to disk: gridftp or bbftp (loadbalanced) Aggregated peer-to-peer transfers Files written, deleted and rewritten (process driven from CERN) ~3 weeks preparation Goal: 1 week running without intervention Tier-1 responsibility: tune gridftp headnodes for best transfer performance

6 Test 2: simple but one-to-many Additional Tier-0 complexity: export data to more than one Tier-1 in parallel

7 Test 3: slide in MSS Issue with SRM interoperability –GGF/GSM WG initiative to define a SRMv1.1 evolution SRM-basic Standardized error reporting

8 Test 4/5 (parallel) Test 4: slot in transfer scheduling: a scheduling service controls the replication Test 5: add Replica catalogue and management services

9 Timescales JuneJulyAugSeptOctNovDecJan05 Transfer scheduling FNAL BNL TRIUMF RAL SRM-Basic ready(?) one-to-many?

10 Next steps Statements from the other Tier-1s –NIKHEF, GridKA, Lyon –PIC, CNAF, others? Who is driving/coordinating? site contacts? Meetings ? Speed up SRM-basic specification process...

Download ppt "Exporting Raw/ESD data from Tier-0 Tier-1s Wrap-up."

Similar presentations

Ads by Google