Efi.uchicago.edu ci.uchicago.edu FAX status report Ilija Vukotic Computation and Enrico Fermi Institutes University of Chicago US ATLAS Computing Integration.

Slides:

Advertisements

Similar presentations

FAX status. Overview Status of endpoints and redirectors Monitoring Failover Overflow.

Advertisements

News from the HEPiX IPv6 Working Group David Kelsey (STFC-RAL) HEPiX, Oxford 24 Mar 2015.

Outline Network related issues and thinking for FAX Cost among sites, who has problems Analytics of FAX meta data, what are the problems  The main object.

Efi.uchicago.edu ci.uchicago.edu FAX update Rob Gardner Computation and Enrico Fermi Institutes University of Chicago Sep 9, 2013.

LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.

T2D: review Criteria* All transfers from the candidate T2D to 10/12 T1s for big ﬁles (‘L’) must be above 5 MB/s during the last week and during 3 out of.

US ATLAS Western Tier 2 Status and Plan Wei Yang ATLAS Physics Analysis Retreat SLAC March 5, 2007.

ATLAS federated xrootd monitoring requirements Rob Gardner July 26, 2012.

CERN IT Department CH-1211 Geneva 23 Switzerland t The Experiment Dashboard ISGC th April 2008 Pablo Saiz, Julia Andreeva, Benjamin.

News from the HEPiX IPv6 Working Group David Kelsey (STFC-RAL) WLCG GDB, CERN 8 July 2015.

FAX UPDATE 1 ST JULY Discussion points: FAX failover summary and issues Mailing issues Panda re-brokering to sites using FAX cost and access Issue.

FAX UPDATE 26 TH AUGUST Running issues FAX failover Moving to new AMQ server Informing on endpoint status Monitoring developments Monitoring validation.

Efi.uchicago.edu ci.uchicago.edu ATLAS Experiment Status Run2 Plans Federation Requirements Ilija Vukotic XRootD UCSD San Diego 27 January,

Efi.uchicago.edu ci.uchicago.edu Towards FAX usability Rob Gardner, Ilija Vukotic Computation and Enrico Fermi Institutes University of Chicago US ATLAS.

Efi.uchicago.edu ci.uchicago.edu FAX meeting intro and news Rob Gardner Computation and Enrico Fermi Institutes University of Chicago ATLAS Federated Xrootd.

MW Readiness Verification Status Andrea Manzi IT/SDC 21/01/ /01/15 2.

Status & Plan of the Xrootd Federation Wei Yang 13/19/12 US ATLAS Computing Facility Meeting at 2012 OSG AHM, University of Nebraska, Lincoln.

CERN IT Department CH-1211 Geneva 23 Switzerland GT WG on Storage Federations First introduction Fabrizio Furano

Efi.uchicago.edu ci.uchicago.edu FAX Dress Rehearsal Status Report Ilija Vukotic on behalf of the atlas-adc-federated-xrootd working group Computation.

PanDA Update Kaushik De Univ. of Texas at Arlington XRootD Workshop, UCSD January 27, 2015.

Efi.uchicago.edu ci.uchicago.edu Using FAX to test intra-US links Ilija Vukotic on behalf of the atlas-adc-federated-xrootd working group Computing Integration.

Efi.uchicago.edu ci.uchicago.edu FAX status developments performance future Rob Gardner Yang Wei Andrew Hanushevsky Ilija Vukotic.

1 LHCb on the Grid Raja Nandakumar (with contributions from Greig Cowan) ‏ GridPP21 3 rd September 2008.

Factors affecting ANALY_MWT2 performance MWT2 team August 28, 2012.

USATLAS dCache System and Service Challenge at BNL Zhenping (Jane) Liu RHIC/ATLAS Computing Facility, Physics Department Brookhaven National Lab 10/13/2005.

Julia Andreeva, CERN IT-ES GDB Every experiment does evaluation of the site status and experiment activities at the site As a rule the state.

Marco Cattaneo LHCb computing status for LHCC referees meeting 14 th June

Efi.uchicago.edu ci.uchicago.edu Status of the FAX federation Rob Gardner Computation and Enrico Fermi Institutes University of Chicago ATLAS Tier 1 /

ATLAS Bulk Pre-stageing Tests Graeme Stewart University of Glasgow.

Network awareness and network as a resource (and its integration with WMS) Artem Petrosyan (University of Texas at Arlington) BigPanDA Workshop, CERN,

EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.

SLACFederated Storage Workshop Summary For pre-GDB (Data Access) Meeting 5/13/14 Andrew Hanushevsky SLAC National Accelerator Laboratory.

Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT DPM / LFC and FTS news Ricardo Rocha ( on behalf of the IT/GT/DMS.

Efi.uchicago.edu ci.uchicago.edu FAX status report Ilija Vukotic on behalf of the atlas-adc-federated-xrootd working group S&C week Jun 2, 2014.

The ATLAS Cloud Model Simone Campana. LCG sites and ATLAS sites LCG counts almost 200 sites. –Almost all of them support the ATLAS VO. –The ATLAS production.

PanDA Status Report Kaushik De Univ. of Texas at Arlington ANSE Meeting, Nashville May 13, 2014.

FAX PERFORMANCE TIM, Tokyo May PERFORMANCE TIM, TOKYO, MAY 2013ILIJA VUKOTIC 2  Metrics  Data Coverage  Number of users.

PERFORMANCE AND ANALYSIS WORKFLOW ISSUES US ATLAS Distributed Facility Workshop November 2012, Santa Cruz.

Plans for Service Challenge 3 Ian Bird LHCC Referees Meeting 27 th June 2005.

ALICE DATA ACCESS MODEL Outline 05/13/2014 ALICE Data Access Model 2  ALICE data access model  Infrastructure and SE monitoring.

FAX UPDATE 12 TH AUGUST Discussion points: Developments FAX failover monitoring and issues SSB Mailing issues Panda re-brokering to FAX Monitoring.

MND review. Main directions of work  Development and support of the Experiment Dashboard Applications - Data management monitoring - Job processing monitoring.

Efi.uchicago.edu ci.uchicago.edu Data Federation Strategies for ATLAS using XRootD Ilija Vukotic On behalf of the ATLAS Collaboration Computation and Enrico.

LCG Service Challenges SC2 Goals Jamie Shiers, CERN-IT-GD 24 February 2005.

Efi.uchicago.edu ci.uchicago.edu Ramping up FAX and WAN direct access Rob Gardner on behalf of the atlas-adc-federated-xrootd working group Computation.

Efi.uchicago.edu ci.uchicago.edu Storage federations, caches & WMS Rob Gardner Computation and Enrico Fermi Institutes University of Chicago BigPanDA Workshop.

WLCG Operations Coordination Andrea Sciabà IT/SDC 10 th July 2013.

ATLAS Distributed Computing ATLAS session WLCG pre-CHEP Workshop New York May 19-20, 2012 Alexei Klimentov Stephane Jezequel Ikuo Ueda For ATLAS Distributed.

Summary of SC4 Disk-Disk Transfers LCG MB, April Jamie Shiers, CERN.

XRootD Monitoring Report A.Beche D.Giordano. Outlines  Talk 1: XRootD Monitoring Dashboard  Context  Dataflow and deployment model  Database: storage.

News from the HEPiX IPv6 Working Group David Kelsey (STFC-RAL) HEPIX, BNL 13 Oct 2015.

Acronyms GAS - Grid Acronym Soup, LCG - LHC Computing Project EGEE - Enabling Grids for E-sciencE.

PanDA Configurator and Network Aware Brokerage Fernando Barreiro Megino, Kaushik De, Tadashi Maeno 14 March 2015, US ATLAS Distributed Facilities Meeting,

SLACFederated Storage Workshop Summary Andrew Hanushevsky SLAC National Accelerator Laboratory April 10-11, 2014 SLAC.

Data Distribution Performance Hironori Ito Brookhaven National Laboratory.

Efi.uchicago.edu ci.uchicago.edu FAX splinter session Rob Gardner Computation and Enrico Fermi Institutes University of Chicago ATLAS Tier 1 / Tier 2 /

Efi.uchicago.edu ci.uchicago.edu Federating ATLAS storage using XrootD (FAX) Rob Gardner on behalf of the atlas-adc-federated-xrootd working group Computation.

US ATLAS DDM Operations Alexei Klimentov, BNL US ATLAS Tier-2 Workshop UCSD, Mar 8 th 2007.

Efi.uchicago.edu ci.uchicago.edu Sharing Network Resources Ilija Vukotic Computation and Enrico Fermi Institutes University of Chicago Federated Storage.

ATLAS Computing Model Ghita Rahal CC-IN2P3 Tutorial Atlas CC, Lyon

HEPiX IPv6 Working Group David Kelsey (STFC-RAL) GridPP33 Ambleside 22 Aug 2014.

Efi.uchicago.edu ci.uchicago.edu FAX status report Ilija Vukotic on behalf of the atlas-adc-federated-xrootd working group Computation and Enrico Fermi.

Efi.uchicago.edu ci.uchicago.edu Caching FAX accesses Ilija Vukotic ADC TIM - Chicago October 28, 2014.

Efi.uchicago.edu ci.uchicago.edu FAX splinter session Rob Gardner Computation and Enrico Fermi Institutes University of Chicago ATLAS Tier 1 / Tier 2 /

Computing Operations Roadmap

LHCOPN update Brookhaven, 4th of April 2017

FDR readiness & testing plan

Brookhaven National Laboratory Storage service Group Hironori Ito

LHC Data Analysis using a worldwide computing grid

IPv6 update Duncan Rand Imperial College London

Presentation transcript:

efi.uchicago.edu ci.uchicago.edu FAX status report Ilija Vukotic Computation and Enrico Fermi Institutes University of Chicago US ATLAS Computing Integration and Operations Meeting April 16, 2014

efi.uchicago.edu ci.uchicago.edu 2 Global Deployment Status Cloud T1sT2DsT2 DoneTotalDoneTotalDoneTotal CA US FR DE IT ES NL UK TW ND CERN RU Total SFU-LCG2 (CA cloud) joined – Direct dCache – Connected to xrd- central Mayor T1s missing – CA –TRIUMF – good progress – ES – pic – working with them – FR – Lyon – progress stalled – NL – Sara, Nikhef – no progress

efi.uchicago.edu ci.uchicago.edu 3 North American Redirectors Restructuring of the redirection tree in North America Hosted at BNL; FAX operations team provides configuration and operational support – atlas-xrd-us.usatlas.org o North American region – atlas-xrd-east.usatlas.org o - BNL, BU, Victoria, Scinet – atlas-xrd-central.usatlas.org o MWT2, AGLT2, OU, SWT2 – atlas-xrd-west.usatlas.org o SLAC, TRIUMF, McGill, SFU Status and timeframe: – Hosts and configurations ready. Waiting for git repository’s connection.

efi.uchicago.edu ci.uchicago.edu 4 Global SSB: most sites are stable

efi.uchicago.edu ci.uchicago.edu 5 US FAX status: past three months

efi.uchicago.edu ci.uchicago.edu 6 Operational Network saturation occurrence at FZK-LCG2 ALICE analysis is suspect If it needs an analysis to find who/what is responsible, then the monitoring is inadequate. FTS XROOT Throughput Transfers finished

efi.uchicago.edu ci.uchicago.edu 7 Cost Matrix - US Averaged over last week DEFINITION: Rate in MB/s measured between WN and a FAX endpoint

efi.uchicago.edu ci.uchicago.edu 8 Failover Statistics – last two weeks

efi.uchicago.edu ci.uchicago.edu 9 Testing & Development New dCache redirector plugin developed – Provides simpler federating, improves efficiency and stability of dCache endpoints. 100Gbps tests as soon as MWT2-BNL circuit available Testing overflow now the new test pilot is available Working with EOS on finding solution to a too many DDM endpoints issue Wei continuously testing using input files from analysis jobs. Restarting frameworks measuring and following IO characteristics of ATLAS data formats – in particular xAOD.

efi.uchicago.edu ci.uchicago.edu 10 Cost matrix YesterdayLast 30 days

efi.uchicago.edu ci.uchicago.edu 11 EXTRA