FAX Redirection Topology Wei Yang 7/16/121. Redirector hardware at CERN Redundant redirectors for EU, UK, DE, FR – Redundant (the “+” sign below) VMs.

Slides:

Advertisements

Similar presentations

Potential Data Access Architectures using xrootd OSG All Hands Meeting Harvard University March 7-11, 2011 Andrew Hanushevsky, SLAC

Advertisements

Symon Perriman Program Manager II Clustering & High-Availability Microsoft Corporation SESSION CODE: VIR303.

Distributed Xrootd Derek Weitzel & Brian Bockelman.

Xrootd Update OSG All Hands Meeting University of Nebraska March 19-23, 2012 Andrew Hanushevsky, SLAC

Copy on Demand with Internal Xrootd Federation Wei Yang SLAC National Accelerator Laboratory Create Federated Data Stores for the LHC IN2P3-CC,

Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT dpm-xrootd v3 Creating Federated Data Stores for the LHC David.

Duke and ANL ASC Tier 3 (stand alone Tier 3’s) Doug Benjamin Duke University.

FAX status. Overview Status of endpoints and redirectors Monitoring Failover Overflow.

1 Andrew Hanushevsky - HEPiX, October 6-8, 1999 Mass Storage For BaBar at SLAC Andrew Hanushevsky Stanford.

Outline Network related issues and thinking for FAX Cost among sites, who has problems Analytics of FAX meta data, what are the problems  The main object.

CVMFS AT TIER2S Sarah Williams Indiana University.

Efi.uchicago.edu ci.uchicago.edu FAX status report Ilija Vukotic Computation and Enrico Fermi Institutes University of Chicago US ATLAS Computing Integration.

The Next Generation Root File Server Andrew Hanushevsky Stanford Linear Accelerator Center 27-September-2004

Xrootd Demonstrator Infrastructure OSG All Hands Meeting Harvard University March 7-11, 2011 Andrew Hanushevsky, SLAC

Scalla/xrootd Andrew Hanushevsky SLAC National Accelerator Laboratory Stanford University 29-October-09 ATLAS Tier 3 Meeting at ANL

Multi-Tiered Storage with Xrootd at ATLAS Western Tier 2 Andrew Hanushevsky Wei Yang SLAC National Accelerator Laboratory 1CHEP2012, New York

Google Sitemaps Case Study Eric Papczun SES Chicago Bulk Submit 2.0 December 5 th, 2006.

Scalla/xrootd Introduction Andrew Hanushevsky, SLAC SLAC National Accelerator Laboratory Stanford University 6-April-09 ATLAS Western Tier 2 User’s Forum.

July-2008Fabrizio Furano - The Scalla suite and the Xrootd1 cmsd xrootd cmsd xrootd cmsd xrootd cmsd xrootd Client Client A small 2-level cluster. Can.

SLAC Experience on Bestman and Xrootd Storage Wei Yang Alex Sim US ATLAS Tier2/Tier3 meeting at Univ. of Chicago Aug 19-20,

11-July-2008Fabrizio Furano - Data access and Storage: new directions1.

Efi.uchicago.edu ci.uchicago.edu Towards FAX usability Rob Gardner, Ilija Vukotic Computation and Enrico Fermi Institutes University of Chicago US ATLAS.

Web Cache Redirection using a Layer-4 switch: Architecture, issues, tradeoffs, and trends Shirish Sathaye Vice-President of Engineering.

Wahid, Sam, Alastair. Now installed on production storage Edinburgh: srm.glite.ecdf.ed.ac.uk  Local and global redir work (port open) e.g. root://srm.glite.ecdf.ed.ac.uk//atlas/dq2/mc12_8TeV/NTUP_SMWZ/e1242_a159_a165_r3549_p1067/mc1.

July-2008Fabrizio Furano - The Scalla suite and the Xrootd1.

Architecture and ATLAS Western Tier 2 Wei Yang ATLAS Western Tier 2 User Forum meeting SLAC April

Redirector xrootd proxy mgr Redirector xrootd proxy mgr Xrd proxy data server N2N Xrd proxy data server N2N Global Redirector Client Backend Xrootd storage.

Status & Plan of the Xrootd Federation Wei Yang 13/19/12 US ATLAS Computing Facility Meeting at 2012 OSG AHM, University of Nebraska, Lincoln.

Efi.uchicago.edu ci.uchicago.edu FAX status developments performance future Rob Gardner Yang Wei Andrew Hanushevsky Ilija Vukotic.

02-June-2008Fabrizio Furano - Data access and Storage: new directions1.

Accelerating Debugging In A Highly Distributed Environment CHEP 2015 OIST Okinawa, Japan April 28, 2015 Andrew Hanushevsky, SLAC

Performance Tests of DPM Sites for CMS AAA Federica Fanzago on behalf of the AAA team.

Efi.uchicago.edu ci.uchicago.edu Status of the FAX federation Rob Gardner Computation and Enrico Fermi Institutes University of Chicago ATLAS Tier 1 /

SLACFederated Storage Workshop Summary For pre-GDB (Data Access) Meeting 5/13/14 Andrew Hanushevsky SLAC National Accelerator Laboratory.

CMS Issues. Background – RAL Infrastructure TM Nsd Xrd- mgr TM Nsd Xrd- mgr TM Rhd stagerd TGW Rhd stagerd TGW Cupv Vmgr Vdqm nsd Cupv Vmgr Vdqm nsd Cupv.

ATLAS XRootd Demonstrator Doug Benjamin Duke University On behalf of ATLAS.

Scalla Advancements xrootd /cmsd (f.k.a. olbd) Fabrizio Furano CERN – IT/PSS Andrew Hanushevsky Stanford Linear Accelerator Center US Atlas Tier 2/3 Workshop.

XRootD & ROOT Considered Root Workshop Saas-Fee September 15-18, 2015 Andrew Hanushevsky, SLAC

WEEK 11 – TOPOLOGIES, TCP/IP, SHARING & SECURITY IT1001- Personal Computer Hardware System & Operations.

Data & Storage Services CERN IT Department CH-1211 Genève 23 Switzerland t DSS XROOTD news New release New features.

Xrootd Proxy Service Andrew Hanushevsky Heinz Stockinger Stanford Linear Accelerator Center SAG September-04

Scalla In’s & Out’s xrootdcmsd xrootd /cmsd Andrew Hanushevsky SLAC National Accelerator Laboratory OSG Administrator’s Work Shop Stanford University/SLAC.

Efi.uchicago.edu ci.uchicago.edu Data Federation Strategies for ATLAS using XRootD Ilija Vukotic On behalf of the ATLAS Collaboration Computation and Enrico.

Federated Data Stores Volume, Velocity & Variety Future of Big Data Management Workshop Imperial College London June 27-28, 2013 Andrew Hanushevsky, SLAC.

1 Xrootd-SRM Andy Hanushevsky, SLAC Alex Romosan, LBNL August, 2006.

11-June-2008Fabrizio Furano - Data access and Storage: new directions1.

Bestman & Xrootd Storage System at SLAC Wei Yang Andy Hanushevsky Alex Sim Junmin Gu.

Introduce Caching Technologies using Xrootd Wei Yang 10/28/14ATLAS TIM 2014 Univ. Chicago1.

09-Apr-2008Fabrizio Furano - Scalla/xrootd status and features1.

An Analysis of Data Access Methods within WLCG Shaun de Witt, Andrew Lahiff (STFC)

Data Store: SQL File Server: Documents Drug Disease Demographics Treatment Geography Query Topological Association and Index in Middle Tier:

New Features of Xrootd SE Wei Yang US ATLAS Tier 2/Tier 3 meeting, University of Texas, Arlington,

SLACFederated Storage Workshop Summary Andrew Hanushevsky SLAC National Accelerator Laboratory April 10-11, 2014 SLAC.

Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Solutions for WAN data access: xrootd and NFSv4.1 Andrea Sciabà.

DPM in FAX (ATLAS Federation) Wahid Bhimji University of Edinburgh As well as others in the UK, IT and Elsewhere.

Efi.uchicago.edu ci.uchicago.edu FAX status report Ilija Vukotic on behalf of the atlas-adc-federated-xrootd working group Computation and Enrico Fermi.

The ALICE Analysis -- News from the battlefield Federico Carminati for the ALICE Computing Project CHEP 2010 – Taiwan.

Tom Byrne, Bruno Canning

Global Data Access – View from the Tier 2

Computing evolution of ATLAS and CMS ATLAS: Alessandro De Salvo CMS: Tommaso Boccali, Daniele Bonacorsi, Claudio Grandi Outline ATLAS computing.

Xrootd explained Cooperation among ALICE SEs

DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S

Data Federation with Xrootd Wei Yang US ATLAS Computing Facility meeting Southern Methodist University, Oct 11-12, 2011.

Storage elements discovery

Brookhaven National Laboratory Storage service Group Hironori Ito

Scalla/XRootd Advancements

How do You attend the meetings?

مقدمة في الشبكات أ.منال العبدلله.

Branch and Bound Example

Bioinformatics 김유환, 문현구, 정태진, 정승우.

Presentation transcript:

FAX Redirection Topology Wei Yang 7/16/121

Redirector hardware at CERN Redundant redirectors for EU, UK, DE, FR – Redundant (the “+” sign below) VMs – More to come atlas-xrd-eu.cern.ch+ – xrootd port 1094, cmsd port 1098 – SLAC test machine can join it, working as expected atlas-xrd-uk.cern.ch+ – Report to EU redirector – Xrootd port 1094, cmsd port ? – SLAC can’t join it, debugging … – Same for DE and FR redirectors 7/16/122

Redirector topology EU rdr DE rdr UK rdr US rdr Glasgow rdr Middle West rdr Site 1 rdr Site 2 rdr Site A rdr Site B rdr cmsd & xrootd redirection Mature xrootd redirection cmsd based redirection search the branch under it xrootd based redirection is used to jump to upper level if cmsd search return nothing US can either report to EU redirector, or as a peer of EU depend on needs, latency, or performance 7/16/123

# CMSD redirection: all.role manager all.manager leftregion:1234 # Upper level redirector all.manager meta xyzzy:1234 $ XROOTD redirection xrootd.redirect xyzzy:1234 ? / all.role manager all.manager rightregion:1234 all.manager meta xyzzy:1234 xrootd.redirect xyzzy:1234 ? / How the topology works Global Redirector “ xyzzy ” Regional Redirector Local RedirectorProxy ServerProxy ManagerLocal Redirector ServerProxy ServerServer all.role meta manager all.manager meta xyzzy:1234 all.role manager all.manager lefthand:1234 all.manager meta leftregion:1234 xrootd.redirect leftregion:1234 ? / all.role manager all.manager righthand:1234 all.manager meta rightregion:1234 xrootd.redirect rightregion:1234 ? / all.role server all.manager leftregion:1234 ofs.osslib psslib.so pss.origin mycluster:1094 all.role proxy manager all.manager meta leftregion:1234 xrootd.redirect xyzzy:1234 ? / Site DSite ASite BSite C 7/16/124Steal from Andrew Hanushevsky

Topology verification Goal: diagnose broken node in the topology Deploy site specific (small) file with known checksum Access from global redirector – Test full redirection chain + N2N For every lower level redirector – Test a file not exists in its domain How to find out that the actual topology is? Who know how do this manually We need to do it automatically and produce a graph 7/16/125