ATLAS STEP09 UK T2 Activity

Slides:

Advertisements

Similar presentations

London Tier2 Status O.van der Aa. Slide 2 LT 2 21/03/2007 London Tier2 Status Current Resource Status 7 GOC Sites using sge, pbs, pbspro –UCL: Central,

Advertisements

GridPP Status David Britton, 3/Sep/ /03/2014 Switching on the LHC The LHC was fully cold by mid August. This is being followed by continued powering.

S.L.LloydATSE e-Science Visit April 2004Slide 1 GridPP – A UK Computing Grid for Particle Physics GridPP 19 UK Universities, CCLRC (RAL & Daresbury) and.

Deployment metrics and planning (aka Potentially the most boring talk this week) GridPP16 Jeremy Coles 27 th June 2006.

Your university or experiment logo here BaBar Status Report Chris Brew GridPP16 QMUL 28/06/2006.

Northgrid Status Alessandra Forti Gridpp24 RHUL 15 April 2010.

GridPP Building a UK Computing Grid for Particle Physics A PPARC funded project.

Slide 1 of 24 Steve Lloyd NW Grid Seminar - 11 May 2006 GridPP and the Grid for Particle Physics Steve Lloyd Queen Mary, University of London NW Grid Seminar.

UK Agency for the support of: High Energy Physics - the nature of matter and mass Particle Astrophysics - laws from natural phenomena Astronomy - the.

GridPP From Prototype to Production David Britton 21/Sep/06 1.Context – Introduction to GridPP 2.Performance of the GridPP/EGEE/wLCG Grid 3.Some Successes.

Tony Doyle “GridPP2 Proposal”, GridPP7 Collab. Meeting, Oxford, 1 July 2003.

Southgrid Status Pete Gronbech: 27th June 2006 GridPP 16 QMUL.

Wahid Bhimji Andy Washbrook And others including ECDF systems team Not a comprehensive update but what ever occurred to me yesterday.

15/07/2010Swiss WLCG Operations Meeting Summary of the last GridKA Cloud Meeting (07 July 2010) Marc Goulette (University of Geneva)

London Tier 2 Status Report GridPP 13, Durham, 4 th July 2005 Owen Maroney, David Colling.

UKI-SouthGrid Overview Face-2-Face Meeting Pete Gronbech SouthGrid Technical Coordinator Oxford June 2013.

Slide David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow GridPP Vendor Day 30 th April.

December 17th 2008RAL PPD Computing Christmas Lectures 11 ATLAS Distributed Computing Stephen Burke RAL.

Quarterly report SouthernTier-2 Quarter P.D. Gronbech.

А.Минаенко Совещание по физике и компьютингу, 16 сентября 2009 г., ИФВЭ, Протвино Текущее состояние и ближайшие перспективы компьютинга для АТЛАСа в России.

Computing Infrastructure Status. LHCb Computing Status LHCb LHCC mini-review, February The LHCb Computing Model: a reminder m Simulation is using.

LT 2 London Tier2 Status Olivier van der Aa LT2 Team M. Aggarwal, D. Colling, A. Fage, S. George, K. Georgiou, W. Hay, P. Kyberd, A. Martin, G. Mazza,

12th November 2003LHCb Software Week1 UK Computing Glenn Patrick Rutherford Appleton Laboratory.

Monitoring the Grid at local, national, and Global levels Pete Gronbech GridPP Project Manager ACAT - Brunel Sept 2011.

1 st EGEE Conference – April UK and Ireland Partner Dave Kant Deputy ROC Manager.

Concept: Well-managed provisioning of storage space on OSG sites owned by large communities, for usage by other science communities in OSG. Examples –Providers:

GridPP3 project status Sarah Pearce 14 April 2010 GridPP24 RHUL.

John Gordon STFC-RAL Tier1 Status 9 th July, 2008 Grid Deployment Board.

А.Минаенко Совещание по физике и компьютингу, 03 февраля 2010 г. НИИЯФ МГУ, Москва Текущее состояние и ближайшие перспективы компьютинга для АТЛАСа в России.

Project Management Sarah Pearce 3 September GridPP21.

John Gordon CCLRC e-Science Centre LCG Deployment in the UK John Gordon GridPP10.

GridPP Deployment & Operations GridPP has built a Computing Grid of more than 5,000 CPUs, with equipment based at many of the particle physics centres.

Wahid, Sam, Alastair. Now installed on production storage Edinburgh: srm.glite.ecdf.ed.ac.uk  Local and global redir work (port open) e.g. root://srm.glite.ecdf.ed.ac.uk//atlas/dq2/mc12_8TeV/NTUP_SMWZ/e1242_a159_a165_r3549_p1067/mc1.

11 March 2008 GridPP20 Collaboration meeting David Britton - University of Glasgow GridPP Status GridPP20 Collaboration Meeting, Dublin David Britton,

GridPP: Executive Summary Tony Doyle. Tony Doyle - University of Glasgow Oversight Committee 8 February 2007 Outline Exec 2 Summary Grid status High level.

UK middleware deployment GridPP27 - CERN 15 th September 2011 GridPP27 - CERN 15 th September 2011 Status & plans Jeremy Coles.

GridPP Building a UK Computing Grid for Particle Physics Professor Steve Lloyd, Queen Mary, University of London Chair of the GridPP Collaboration Board.

GridPP Deployment Status GridPP15 Jeremy Coles 11 th January 2006.

London Tier 2 Status Report GridPP 11, Liverpool, 15 September 2004 Ben Waugh on behalf of Owen Maroney.

US ATLAS Computing Operations Kaushik De University of Texas At Arlington U.S. ATLAS Tier 2/Tier 3 Workshop, FNAL March 8, 2010.

GridPP3 project status Sarah Pearce 24 April 2010 GridPP25 Ambleside.

Data Management: US Focus Kaushik De, Armen Vartapetian Univ. of Texas at Arlington US ATLAS Facility, SLAC Apr 7, 2014.

EGI-InSPIRE EGI-InSPIRE RI DDM solutions for disk space resource optimization Fernando H. Barreiro Megino (CERN-IT Experiment Support)

Performance analysis extracts from GridPP OC metrics report For UKI operations meeting 15 th June 2005.

14/03/2007A.Minaenko1 ATLAS computing in Russia A.Minaenko Institute for High Energy Physics, Protvino JWGC meeting 14/03/07.

RALPP Site Report HEP Sys Man, 11 th May 2012 Rob Harper.

J Jensen/J Gordon RAL Storage Storage at RAL Service Challenge Meeting 27 Jan 2005.

Slide § David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow GridPP delivering The UK Grid.

18/12/03PPD Christmas Lectures 2003 Grid in the Department A Guide for the Uninvolved PPD Computing Group Christmas Lecture 2003 Chris Brew.

Status: ATLAS Grid Computing

London Tier-2 Quarter Owen Maroney

Ian Bird WLCG Workshop San Francisco, 8th October 2016

Xiaomei Zhang CMS IHEP Group Meeting December

GridPP Storage perspectives

Simone Campana CERN-IT

ATLAS activities in the IT cloud in April 2008

Understanding the nature of matter -

The Data Lifetime model

LHCb Computing Model and Data Handling Angelo Carbone 5° workshop italiano sulla fisica p-p ad LHC 31st January 2008.

Philippe Charpentier CERN – LHCb On behalf of the LHCb Computing Group

Readiness of ATLAS Computing - A personal view

UK Status and Plans Scientific Computing Forum 27th Oct 2017

Building a UK Computing Grid for Particle Physics

ATLAS Sites Jamboree, CERN January, 2017

Tier-1 Status Progress and Difficulties A View from RAL

FDR readiness & testing plan

CC and LQCD dimanche 13 janvier 2019dimanche 13 janvier 2019

Tests with VOLATILE disk

The LHCb Computing Data Challenge DC06

SEPTEMBER 2019 ISSUE/42 nd VOLUME.

Presentation transcript:

ATLAS STEP09 UK T2 Activity Brian Davies

pre-GDB dCache T2 Feedback Highest Rate 1.4GB/s pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback UK Overall did well with pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback T1 Jobs No User Analysis/HammerCLoud Most Jobs failures due to Storage Issues 55% Failures occurred in 3% of time STORAGE STABILITY A LARGE FACTOR IN JOB SUCCESS!! pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback T2 Production Success pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Site Balancing CPU Pledged CPUs seen Disk pledge Disk Delivered cpu/disk pledge pledge/del cpu pledge/del disk CPU % Disk % LANC’ 351 540 171 151.72 2.05 1.54 0.89 0.080 0.155 0.52 GLA’ 859 1651 417 145.14 2.06 1.92 0.35 0.244 0.148 1.65 MAN’ 804 780 391 86.79 0.97 0.22 0.115 0.088 1.30 PPD 346 400 168 85 1.16 0.51 0.059 0.087 0.68 QMUL 161 1115 78 29.46 6.93 0.38 0.165 0.030 5.49 RHUL 144 420 70 139 2.92 1.99 0.062 0.142 0.44 LIV’ 114 650 55 86.84 2.07 5.70 1.58 0.096 1.09 SHEF’ 19 176 9 38.7 2.11 9.26 4.30 0.026 0.039 0.66 UCL 16 45 8 46.2 2.00 2.81 5.78 0.007 0.047 0.14 OX’ 119 350 58 71.06 2.94 1.23 0.052 0.072 0.71 CAM’ 35 180 17 34.09 5.14 2.01 0.027 0.035 0.77 B’HAM 23 60 11 37.38 2.09 2.61 3.40 0.009 0.038 0.23 DUR’ 66 32 30 6.06 0.94 0.031 1.93 3057 6767 1485 981.38 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Failure Modes pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback T2 HammerCloud Results UKI-LT2-QMUL_MCDISK 150 1488 UKI-LT2-RHUL_MCDISK 100 400 UKI-LT2-UCL-CENTRAL_MCDISK 50 128 UKI-NORTHGRID-LANCS-HEP_MCDISK 656 UKI-NORTHGRID-LIV-HEP_MCDISK 760 UKI-NORTHGRID-MAN-HEP1_MCDISK 890 UKI-NORTHGRID-MAN-HEP2_MCDISK UKI-NORTHGRID-SHEF-HEP_MCDISK 200 UKI-SCOTGRID-GLASGOW_MCDISK 250 1912 UKI-SOUTHGRID-BHAM-HEP_MCDISK 10 308 UKI-SOUTHGRID-CAM-HEP_MCDISK 152 UKI-SOUTHGRID-OX-HEP_MCDISK 75 1152 UKI-SOUTHGRID-RALPP_MCDISK 1020 ANALY_BHAM 10 308 ANALY_CAM 50 152 ANALY_GLASGOW 250 1912 ANALY_LANCS 100 656 ANALY_LIV 150 760 ANALY_MANC1 890 ANALY_MANC2 ANALY_OX 75 1152 ANALY_QMUL 1488 ANALY_RALPP 1020 ANALY_RHUL 400 ANALY_SHEF 200 ANALY_UCLCENT 128 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback T2 HammerCloud Results UK_PANDA SUBMITTED RUNNING COMPLETED FAILED Efficiency # Files Expected # Files Processed ANALY_CAM 3045 1656 0.65 10805 ? ANALY_BHAM 181 2640 275 0.91 5624 ANALY_MANC1 5033 2915 0.63 10159 ANALY_MANC2 1 3567 5780 0.38 8695 ANALY_GLASGOW 3 73726 1314 0.98 189702 ANALY_SHEF 830 5106 1436 0.78 11701 ANALY_RHUL 14657 3220 0.82 36759 ANALY_LANCS 3472 2339 0.6 7929 ANALY_UCLCENT 241 123 ANALY_LIV 7642 1660 27628 ANALY_OX 169 128 ANALY_RALPP 8 1612 907 0.64 5713 ANALY_QMUL 1475 2619 0.36 3397 TOTAL 1425 15 121975 24372 0.83 318112 84.9% World Ave pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback T2 HammerCloud Results UK SUBMITTED RUNNING COMPLETED FAILED Efficiency # Files Expected # Files Processed UKI-NORTHGRID-LANCS-HEP_MCDISK 405 903 0.31 1472 1311 UKI-SCOTGRID-DURHAM_MCDISK 22 532 0.04 163 154 UKI-LT2-UCL-CENTRAL_MCDISK 105 665 0.14 612 435 UKI-SOUTHGRID-BHAM-HEP_MCDISK 102 11 439 363 0.55 2294 2011 UKI-LT2-QMUL_MCDISK 298 31 853 13335 0.06 2268 772 UKI-NORTHGRID-SHEF-HEP_MCDISK 5 570 0.01 36 UKI-NORTHGRID-LIV-HEP_MCDISK 189 286 1267 995 0.56 9707 9705 UKI-SOUTHGRID-OX-HEP_MCDISK 140 8 3891 739 0.84 30128 29006 UKI-SCOTGRID-GLASGOW_MCDISK 376 594 2669 817 0.77 14398 13161 UKI-LT2-RHUL_MCDISK 158 28 6849 5532 29077 23287 UKI-SOUTHGRID-CAM-HEP_MCDISK 576 630 0.48 5288 3973 UKI-SOUTHGRID-RALPP_MCDISK 113 181 6634 797 0.89 40356 38965 TOTAL 1376 1139 23715 25878 135799 122816 74% World Ave pre-GDB dCache T2 Feedback 18 September, 2018

Total UK FTS For ATLAS During STEP’09 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback ScotGrid Site Token Total Size (TB) Unused Size (TB) UKI-SCOTGRID-DURHAM - 9.9 8 DATADISK 2.2 PRODDISK 2.1 MCDISK 5.5 3.8 UKI-SCOTGRID-GLASGOW 153.9 88.3 70.4 51.1 40.7 2.9 4.4 USERDISK 1.1 0.9 PPSDATADISK 15.4 15.3 SCRATCHDISK LOCALGROUPDISK UKI-SCOTGRID-ECDF 4.9 0.5 0.2 1 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback NorthGrid Site Token Total Size (TB) Unused Size (TB) UKI-NORTHGRID-LIV-HEP - 83.5 42 DATADISK 33 19.4 MCDISK 31.9 9.6 PRODDISK 2.2 2 USERDISK 0.1 SCRATCHDISK 5.5 4.9 LOCALGROUPDISK 10.9 6 UKI-NORTHGRID-MAN-HEP 83.6 65.2 HEP1_DATADISK 18.7 18.1 HEP1_MCDISK 11 HEP1_PRODDISK 5.3 HEP1_SCRATCHDISK 3.3 3.1 -HEP2_DATADISK HEP2_MCDISK 4.4 HEP2_PRODDISK 5.4 HEP2_SCRATCHDISK HEP1_LOCALGROUPDISK 2.3 HEP2_LOCALGROUPDISK Site Token Total Size (TB) Unused Size (TB) UKI-NORTHGRID-LANCS-HEP - 151.7 99.2 DATADISK 59.4 45.1 MCDISK 61.6 30.2 PRODDISK 11 10.6 USERDISK 1.1 0.9 SCRATCHDISK 6.6 5.5 PHYS-BEAUTY 4.4 4.1 LOCALGROUPDISK 7.7 2.8 UKI-NORTHGRID-SHEF-HEP 34.3 22.6 16.5 16.2 13.2 2.9 2.2 1.5 0.2 0.8 OCALGROUPDISK pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback SouthGrid Site Token Total Size (TB) Unused Size (TB) UKI-SOUTHGRID-BHAM-HEP - 35.2 28.6 DATADISK 13.2 12.8 MCDISK 5.5 1.3 PRODDISK 2.2 SCRATCHDISK 11 10.9 LOCALGROUPDISK 3.3 UKI-SOUTHGRID-CAM-HEP 31.9 16.8 16.5 8.3 _MCDISK 6.6 1.4 1.6 USERDISK 3.1 2.4 Site Token Total Size (TB) Unused Size (TB) UKI-SOUTHGRID-OX-HEP - 68.9 33.1 DATADISK 22 8.4 MCDISK 28.6 10 PRODDISK 2.2 1.9 USERDISK 0.7 0.3 SCRATCHDISK 8.8 8 LOCALGROUPDISK 6.6 4.6 UKI-SOUTHGRID-RALPP 78.8 58.3 35 33.9 30 11.4 2 1.8 2.5 2.3 9 8.9 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Raw numbers LT2 Site Token Total Size (TB) Unused Size (TB) UKI-LT2-RHUL - 188.1 125.8 DATADISK 94.6 80.3 MCDISK 49.5 5.9 PRODDISK 5.5 4.1 USERDISK 0.1 SCRATCHDISK 6.6 5.4 LOCALGROUPDISK 31.9 30.1 UKI-LT2-Brunel 2.7 2.6 0.5 2.2 2.1 UKI-LT2-UCL-CENTRAL 44 37.9 17.6 16.8 11 5.7 GROUPDISK 8.8 UKI-LT2-QMUL 121.5 96.9 75.2 74.9 26.8 3.7 10.7 10.6 5.3 3.3 2.4 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback STEP’09 Data Placement Token Share (%) Volume (TB) Reserved (TB) Headroom (TB) DS complete nFiles Actual (TB) UKI-LT2-QMUL_DATADISK 5 5.6 75.2 69.6 685/685 2649 6 UKI-LT2-RHUL_DATADISK 23 25.8 94.6 68.8 3126/3126 12496 28 UKI-LT2-UCL-CENTRAL_DATADISK 10 11.2 17.6 6.4 1363/1363 5150 11.8 UKI-NORTHGRID-LANCS-HEP_DATADISK 30 33.6 59.4 4075/4075 20213 42.8 UKI-NORTHGRID-LIV-HEP_DATADISK 15 16.8 33 16.2 2018/2018 6485 15.6 UKI-NORTHGRID-MAN-HEP1_DATADISK 18.7 1.9 2041/2041 7430 17.1 UKI-NORTHGRID-MAN-HEP2_DATADISK 25.3 8.5 10642 22.3 UKI-NORTHGRID-SHEF-HEP_DATADISK 16.5 5.3 6302 13.6 UKI-SCOTGRID-GLASGOW_DATADISK 40 44.8 70.4 25.6 5359/5359 16720 UKI-SOUTHGRID-BHAM-HEP_DATADISK 13.2 7.6 2027 UKI-SOUTHGRID-CAM-HEP_DATADISK 10.9 2008 UKI-SOUTHGRID-OX-HEP_DATADISK 2 2.2 22 19.8 277/277 1533 3.2 UKI-SOUTHGRID-RALPP_DATADISK 25 35 7 3355/3360 14965 32.4 Totals 200 224 497.2 273.2 175.6 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback CERN Traffic pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback SJ5 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Lancaster Lightpath pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback UK Production pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Submissions pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Gen pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Gen pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback North pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback North pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback North pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback South pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback South pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback South pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback scot pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Lt2 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Lt2 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback FTS Outputs 2nd Week Ends 5pm Friday 12th June pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 2nd week step fts plots pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback general pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback general pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Gen atlas as source pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Gen atlas as dest pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback North pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback North pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback North pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback South pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback South pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback South pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Scot pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Lt2 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Lt2 pre-GDB dCache T2 Feedback 18 September, 2018

pre-GDB dCache T2 Feedback Lt2 pre-GDB dCache T2 Feedback 18 September, 2018