1. March 1, 2016 1 CDF Grid Daejung Kong and Kihyeon Cho* (High Energy Physics Team, KISTI) * Corresponding author June 25, 2009 Korea CDF Meeting, SNU, Korea

2. Contents Supercomputing Center at KISTI Change of Man Powers @ CDF Grid Network test between KISTI-Fermilab CDF Grid Transition Pacific CAF Summary 2

3. 3 응용지원실 융합자원실 응용지워팀 선도연구팀선도연구팀 슈퍼컴 자원팀 연구망 자원팀 사이버인프라 응용연구자 KISTI 슈퍼컴퓨팅 본부 SC-SP SMB 팀 과학기술정보보호팀 차세대연구환경개발실 기반기술개발실 HEP 연구 팀 응용환경개발팀 응용서비스팀 서비스기반기술팀 e-Cell 팀 NSDC 구축팀 과학기술융합망팀 GEOSS 팀 일반 사용자 전략분야 사용자 서비스기술 사용자 협업가시화팀 e-Science 그리드기술개발팀 Ref. 이지수본부장 (2009.3.12)

4. 4 Who is who? 고에너지물리연구팀 (Physicists) – 조기현 ( 팀장 ) – 김정현, 공대정, 남수현 e-Science 그리드 기술개발팀 (Developers) – 황순욱 ( 팀장 ) – 유상수, 김유상, 김남규, 안선일 등 NSDC 구축팀 (Computing Technicians) – 장행진 ( 팀장 ) – 김법균, 유진승, 윤희준, Christopher 등 Other teams – 김현우, 이상동, 김정한, 홍태용 등

5. Change of Manpower @ CDF Grid Donatatella quitted off-line co-heads – Rick Snider and Rick StDennis are the offline co-heads since April 2009. Hyunwoo Kim quitted SAM DH shift on May 2009 – 1 week per every 6 week – Daejung Kong will replace Hyunwoo Kim. Minho Jeung quitted system manger of PacCAF. – Daejung Kong replaces Minho Jeung. 5

6. 6 KEK GLORIAD-KR/10Gbps HongKong CSTNet-APAN-JP/1Gbps Fermilab GLORIAD-KR/10Gbps Seattle ESnet/10Gbps CERN/IN2P3 GLORIAD/1Gbps Chicago StarLight/10Gbps KISTI Networks BW: Upper bound of TCP throughput MSS: Max Segment Size RTT: Round Trip Time P: Packet Loss Mathis Equation

7. 2016년 3월 1일 2016년 3월 1일 2016년 3월 1일 7 KISTI-Fermilab Jan. 6~20, 2009 (2 weeks) Packet loss is zero. RTT is stable. We avoid Chicago area congestions by direct network from Seattle.

8. Iperf test from Fermilab-KISTI (4/20/09) Iperf test from FNAL to KISTI root@lhcopnmon1-mgm wenji]# iperf -c 134.75.29.10 -t 50 -i 5 ------------------------------------------------------------ Client connecting to 134.75.29.10, TCP port 5001 TCP window size: 1.00 MByte (default) ------------------------------------------------------------ [ 3] local 131.225.204.145 port 59361 connected with 134.75.29.10 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 5.50 MBytes 9.23 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 10.8 MBytes 18.2 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 24.0 MBytes 40.2 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 15.0-20.0 sec 53.3 MBytes 89.5 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 20.0-25.0 sec 86.3 MBytes 145 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 25.0-30.0 sec 88.8 MBytes 149 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 30.0-35.0 sec 91.1 MBytes 153 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 35.0-40.0 sec 89.8 MBytes 151 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 40.0-45.0 sec 89.6 MBytes 150 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 45.0-50.0 sec 87.3 MBytes 146 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-50.0 sec 627 MBytes 105 Mbits/sec 8 The round trip time is around 167 ms, which is high, but typical for trans-pacific network paths.

9. Iperf test from KISTI-Fermilab Iperf test from KISTI to FNAL kreonet@0[owamp]$ iperf -c lhcopnmon1-mgm.fnal.gov -t 50 -i 5 -w 2M ------------------------------------------------------------ Client connecting to lhcopnmon1-mgm.fnal.gov, TCP port 5001 TCP window size: 4.00 MByte (WARNING: requested 2.00 MByte) ------------------------------------------------------------ [ 3] local 134.75.29.10 port 44959 connected with 131.225.110.144 port 5001 [ 3] 0.0- 5.0 sec 58.4 MBytes 98.0 Mbits/sec [ 3] 5.0-10.0 sec 81.3 MBytes 136 Mbits/sec [ 3] 10.0-15.0 sec 82.5 MBytes 138 Mbits/sec [ 3] 15.0-20.0 sec 81.4 MBytes 137 Mbits/sec [ 3] 20.0-25.0 sec 81.2 MBytes 136 Mbits/sec [ 3] 25.0-30.0 sec 81.3 MBytes 136 Mbits/sec [ 3] 30.0-35.0 sec 81.6 MBytes 137 Mbits/sec [ 3] 35.0-40.0 sec 81.4 MBytes 137 Mbits/sec [ 3] 40.0-45.0 sec 82.9 MBytes 139 Mbits/sec [ 3] 45.0-50.0 sec 81.3 MBytes 136 Mbits/sec [ 3] 0.0-50.0 sec 793 MBytes 133 Mbits/sec 9

10. Packet Trace from Fermilab to KISTI 2016년 3월 1일 2016년 3월 1일 2016년 3월 1일 10

11. 11 Ref. Dr. Donatella Lucchesi

12. 12 CDF Grid (2009.4] Grid CAFHead nodeWork node Grid Middleware Method Data Handling NAM CAF Fermilab (USA) USSD (USA), etc. OSG Condor over Globus CDF Grid Fermilab (USA) Fermilab (USA), etc. OSG Condor over Globs Data Handling Lcg CAF CNAF (Italia) IN2P3 (France), etc. gLite WMS (Workload Management System) CNAF CAF CNAF (Italia) CNAF (Iltaia) gLIteGlideinWMS Partially SAM+SRM Pac CAFAS (Taiwan) KISTI (Korea) Etc. OSG, gLite Condor over Globus

13. 13 Pac CAF at KISTI Service Component Legend Repository Grid CAF BoundaryGrid Site BoundaryGrid Pool Job Data Transportation Control Data Transportation Grid Site (KR-KISTI-HEP) Worker Node (hep015) ……………… (hep004) Work Node (hep003) dCache Pool (se-hep) Grid Farm (Pacific CAF) VOMS Authentication Glideins dCache User Interface CafGui CafSubmit CafMon CAF Head Node Submitter Monitor Collector Negotiator Schedd Globus.... Job Submission Web monitoring Job Output Condor CE (hep001).... 25 core processors 45 TB

14. 2016년 3월 1일 2016년 3월 1일 2016년 3월 1일 14 CDF Resource Usage - Total Ref. Dr. Donatella Lucchesi

15. Relocation of Pacific CAF Date: June 1 ~ 12 From Cluster Room to Supercomputing Room More stable condition – UPS power and airconditioner Host CA replacement (June 25) 25 core process, 45 TB 15

16. 16 CDF Grid Computing Center @ KISTI Pacific CAF @ KISTICGCC @ KISTI Head nodepaccaf.phys.sinica.edu.twNew Head node VOMS servervoms.cnaf.infn.itsame Work nodes 25 core processes (gLite) 500 core processes? (gLite, OSG) Storage30 T byte250 T byte? Data HandlingN/AAvailable? PacCAF => CGCC

17. 17 Summary Data processing for CDF has been developed from CAF (Central Analysis Farm) to Grid. KISTI is one of leading institutes of CDF data processing.

18. 18 Thank you. (http://hep.kisti.re.kr)

19. 2016년 3월 1일 2016년 3월 1일 2016년 3월 1일 19

20. 20 Offsite Usage – USA & Asia Donatella clearly mentioned about KISTI in International Finance Committee Meeting on November ’08. => CGCC (CDF Grid Computing Center)

21. 4/20/09 Iperf bandwidth Test – For bandwidth test, the throughput for single stream for both directions can reach beyond 100Mbps. This throughput performance is not bad for large RTT network paths. – The packet trace analysis does not show packet drops and reordering. – The FNAL’s TCP congestion control algorithm is CUBIC; KISTI’s TCP congestion control algorithm is RENO. Suggest KISTI changing its TCP congestion control algorithm to CUBIC or other high-throughput TCP variants. 2016년 3월 1일 2016년 3월 1일 2016년 3월 1일 21

22. 22 Monitoring of Pac CAF