JPS Matsue1 PHENIX Computing Center in Japan (PHENIX CC-J) の採用技術 澤田真也（ KEK ） 市原卓、渡邊康（理研、理研 BNL 研究センター） 後藤雄二、竹谷篤、林直樹（理研） 延與秀人、四日市悟（京大）、浜垣秀樹（東大 CNS ）

PHENIX CC-J JPS Matsue2 CC-J の構成要素 Linux farm Data server HPSS Network Misc. softwares & tools

PHENIX CC-J JPS Matsue3 Linux farm Two boxes of AltaCluster –16 nodes = 32 CPUs (will be doubled soon) –Pentium II 450MHz (18.5 SpecINT95/cpu) –Remote boot, remote monitoring, … Linux RedHat5.2, Kernel with NFSv3 Patch PBS Batch Queuing System Memory: 512 MB/node Local Disk: 9-14 GB/node –Benchmark test (Bonnie):write xxMB/s, read xxMB/s NFS mounted RAID5 Disks on SUN E BaseT NIC on each node & Catalyst 2948G (gigabit Switching Hub)

PHENIX CC-J JPS Matsue4 AltaCluster

PHENIX CC-J JPS Matsue5 Data Server SUN E450: 400 MHz 2 CPU, 1GB Memory, 360GB Raid disk (One more E450 will be purchased soon.) –General ‘home’ machine 288GB Raid5 disk (1.6TB Raid5 will be purchased soon.) –Working space for users Alteon Ace 180 Gigabit Switch (Jumbo frame operation)

PHENIX CC-J JPS Matsue6 RAID performance measurement Preliminary measurement on 16 Apr 1999 (T. Ichihara RIKEN) Hardware SUN E450 (Dual Ultra2 sparc, 400MHz, 1280 MB Memoy)

PHENIX CC-J JPS Matsue7 NFS performance measurement Test with bonnie （ bonnie -s 100 : ） –from a Linux node to RAID on ccjsun with NFS –ap14 (kernel ) –ap15 (kernel NFSv3)  Use NFSv3!

PHENIX CC-J JPS Matsue8 HPSS (High Performance Storage System) Hierarchical storage system HPSS server (SP2 5-node 20-CPU, with SP switch and Gigabit NIC) 144 GB HPSS Cache disk (SSA Raid5) GB Work disk (Raid 5) HPSS 4.1.1, AIX STK Robot (4 RedWood drives, 100TB tape media) Alteon Ace 180 Gigabit Switch (Jumbo frame operation) Gigabit (jumbo frame) network and Hippi connection to SUN/Linux fpt or ‘pftp’ (parallel ftp) is used for data access between HPSS and SUN/Linux nodes.

PHENIX CC-J JPS Matsue9 Overview of HPSS-CCJ

PHENIX CC-J JPS Matsue10 HPSS Hardware

PHENIX CC-J JPS Matsue11 HPSS Software Configuration

PHENIX CC-J JPS Matsue12 STK Tape Robot Redwood drives: ~11MB/s/drive Currently we have 4 drives. Thus totally about 45MB/s can be achieved. 50GB/cartridge * 2000cartridges = 100TB Data (raw data and DSTs) will be transported with tape cartridges between RIKEN and BNL.

PHENIX CC-J JPS Matsue13 Network LAN –Gigabit ethernet with jumbo frame (9kB frame (normal: 1.5kB) available on AIX 4.2 or later) and HiPPI –Gbit has a similar performance with HiPPI –Gbit will be used. WAN –HEPNET-J/SINET between Japanese institutions –APAN between RIKEN and ESnet sites (BNL etc.)

PHENIX CC-J JPS Matsue14 Network Performance Test with netperf – More study needed to get nearly Gbit performance

PHENIX CC-J JPS Matsue15 Data Transfer Performance Test results with pftp (parallel ftp) between Linux nodes and HPSS –100BaseT on Linux limits the performance?

PHENIX CC-J JPS Matsue16 WAN Remote Host is ns.bnl.gov packet size is 100 from Fri Aug 20 0:19:10 Japan 1999 to Sun Aug 29 23:49:10 Japan 1999 There is a time tic every day Remote Host is cnsuty.cns.s.u-tokyo.ac.jp packet size is 100 from Fri Aug 20 0:19:09 Japan 1999 to Sun Aug 29 23:49:09 Japan 1999 There is a time tic every day

PHENIX CC-J JPS Matsue17 Key Software PBS: Batch Queuing System – –Free package developed mainly at NAS of NASA AFS: File system with Kerberos –Important files (source codes, libraries etc.) are on AFS at BNL. –Mirroring from BNL Monitoring: MRTG –CPU, memory, disk usage of each node as well as transmission rate via network – – html

PHENIX CC-J JPS Matsue18 PHENIX Software

PHENIX CC-J JPS Matsue19 Summary CC-J を構成する「部品」は一通りそろった。 各部品および全体としてのさまざまな性能をﾁｪｯｸしている。 おおむね所期の性能を出している。（予定通りの数が入れば要 求を満たす。） なお、細かい点でのバグ出し、性能の理解を進め、初期の要求 を満たす。