Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Off-line Computing for the AIACE and HERMES Experiments Physics Motivation: – the Hermes experiment – the AIACE experiment The Offline Tree The Offline Computing Facility (LNF) Summary and Conclusions Federico Ronchetti INFN, GRUPPO III Laboratori Nazionali di Frascati

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 E.M. Nuclear Physics AIACE and HERMES: probing nuclear and sub- nuclear structure via (real or virtual) photon interaction. Advantages: very well known and flexible probe. Disadvantage: the electro-magnetic interaction is a relatively weak probe (i.e. low statistics). In recent years: –availability of high-current (100  A) high duty-cycle (100%) electron accelerators –EM processes can be studied with a statistical accuracy comparable to strong processes.

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Detector Complexity Older experiments: –simple detector electronics ( few 10 2 ch) –simple software development (VME/FORTRAN) and analysis (HBOOK/PAW/GEANT3) –low data rates (few 100 kB/s) Nowadays: –much larger detector complexity is (20÷40×10 4 ch) –more mature software development (UNIX/LINUX, CVS, C/C++) Java (Slow Control) and analysis tools (ROOT/GEANT4) –higher data rates ( few 10 MB/s) Adequate network and computing support: –ONLINE: most of the requirements are covered using commercial/industrial electronics; –OFFLINE: medium-low cost computing and open source software.

TOF DC CC TORUS The Hall B The CLAS detector The CLAS CLAS Highlights 6 GeV maximum energy fixed target e  N 25 MB/s ( L = cm -2 s -1 ) Accelerator current  torodial field (2.5 T m) electronic channles BEAM ECAL

The HERMES Detector HERMES Highlights: 27.5 GeV electron beam HERA Collider : e+  p L = 10  cm  s  Solenoidal magnetic field Maximum current I = 40  A electronic channels

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 CLAS Data Acquisition Detector READOUT – FASTBUS ADC/TDCs – VME/FASTBUS PowerPC controllers (ROC) – VxWorks real time OS – 100 Mbit/s Ethernet RunControl – Sun Ultra Sparc Cluster – Cross compilation of VxWorks readout code – ROC download Event Distribution and Building – Online Monitoring – Event Display – Disk Storage (RAID) – Tape Storage (SILO)

The Offline Tree JLAB - DATA COOKING RAW data files (~ 5 TB/Exp) Calibration Event Reconstruction – ADC/TDC  energy, time – 4-momentum determination Data Reduction – Skimmed files LNF - GEANT SIMULATION & DATA ANALYSIS RAW simulation files (~ 200 GB) – Event Generation – Post Processing – Reconstructions Skimmed data (~ 500 GB) – Analysis – N-tuple production

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Offline LNF Offline analysis: relatively high volume of data: 1 ~ TB (L=10 30 cm -2 s -1 ) Number of staff scientist/users involved in the offline analysis is of the order of 20 System administration/librarian responsibility belong to a single scientist / technologist not devoting 100% of his/her time to this particular task. Running an efficient offline computing facility requires: medium term investments planning decoupling of different issues, as: –network (optimize infrastructure) –storage: disk and tape (manageability, reliability) –computing power –terminal access (security, ease of maintenance)

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Network Infrastructure First issue: overcome limitations in the local network bandwidth. Just few years ago: offline analysis bandwidth limited to 10 Mbit/s. Upgrade of the building switch to deliver 100 Mbit/s speed to all user workstations and gigabit speeds to the analysis machines: –CISCO Catalyst 6000: 12 full duplex Gigabit ports This upgrade allowed us to deploy an efficient facility for “medium scale” offline computing.

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 NAS (Network Attached Storage): –Reliability: implements hardware RAID disk array with hot spares, directory snapshot and file system recovery capability. –Fast: the OS kernel only knows disk, memory and network I/O –Versatile: exports can be accessed via NFS or other common protocols (i.e. NETBEUI/AppleTalk) A Procom Netforce 1750 is operating in the nuclear physics building since 1 year: –1.5 TB with SCSI RAID 5 support (2.2 TB by the end of the year) –Single PIII CPU, 1 GB RAM –Custom OS based on Free BSD –100/1000 MBit/s multiple network interfaces –1 GB LTO Autoloader –Remote (http/telnet) management Storage

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Linux Cluster AIACE/HERMES: –Linux (RHL/SUSE) based offline software distributions –data analysis and simulation Core Linux analysis machines –6 dual 1U 2.4 GHz PIV Xeon (12 CPUs) rack-mounted nodes –1 GB RAM DDR EEC –dual Gigabit NIC –hot removable 2x32 GB SCSI local disks (system installation and file cache) –clustering software: MOSIX and/or DQS (installation underway) –mission-critical data and programs reside on the NAS

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Server Based Computing User access to clustered computing resources: –past years: X-terminals –nowadays: Windows PCs (large administrative burden and large fraction of task replication) A possible solution: Server Based Computing –2 dual 1U 2.4 GHz PIV Xeon (4 CPUs) rack-mounted nodes 1 GB RAM (analogous to the Linux analysis nodes) –Windows 2000 Advanced Server –Terminal Services (Remote Desktop Display) –Transparent file sharing via NAS.

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Thin Clients Linux based thin clients: display X, Windows, and local Web applications. Native X support FLASH resident ssh, RDP, ICA and telnet applications: Only the server requires full application installation Thin clients are diskless and can be managed remotely RDP clients can be easily installed used on pre-existing PCs in order to reconvert them into thin clients

CAT Putting All Together CAT.6000 SWITCH PROCOM 1750 (NAS) LINUX Master Node Farm nodes WIN2K AD. SERVER PC Thin Client WI-FI applications

Workshop sulle problematiche di calcolo e reti nell'INFN, Paestum, 12 Giugno 2003 Outlook and Conclusions The new generation of nuclear physics experiments at intermediate energies –Increase in detector complexity and statistical accuracy. –Increase in the number of final users working on the analysis Demand for: –more structured software development –medium sized computing facilities –clustered computing (Linux) –distributed applications (thin clients) A new approach and mentality needs to be developed by the larger groups operating in nuclear physics.