1. IN2P3 Status Report HTASC March 2003 Fabio HERNANDEZ et al. from CC-in2p3 François ETIENNE fabio@in2p3.fr etienne@in2p3.fr

2. HTASC - 14 March 2003 2 Outline  User community  Update on computing services  Update on storage services  Network status  Grid status

3. HTASC - 14 March 2003 3 0,5 1995 2 1996 4 1997 6 1999 34 Mb/s 2000 IN2P3 current context  18 labs  1 Computer Center  2500 users  40 experiments CC IN2P3 -CERN connection 155 Mb/s 2001 1 Gb/s 2003 CC IN2P3 -SLAC connection 30 Mb/s 2001 155 Mb/s 2002 600 Mb/s 2003

4. HTASC - 14 March 2003 4 RENATER current context  Deployed : oct. 2002  More grid than star-shape  Most links = 2.4 Gbps  still 2 main nodes : Paris, Lyon

5. HTASC - 14 March 2003 5 User community  Experiments: LHC (Atlas, CMS, Alice, LHCB), BaBar (SLAC), D0 (FNAL), PHENIX (Brookhaven), astrophysics (17 expts : EROS, SuperNovae, Auger, Virgo…) 2500 users from different countries TIER A BaBar 20% CPU power were consumed by non-French users in 2002  Starting to provide services to biologists at a local/regional level (4 teams and ~3% of cpu over the last 6 months, WP10 EDG, Heaven cluster)  User community steadily growing

6. HTASC - 14 March 2003 6 Experiments CPU (UI ~ 5 SI-95) Aleph300 000 Alice1 000 000 Ams3 000 000 Antares500 000 Archeops300 000 Atlas3 500 000 Auger3 000 000 Babar16 000 000 Clas600 000 Cmb25 000 Cms2 500 000 D015 000 000 Delphi30 000 Edelweiss100 000 Eros500 000 Euso25 000 Glast50 000 H1500 000 Hess500 000 Indra40 000 Experiments BIOLOGY1 000 000 (several teams) Lhcb3 500 000 NA48600 000 NA50200 000 Nemo500 000 Ngs-Opera10 000 Phenix400 000 Planck-S.5 000 Siren8 000 000 Snovae300 000 Star5 000 Tesla100 000 Thémis200 000 Virgo400 000 WA9850 000 Total experiments above : 35-40 CPU (UI) : ~ 60 000 000 hours (~ 300 Mh SI-95) Experiments CPU request

7. HTASC - 14 March 2003 7 Computing Services  Supported platforms: Linux, SunOS, AIX  Dropped support for HP-UX  Currently migrating to RedHat Linux 7.2 and SunOS 5.8 Waiting for remaining users and EDG to drop support for RH6.2  More CPU power added over the last six months : 72 bi-processor Intel Pentium 1.4 GHz, 2 GB RAM, 120 GB disk (november) 192 bi-processor Intel Pentium 2.4 GHz, 2 GB RAM (february)  Today, the computing capacity (batch+interactive) is Linux: 920 CPUs SunOS: 62 CPUs AIX: 70 CPUsTotal > 1 000 CPUs  Worker nodes storage capacity used for temporary data (reset after job execution)

8. HTASC - 14 March 2003 8 Storage Services  Extensive use of AFS for user and group files  HPSS and staging system for physics data  Mix of several platforms/protocols SunOS, AIX, Tru64 SCSI, FibreChannel AFS, NFS, RFIO  Shared disk capacity (IBM, Hitachi, Sun) ~50TB  AFS User Home directories Code, programs and some experimental data  Xtage Temporary disk system for data on tape

9. HTASC - 14 March 2003 9 Storage Services (cont.)  Mass storage (HPSS): 250 TB now, 500 TB expected in dec 03 Installed capacity on tape: 700 TB Up to 8.8 TB/day Originally purchased for Babar but now used by most experiments Babar Objectivity: 130 TB and 25 TB cache disk, others: 120 TB and 4.4TB STK 9840 (20GB tapes, fast mount) and STK 9940 (200GB tapes, slower mount, higher I/O) Accessed by RFIO, mainly rfcp. Supports files larger than 2GB Direct HPSS access from network through BBFTP

10. HTASC - 14 March 2003 10 Storage Services (cont.)  Semi-permanent storage Suited for small files(which deteriorate HPSS performances) Access with NFS or RFIO API Back-up possible for experiments whose CC-IN2P3 is the « base-site » (Auger, Antares) Working on RFIO transparent access  Back-up, Archive: TSM (Tivoli Storage Manager)  For Home directories, critical experimental data, HPSS metadata, Oracle data  TSM allows data archival (Elliot).  For back up of external data (eg. From Admin. Data of IN2P3, from Biology labs, etc)

11. HTASC - 14 March 2003 11 Disks AFS : 4 TB HPSS : 4,4 TB Objectivity : 25 TB Oracle :0.4 TB Xstage :1,2 TB Semi-perm. : 1,9 TB TSM : 0.3 TB Local : 10 TB Tapes 1 robot STK – 6 silos, 36 000 slots 12 drives 9940B 200 GB/tape (7 hpss, 3 TSM, 2 others) 35 drives 9840 20 GB (28 hpss, 4 TSM, 3 others) 8 drives IBM-34900,8 GB (service will stop by end 2003) 1 Robot DLT – 400 Slots 6 DLT 4000 4 DLT 7000 Storage Service (cont)

12. HTASC - 14 March 2003 12 Network  International connectivity through… RENATER+GEANT to the US (600 Mbps via ESNET and ABILENE in NY) and Europe CERN to the US as alternate (600 Mbps)  Babar is using both links to the US for transferring data between SLAC and Lyon Specific software developed for "filling the pipe" (bbFTP) being extensively used by Babar and D0, amongst others  Dedicated 1 Gb link between Lyon and CERN since january 2003  LAN is composed of a mixture of FastEthernet and GigabitEthernet Ubiquitous wireless service  Connectivity to the other IN2P3 laboratories across the country by RENATER-3 (the French academic and research network, 2.4 Gbps links) All labs have a private connection to RENATER POPs

13. HTASC - 14 March 2003 13 Grid-related activities  Fully involved in the DataGRID project & partly in DataTag (INRIA)  One of the 5 major test bed sites  Currently all the "conventional" production environment is accessible through the grid interface Jobs submitted to the grid are managed by BQS, the home-grown batch management system Grid jobs can use the same pool of resources than normal jobs (~1000 CPUs) Access to mass storage (HPSS) from remote sites enabled through bbFTP  Benefits: Tests of DataGRID software in a production environment Scalability tests can be performed Users access exactly the same working environment and data whatever the interface they choose to access our facility Operational issues detected early

14. HTASC - 14 March 2003 14 Grid-related activities (cont.)  Disadvantages Local resources needed for integration of the production environment (AFS, BQS, …). More work needed to achieve a seamless integration between the local and grid worlds Users want us to provide a grid service: how to provide a service around a "moving target" software project?  Some experiments already using the grid interface for "semi- production" Other expressed interest in using it as soon as it gets more stable  Starting from march 2003, the resource broker and associated services for Applications and Development DataGRID testbeds will be hosted and operated in Lyon

15. HTASC - 14 March 2003 15 Grid-related activities (cont.)  Involved in several other grid projects at regional and national levels  Cooperation agreement signed with IBM to work on grid technology Exchange of experiences Grid technology evaluation Perform experiments of this technology in a production environment Explore technologies for virtualization of storage …

16. HTASC - 14 March 2003 16 DataGRID @ CNRS –IN2P3  Coordination of: WP6 Integration Testbed WP7 Networking WP10 Bioinformatics IPSLEarth Observation (Paris) BBEBioinformatics (Lyon) CREATISImaging and signal processing (Lyon) RESAMHigh Speed networking (Lyon) LIPParallel computing (Lyon) IBCPBioinformatics (Lyon) URECNetworking (Paris –Grenoble) LIMOSBioinformatics (Clermont Ferrant) LBPBioinformatics (Clermont Ferrant) LPCIN2P3 (Clermont-Ferrant) LALIN2P3 (Paris) SubatechIN2P3 (Nantes) LLR-XIN2P3 (Paris) ISNIn2P3 (Grenoble) CC-In2P3IN2P3 (Lyon) LPNHEIN2P3 (Paris) CPPMIN2P3 (Marseille) LAPPIN2P3 (Annecy)