1. MAP Project T. Bowcock, A. Kinvig, I. Last M. McCubbin, A. Moreton C. Parkes, G. Patel University of Liverpool

2. Friday, 04 February 2000 University Of Liverpool Introduction  Monte Carlo Array Processor justification  Status Hardware Software  COMPASS  Summary

3. Friday, 04 February 2000 University Of Liverpool Monte Carlo  At LHCb about 1 interaction /25ns ! 4*10 14 /year if you want to do physics you need to know the backgrounds generating just the signals doesn’t work need to generate large MC samples O (10 7 ) to O (10 8 ) events. LHCb needs to do this now!

4. Friday, 04 February 2000 University Of Liverpool Philosophy  Cheapest possible that works No Gbit ethernet until price falls Don’t buy top of range processors No SMP boards No tapes obsolete? Develop architecture with future in mind]

5. Friday, 04 February 2000 University Of Liverpool MAP hardware  300 processors 400MHz PII 128 Mbytes memory 3 Gbytes disk D-Link 100BaseT ethernet +hubs commercial units BUT custom boxes for packing and cooling

6. Friday, 04 February 2000 University Of Liverpool MAP

7. Friday, 04 February 2000 University Of Liverpool MAP cont’d

8. Friday, 04 February 2000 University Of Liverpool MAP cont’d

9. Friday, 04 February 2000 University Of Liverpool MAP Architecture Master External Ethernet MAP Slaves Hub 100BaseT

10. Friday, 04 February 2000 University Of Liverpool MAP software  Overview Linux based on RedHat 5.2 stripped down version Batch System Network Control At the UDP level Robust Packet Handling Overloading of master ethernet interfaces (300 at once) implied need for total control of data flow Broadcast of control required phased reply

11. Friday, 04 February 2000 University Of Liverpool MAP user  Prepare a job  Submit to Batch Queue  Histograms/Ntuples transmitted back at end of job/DST’s  Random Numbers handled automatically

12. Friday, 04 February 2000 University Of Liverpool MAP Status  In production for about 6 weeks  300 Processors produced about 240,000 LHCb events 24/hrs 5 million events produced to date Also produced DELPHI DST’s (500,000 24Hrs)  All Processors tested Further Air-Conditioning installed fully commissioned 22/11/99

13. Friday, 04 February 2000 University Of Liverpool MAP Issues  Packet Loss At UDP (or frame level) have to handle with code. Now not a probem(!) Higher performance with shielded cables?  no  Power Infrastructure for cooling  Power up/down

14. Friday, 04 February 2000 University Of Liverpool Emergency Power Down  Unplanned power interruption Exploding substation! About 4% of PC’s need manual intervention

15. Friday, 04 February 2000 University Of Liverpool MAP capabilites  Can be used in “throwaway” mode Also write events as genenerated  MAP possesses 1Tbyte internal storage 3 Gbytes/machine events stored locally (1million events) repeatedly analyse QUICKLY  MAP can handle interprocess communication

16. Friday, 04 February 2000 University Of Liverpool MAP++

17. COMPASS Computerized Analysis and Storage Server

18. Friday, 04 February 2000 University Of Liverpool COMPASS  Purpose Will show this in place and working with MAP Model for LHC analysis store events on disks (cheap!) move JOB to the DATA NO HSM

19. Friday, 04 February 2000 University Of Liverpool Outline  Hardware  Linux Device Drivers  Linux Installation and Limits  Benchmarking Tests  Results  Future

20. Friday, 04 February 2000 University Of Liverpool Trial Hardware  Dell PowerEdge Server, 450 Mhz Pentium III, 256 Mb RAM with 4 internal SCSI disks.  4 PowerVault 1200 Disk Servers each with 8 Ultra Wide SCSI LVD disks.(spindle 7200 rpm) Total > 1Tb disk space  Adaptec Ultra Wide SCSI cards.

21. Friday, 04 February 2000 University Of Liverpool ITS  Purchased Rack mounted 1TByte based on 50GByte 7200 rpm disks Redundant Power Supplies 15KGBP/Tbyte including 2 500MHz PIII  More storage underway

22. Friday, 04 February 2000 University Of Liverpool Linux Devices Drivers  Linux Device Drivers: Devices accessed through special files in /dev directory specifying block or character device and major / minor number pairs. Major number refers to a device driver e.g. 8 is a SCSI disk (see /usr/src/linux/include/linux/major.h) For disks, minor number refers to disk / partition on disk e.g. /dev/sda major:8 minor:0 first SCSI disk found on system /dev/sda1 major:8 minor:1 first partition /dev/sda15 major:8 minor:15 last partition on first disk /dev/sdb major:8 minor:16 second SCSI disk found on system minor numbers are 8-bit i.e. only have values in range 0-255  only 16 disks per disk major number.

23. Friday, 04 February 2000 University Of Liverpool Linux Installation & Limits  RedHat Linux 5.2: Kernel 2.0.x Used at Liverpool and CERN – problem: only one SCSI major number is defined – maximum of 16 SCSI disks allowed. Kernel “hacking” necessary to register new SCSI major number with system.  RedHat Linux 6.0: Kernel 2.2.x Defines 8 SCSI major numbers : 8, 65-71- max. 128 SCSI disks. Have to create some special files in /dev by hand – relatively trivial with mknod  Physical limit of only 4 PCI slots for SCSI cards on motherboard

24. Friday, 04 February 2000 University Of Liverpool Benchmarking Tests  Use CERN sequential IO tests for read / write / calibration. Block sizes from 1024 Bytes to 0.5MBytes Calculates average write rate over previous 10 writes Read... Calibration: Comment out write statement and run write tests again.  Modified version of above calculates averages over the whole file.

25. Friday, 04 February 2000 University Of Liverpool Results  All disks accessible  Performance uniform writing about 20MBytess reading at 50 MBytes (or better) large block-sizes faster

26. Friday, 04 February 2000 University Of Liverpool Future  Can we find funding for large(r) scale prototype? Applications outside of Physics Interdisciplinary funding

27. Friday, 04 February 2000 University Of Liverpool Summary  MAP yields high performance at low cost  Storage can be cheap  R&D to Enhance performance  Production for LHCb vertex detector