1. LECC 2006 26.09.2006V. Bobillier1 Overview of LHCb Electronics installation aspects V. Bobillier; L. Roy; J. Christiansen CERN - LHCb

2. LECC 2006 26.09.2006V. Bobillier2 LHCb experimental cavern Racks and rack control and monitoring system Detector safety system and its implementation in LHCb Mains power distribution for the experiment Low voltage power supplies choices Cabling constraints and installation Optical fiber cables Global grounding in cavern Status of installation

3. LECC 2006 26.09.2006V. Bobillier3 The experimental cavern LHCb Counting house 3 levels 150 racks Detector Protective shielding wall

4. LECC 2006 26.09.2006V. Bobillier4 The experimental cavern LHCb Status in June 06

5. LECC 2006 26.09.2006V. Bobillier5 Front-end electronics racks (~50): standard type for crates and electronics boards (vertical recirculating air flow) - Air circulated by turbine units - Heat exchanger cool air - Fan trays distribute air homogeneously PC farm racks (~80): Special custom made cooling rear door (horizontal open air flow) - Fresh air taken externally from front of rack - PC cooled down by air - Air is cooled down before being extracted in rear of rack Rack types Front view of front-end electronics racks Side view of PC racks

6. LECC 2006 26.09.2006V. Bobillier6 In every Front-end electronics racks. Monitors: Temperature Humidity Smoke detection (DSS only) Acts on (via ECS or DSS in emergency cases): Mains electrical power distribution for a group of racks. Gives alarms (possibility to act remotely to trigger fire extinguisher systems). PC racks: Simplified system based on same principle is implemented (temperature monitoring made via PCs; Fans monitored via ELMBs (ECS); smoke detector and fire extinguishing system for entire counting house). Rack control and monitoring system

7. LECC 2006 26.09.2006V. Bobillier7 Protects equipment from being destroyed when conditions are out of standards. Independent from other systems, redundant, has its own UPS and is backup supplied by Diesel generator. System has been made in collaboration with IT/CO. Decision matrix in the central PLC takes appropriate actions. Linked to ECS (Experiment Control System) and CSS (CERN Safety System). System consists of simple temperature, humidity, water leak sensors and outputs (for actions). Principal actions are: - Cut mains power - Close cooling water valves - Turn HV off - Trigger fire extinguishers (not automated)  Water mist for CPU farm  CO2 for F-E racks DSS (detector safety system)

8. LECC 2006 26.09.2006V. Bobillier8 Mains electrical power Electrical distribution principle for the experiment: 2 sets of 1.25 MVA transformers and distribution panels: Separation between computing farm and front-end electronics. Distribution to racks via 2 levels of distribution panels Local PLC system for supervision (monitoring and basic control on/off). Main components of DAQ (main switch; control PCs; readout supervisor) backup supplied by UPS (160 kW). x 2 Supervision system

9. LECC 2006 26.09.2006V. Bobillier9 Mains electrical power Problems foreseen: Harmonics produced by PC equipments (poor PFC)  Solution: split distribution in 2 (2 TFOs) Important inrush currents (no soft start) mainly in PCs  Solution: temporization relays in distribution boxes in addition to adapted circuit breaker curves (not sufficient). Measurements performed by CERN electrical support group - Measurements of the rack mounted PCs for the LHC experiments https ://edms.cern.ch/file/503582/1/second_meas_report_rackmounted_PC_final1.pdf https ://edms.cern.ch/file/503582/1/second_meas_report_rackmounted_PC_final1.pdf - Measurements of the electronics crates for LHC experiments https://edms.cern.ch/file/442186/1/Crates_LHC_experiments_rev2.pdf https://edms.cern.ch/file/442186/1/Crates_LHC_experiments_rev2.pdf PC rack THD measurements Crate LVPS THD (total harmonic dose) measurements 300 A PC rack inrush current

10. LECC 2006 26.09.2006V. Bobillier10 Circuit breakers and adjustable temporization relay Position of the 19” electrical distribution box in the electronics racks Mains electrical power In rack distribution box Secondary distribution panels

11. LECC 2006 26.09.2006V. Bobillier11 LVPS in the cavern: From radiation simulations (factor 2 included), worst location : Radiation total dose over 10 years: 5.8*10 3 Rad 1MeV neutrons/cm2 over 10 years: 1.9*10 12 Above 20MeV hadrons/cm2 over 10 years: 5.5*10 10 Magnetic field always below 200 Gauss. Low voltage power supplies Qualification : - Radiation total dose: 14*10 3 Rad - 1MeV neutrons/cm2: 1*10 12 - Above 20MeV hadrons/cm2: 2*10 11

12. LECC 2006 26.09.2006V. Bobillier12 System (~600 channels in total): Advantages: AC/DC converter and monitoring/control of power supply protected from radiation and magnetic field by the shielding wall. Smaller length of large diameter low voltage power cables (about 20 meters). Less dissipated power in cables (~36%). Better voltage stability. All F-E have local linear regulators Total cross section of the cables going from the AC/DC converter to the power supply (DC/DC) reduced by ~ factor 10. Disadvantages: Limited accessibility to the power supplies (DC/DC part). Question of reliability… Basic on/off control and voltage and current monitoring functions per channel. Power supplies (DC/DC) converter must stand radiation. Low voltage power supplies 380 Vdc

13. LECC 2006 26.09.2006V. Bobillier13 Cables have to run from sub-detectors (patch panels and on-detector front-end electronics racks) to counting house racks. Counting house Sub-detectors Cabling

14. LECC 2006 26.09.2006V. Bobillier14 Cabling Cable routes

15. LECC 2006 26.09.2006V. Bobillier15 Cable trays integration In front of the counting house cables must be correctly distributed to enter in the right rack column. Philippe Preau Cabling

16. LECC 2006 26.09.2006V. Bobillier16 Chicane to go through shielding wall Cabling Top view Side view Reduced space for cables in chicane. Only 350 * 2000 mm per side.

17. LECC 2006 26.09.2006V. Bobillier17 Optical fiber links In LHCb more than 7000 optical fibers for readout Total bandwidth ~11,2 Tbit/s Assuming BER at 10 -13 errors/bit results in ~1 error/s  Extensive tests needed. Particular care must be taken during and after installation. BER measurements performed with 6dB additional attenuation in optical links. Total attenuation loss measurements done after installation. Multi-ribbon optical fiber trunk cable (96 fibers, 8 ribbons of 12 fibers, MTP/MPO connectors)

18. LECC 2006 26.09.2006V. Bobillier18 Optical fiber links Partial view of a sub-detector requirements for links between cavern and CH ~ 300 units

19. LECC 2006 26.09.2006V. Bobillier19 Global grounding principle in cavern: In addition of safety earth connections, inter-connections are made to all metallic structures (supporting rails, platforms, etc.) 120mm 2 ground conductor running along every 4 th cable tray. Goal: Force ground loop currents circulating in low impedance grounding network. Problem encountered: Connections with iron reinforcement in concrete…? Grounding

20. LECC 2006 26.09.2006V. Bobillier20 Metallic structure in false floor of barrack is connected in many points to cooling pipes, cable trays and earth cables running under racks. Every CH level is connected to earth in the cavern. All racks are connected via a connecting braid to this mesh. In addition: - All copper cables (LV, HV and ctrl) are shielded. - Cables are installed in metallic closed (covered) cable ducts. Grounding of racks in counting house: Grounding Grounding of the racks.

21. LECC 2006 26.09.2006V. Bobillier21 Rack control and monitoring:All racks ready. System is installed everywhere. Commissioning will be done soon. DSS:DSS is running (general purpose sensors connected, interlocks to mains power network and water valves installed and tested). Sensors for sub-detectors defined to be installed. Mains electrical power: Power available in all racks. PLC supervision system still to be commissioned. LVPS:Waiting for final deliveries. Cabling:Copper cables: 70% pulled. Connector installation will start in coming weeks. Optical fiber cable pulling will start in November. Grounding:In counting house (racks): done. In cavern: still some interconnections to be made. Status

22. LECC 2006 26.09.2006V. Bobillier22 Questions / Comments