ATCA COOLING PROJECT INTERSHIP FINAL PRESENTATION Piotr Koziol

Project description 13/01/2016PIOTR KOZIOL 2  ATCA boards will dissipate a significantly larger power (4x) with respect to the actual equipment. The cooling capabilities of the LHC rack need to be checked.  Identify potential in-rack bottlenecks and airflow resistance sources. Propose possible alternatives/fixes to remove them and test these adaptations. If required and possible run complementary simulations to confirm solution improvement.  Mechanical improvements of the rack or the crates to optimize the cooling performance (i.e.: removal of turbine chassis, modification of crate’s fan system, etc…) For historical reasons the electronics equipment in the so called LHC racks are cooled by a closed vertical recirculating airflow system from bottom to top  The AdvancedTCA (ATCA) telecom industry standard has been selected as the hardware platform for the “Phase II upgrade” of the back-end electronics systems for ATLAS

My tasks in this project Optimization of the program to manage larger number of data Graphic User Interface (GUI) improvement LabVIEW Sensors Installation: Temperature and Velocity ATCA crates Communication with standard PC LHC standard Rack 52U setup Vertical Air Cooling (Different loads and configurations) Horizontal Air Cooling Comparison Vertical vs Horizontal air cooling Cooling Tests Data analysis using Diadem and Excel Data Analysis 13/01/2016PIOTR KOZIOL 3 3 Rack 52U 14 U 15 U Temperature sensors Velocity sensors

Load blades layout and connection schema 13/01/2016PIOTR KOZIOL 4 Connection of the devices architecture Number of sensors: Old Load Blades 14x7 temperature sensor. New Load Blades 14x6 temperature sensors. Agilent data acquisition device which is connected to 30 PT100 sensors. 8xUAS1200PC velocity sensors connected with USB IN Out

Communication With the horizontal cooling crates (16U) it has been required to setup a new communication system It has been prepared using RS232 port and terminal program. In this connection we need to login to the U-boot multi-platform where I was needing to setup IP using clia commands. Then I connected it directly by Ethernet to shelf manager. LabVIEW communicate with chassis using IPMItool external program by IPMI commands. New blades are not fully debugged and they are not working with IPMI commends. Company provide to us very simple program made in JAVA and it is working only with ShMM-500R. We are still waiting for the new firmware from company to new load blades. 13/01/2016 PIOTR KOZIOL 5 Shelf manager with ShMM-500R and ShMM-700R Used for communication with load blades

Graphical user interface 13/01/2016PIOTR KOZIOL 6 Turn on/off fans Turn on/off Load Menu for Velocity/Temperature single Measurements Turn on/off program Air Temperature with rack layout

Vertical cooling test campaign ATCA 1 full load and ATCA 2 only fans ATCA 1 only fans and ATCA 2 full load ATCA 1&2 7+7 ATCA 1&2 full load (to be done) Comparison with simulations carried out by the company 4 heat exchangers & 3 heat exchangers Bottom fans influence Enhanced HX surface exchange Verification of air distribution adding gaps below and above each heat exchanger Vertical configuration 13/01/2016PIOTR KOZIOL 7 TURBINE ATCA2 GAP Heat Exchanger 2 GAP ATCA1 GAP Heat Exchanger 1 GAP Heat Exchanger 3 Air Deflector

Horizontal cooling test campaign 13/01/2016PIOTR KOZIOL 8 ATCA 1 Full load ATCA 2 off ATCA 2 Full load ATCA 1 off ATCA 1&2 7+7 ATCA 1&2 full load (to be done) Bottom fans influence Horizontal configuration ATCA2 GAP ATCA1 Cooling Door GAPGAP GAP Side view of the rack. Air is going like arrows in this picture.

13/01/2016PIOTR KOZIOL 9 Measurements for Horizontal & Vertical tests Air temperature in each slot Crate air temperature and velocity inlet outlet Cooling door water (Horizontal) or cooling water going to heat exchangers, inlet and outlet (Vertical) Water flow measurements Rear door air temperature (outlet), only Horizontal Heat exchangers air temperature outlet, only Vertical Velocity speed of the turbine, only Vertical Taken Measurements Cooling door Water Temp sensor Horizontal crate with velocity sensors (inlet)

Vertical performance 13/01/2016PIOTR KOZIOL 10 W = with WOUT – without G = gaps BF = bottom fans FSHE = full surface of heat exchangers Significant lower Temperatures removing the bottom fans trays -> improved flow distribution Limit: stay below 50 o C!!!!

Measurements of air velocity for vertical cooling 13/01/2016PIOTR KOZIOL 11 HEAT EXCHANGER 2 HEAT EXCHANGER 3 HEAT EXCHANGER 1 AIR DEFLECTOR s04s04 S06S06 s08s08 s10s10 s12s12 s14s14 s09s09 s07s07 s05s05 s03s03 s01s01 s02s02 s13s13 s11s11 s04s04 S06S06 s08s08 s10s10 s12s12 s14s14 s09s09 s07s07 s05s05 s03s03 s01s01 s02s02 s13s13 s11s11 TURBINE v7v7 v1v1 v5v5 TOP FANS BOTTOM FANS TOP FANS BOTTOM FANS v4v4 v7v7 v6v6 v6v6 v6v6 v6v6 v4v4 v4v4 v3v3 v3v3 v7v7 v7v7 v2v2 v4v4 v3v3 Velocity sensors are quite sensitive to Temperature changes

Horizontal measurements crates size comparison 13/01/2016PIOTR KOZIOL 12 16U crate more effective: PH-ESE bought one to be used for future development and tests Limit: stay below 50 o C!!!!

Measurements of air velocity for horizontal cooling 13/01/2016PIOTR KOZIOL 13

Vertical vs. Horizontal 13/01/2016PIOTR KOZIOL 14 Limit: stay below 50 o C!!!!

Conclusions 13/01/2016 PIOTR KOZIOL 15 Increase the number of gaps between chassis or using full surface of the heat exchanger does not change anything in the results for vertical cooling. Without bottom fans trays both vertical and horizontal crates provides similar performance 16U crate is more effective than 14U crate for horizontal cooling. 16U crate for horizontal cooling and crate for vertical cooling without bottom fans have similar effectiveness of cooling

Thanks for your attention 13/01/2016PIOTR KOZIOL 16