1 Machine Checkout and Setup Periods Jan Uythoven Thanks to: O.Aberle, R.Bailey, F.Bordry, O.Brunner, L.Bottura, E.Carlier, P.Charrue, E.Ciapala, S.Claudet, R.Denz, D.Forkel- Wirth, B.Goddard, M.Jimenez, R.Jones, M. Lamont, V. Mertens, P.Ninin, B.Puccio, S.Ramberger, F.Rodriguez Mateos, R.Schmidt, L.Serio, B.Todd, M.Zerlauth

Jan Uythoven, 2005, Machine Checkout Page 2 Machine Checkout: WHEN ?  Year 1: Following Hardware Commissioning  Assume all equipment tests are done in the past 18 months  Most of the system tests are done in the past 18 months  Subsequent years: Following Shutdown Work  Assume most equipment tests are done  Machine Checkout  First time machine complete cold and closed  All systems running together from the control room  Before First Beam

Jan Uythoven, 2005, Machine Checkout Page 3 Tasks - Summary  Redo a subset of tests performed during the Hardware Commissioning period  Because some tests more than a year ago  Because the final configuration of the system might be different, especially the interface to other equipment  If hardware commissioning period was squeezed in time, this will have been at the cost of the system tests  As a result: will need more time for the Machine Checkout  New Tests  Additional system tests  All systems together – machine wide, interdepedencies  Machine completely closed, last access / final checks  Under the responsibility of the Operations Group  Green light from DSO, info for INB

Jan Uythoven, 2005, Machine Checkout Page 4 First beam Phases of Machine Checkout Hardware Commissioning Sectors 1-2 and 2-3 Cooldown of all sectors ELQA tests Close machine completely; Contr. Access Complete Machine Cycle from CCC DSO tests Time Equipment tests per 2 sectors: Interlocks, Power Converters RF, kickers, BDI,.. Machine Checkout phase I Machine Checkout phase II Machine Checkout phase III Some phases in parallel

Jan Uythoven, 2005, Machine Checkout Page 5 Phase I  Infrastructure for the complete machine  Water, Ventilation, Power, Controls Network  Radiation Protection (RAMSES)  Check response ionisation chambers  Interface with other systems  UPS, AUG  Cooldown of the complete machine  Sectors T up to 200 K: need 10 days for cooldown  If T > 80 K: need to redo Electrical Quality Assurance tests (ELQA), first time  3 weeks  If T < 100 K: cooldown needs about 5 days Up to about 5 weeks for sectors warmed up > 80 K. Need to start cooldown in parallel with Hardware Commissioning. Manpower. When full inventory of He available?

Jan Uythoven, 2005, Machine Checkout Page 6 Phase I, continued Access System  Close the complete machine  Test of LHC Access Control System (LACS)  Test of LHC Access Safety System (LASS)  New tasks – before only done per sector (during HC)  New tasks: interface adjacent sectors, machine wide  Important Safety Element in point X must interface over the global machine and beam dump in point 6  Interface with Radiation Protection equipment: access veto  Interface with HR data base, biometrical data  Some equipment was not in final state during Hardware Commissioning tests  Exercise procedures for ‘patrouille’, giving access, back from access, …  Start first with sectors needed for Machine Checkout equipment tests phase II  If necessary, time to sort out problems parallel to phase II programme Ideally only controlled access from phase I onwards

Jan Uythoven, 2005, Machine Checkout Page 7 Phase II  Per two sectors  Check interlocks: PIC, WIC, QPS  See  Test all power converters, two sectors at a time  See  Simultaneously in other sectors  Other equipment checks, see Determines the pace of tests in Phase II

Jan Uythoven, 2005, Machine Checkout Page 8 Phase II Interlock Systems  Before power converters can be tested, test interlock system  Quench Protection System (QPS)  Visual check of equipment, half day per sector  Automatic test procedure, 8 hours (same as monthly procedure): fires the quench heaters without current in magnets  Test interface to BIS -> check in phase III  Powering Interlock Controller (PIC)  Takes information from the QPS  Automated test procedures and visual control  Needs about 8 hours for testing  Warm Interlock Controller (WIC)  No automated procedures foreseen  Some manual testing and visual control: 8 hours

Jan Uythoven, 2005, Machine Checkout Page 9 Phase II Power Converters  Followed by testing of Power Converters two sectors at a time  Run all converters to full power, perform tests at full power  Calibration already done during Hardware Commissioning  12 h continuous running at full power  Need about half a week per two sectors  Proceed around the ring

Jan Uythoven, 2005, Machine Checkout Page 10 Phase II Control System  Control System Communication Mechanisms  Field buses operational machine wide. All elements to be checked: need about 8 days  Already start during phase I  FrontEnds, BackEnds and general services  Should have been made available during the Hardware Commissioning period sector by sector and left running  Timing generation and distribution  6- 8 days for the first year  Databases access  1 day, at least

Jan Uythoven, 2005, Machine Checkout Page 11 Phase II Control System, ctd.  Application software  Tested continuously during the last months / years  Specific projects need testing  Interlocks, realtime feedback, logging, postmortem, safe beam flag etc.  Estimate 10 days for test …..???  Final tests when complete machine available: See phase III Message from many (CO, OP and BDI): DRY RUNS, dry runs & DRY RUNS Quoting TI 8 experience

Jan Uythoven, 2005, Machine Checkout Page 12 Other Equipment tests: RF  Basic work of testing cavities, cryo, LLRF and control during hardware commissioning period  Number of tests which would clearly accelerate RF operations and improve performance: - Reliability of RF system: power & cryogenics - Software and diagnostics developments / improvements - New conditioning hardware for subsequent years - RF noise and ripple measurements - Customisation of klystrons (positioning of resonance) - Linearisation of klystron response - Effect of detuning and coupler position on loop stability - Optimisation of loops for higher intensity - Fine adjustment of trip level for fast transients - Setting-up of the longitudinal damper - Preparation of feed-forward and 1-T feedback Ed’s Wish List RF will use the time made available

Jan Uythoven, 2005, Machine Checkout Page 13 Other Equipment tests: Warm Magnets  Visual inspection  No spanners lying around…  All protection covers in place  Cooling system o.k.  All valves open  No visible leaks  Expect 1 day work per sector, 8 days total

Jan Uythoven, 2005, Machine Checkout Page 14 Other Equipment Tests: Vacuum  Assume machine already cold & closed  Check interlocks  Sector valves  Connection to BIC  Not to forget  Vacuum interlocks on transfer lines: 2 days 3 days per octant 2 teams in parallel 12 days

Jan Uythoven, 2005, Machine Checkout Page 15 Other Equipment tests: Collimators and Beam Absorbers  Redo some of the hardware commissioning tests in local & remote  Motor control, position switches  Temperature sensors, cooling system  Interfaces with more intelligent software from the CCC to manage settings  Not defined yet, should include BIC  Couple of days  In parallel with other tests

Jan Uythoven, 2005, Machine Checkout Page 16 Other Equipment tests: Pulsed Magnets  Beam dump kickers (MKD and MKB point 6)  Injection kickers (MKI points 2 and 8)  Tune and Aperture kickers (MKQA point 4)  For all of them  Verification of local interlocks  Pulse at full power – MKI some re-conditioning time  Interface with many systems across the machine  RF point 4 (rev. freq.)  Energy from dipoles either side point 6  Timing, controls, machine protection, post mortem  OASIS, logging  Cross connections between MKI and MKD Couple of days per system Couple of WEEKS Maximum in HC, start early, profit from RELIABILITY RUN

Jan Uythoven, 2005, Machine Checkout Page 17 Other Equipment tests: BDI  Large distributed systems: BPMs & BLMs  Hardware fully checked during HC, but re-check  Main tasks: software and logging, links to other users  Interlocked Systems – check interface BIC (phase III)  BLM, BCT point 4, BPMs point 6, LDM (Longitudinal Density Monitor)  Diagnostics systems – no special requirements  Q and Q’ measurements, Schottky systems, Sychrotron radiation monitors, OTR screens, Wire scanners Experience has shown that BDI ALWAYS wants to have another access CONTROL SYSTEM DRY RUNS, distributed over about 2 months, so start BEFORE the machine checkout

Jan Uythoven, 2005, Machine Checkout Page 18 Phase III  Status after Phase I & II  All equipment and systems (re-)tested, sector by sector  Individual interlocks tested  Controlled access  Phase III  Machine wide tests from the CCC  Go through complete machine cycles without beam  Accent will be on control system and application software  Interleaved with final tests of access system and green light by DSO  In principle no access should be necessary any more  Control System Tests: Build on experience !  To be tested throughout the yearS with the maximum number of ‘dry runs’  TI 8 tests with beam  Hardware Commissioning of all sectors including system tests  Sector test with beam

Jan Uythoven, 2005, Machine Checkout Page 19 Phase III Complete Machine Cycle Without Beam  Initial machine cycle (45 minutes) to get to initial field conditions  No further tests possible – real magnetic field to be found out with beam  Generate functions for the power converters  Download all functions to power converters – synchronous ramping of all power converters including energy ramping of beam dumping system  Settings management, trimming of functions  Machine wide timing of the systems, pre-pulses etc.  Timing (slow and fast), OASIS to set timing of kickers  Interface with RF, injection kickers, beam dump kickers, machine protection system  Applications – communication with all equipment  All fixed displays, video, trajectory and orbit correction  Feedback applications: trajectory, tune, chromaticity, transverse, longitudinal  Logging of all parameters during the cycle  Vacuum displays, alarm screens, radiation monitoring  Interface with the Beam Interlock System  Safe beam flag depending on energy  Post mortem after beam dump  Some kind of sequencer to automate tasks (LHCexec)

Jan Uythoven, 2005, Machine Checkout Page 20 Phase III, continued  Now obligatory to have machine completely closed  If search broken during phase II: Complete patrouille with several teams: 2 days  Re-test complete access system before DSO test: 2 x 12 hours  Machine protection system has probably been touched upon during phase II  Followed by Beam Interlock System (BIS) tests (or BIC)  Need “Green Light” from all systems – towards the end of test period  16 different users, 150 ‘user boxes’  Every box needs to receive an interlock, provoked by the equipment connected to it. Check 4 x dump request per equipment.  Estimate 3 x 8 hours to test – 2 days  Rubber stamping of systems by DSO – data for INB  Need about 1 day, if everything goes fine  Rubber stamping by ‘Mr. Interlock’  Need formal test procedures and acceptance criteria

Jan Uythoven, 2005, Machine Checkout Page 21 Time Required  Phase I: 1 week minimum  Cooldown and closing machine  Phase II: 3 weeks minimum  Interlocks, equipment & system tests  Phase III: 2 weeks minimum  Complete cycles without beam  Final test access system and BIS  DSO tests Minimum of 6 weeks on planning! Partially parallel with hardware commissioning Equipment setup with beam, see

Jan Uythoven, 2005, Machine Checkout Page 22 Equipment Tests With Beam  BDI  Calibration BPMS, BCTs  Set-up of tune and chromaticity measurements  RF  Low level control (typically 3 weeks)  Phasing  Feed back  Control system  Logging beam data, post mortem  Steering – feedback  Machine protection tests  Check interlocks of the system  Safe beam flag  Beam Dumping System  Beam dump to be set up at the different energies  Verification of post mortem, logging and analysis  Cryogenics with significant beam load Continuously during the first weeks / months of operation

Jan Uythoven, 2005, Machine Checkout Page 23 Not to underestimate: Transfer Lines  TI 8 will need recommissioning  Tested before  With beam up to TI 8 TED end 2004,  Not tested yet: last 100 m + collimators  Sector test end 2006  But will need to be recommissioned before first beam  TI 2: never fully commissioned before  Will probably only be ready for commissioning at about the same time as the last LHC sector (transport of all LHC magnets through the TI 2 tunnel)  Could be done in parallel with hardware commissioning of the last sector: preferred solution but impact on HC schedule  Other option is to do this in parallel with ring 2 beam commissioning: higher risk in case of problems

Jan Uythoven, 2005, Machine Checkout Page 24 Machine Checkout For Subsequent Years  No Hardware Commissioning period but shutdown work to be finished  Outline stays the same with similar tests  Some tests should be faster, because systems are better known  Foresee about 5 weeks minimum: longer than SPS / LEP because systems are more complex, machine protection to be taken more seriously Compare LEP and SPS: About 2 weeks of pre-closure and 2 weeks of real machine checkout when machine is for OP 2 weeks of LEP / SPS ‘cold checkout’ similar to 2 weeks of phase III of what is presented for the LHC

Jan Uythoven, 2005, Machine Checkout Page 25 Conclusions  After the hardware commissioning period the machine is not ready for beam  Confirmed by experience from LEP & SPS, need a period to get ready for beam under the responsibility of OP  Even with all equipment groups 100 % ready on time (!): complete systems tests remain and their interdependencies need to be verified  At least 6 weeks of tests need to figure on the LHC planning for Machine Checkout  Start earlier for cooling down the sectors warmed up > 80 K for ELQA tests, in parallel with hardware commissioning  Start earlier for software and machine protection: DRY RUNS to be planned and scheduled  Many tasks in parallel: need careful planning and good coordination  Some running in of equipment with beam is also required  BDI being the obvious candidate, but also RF, beam dumping system  Don’t forget the transfer lines  Same applies every year after a shutdown