1. TE/MPE/MI OP section meeting 29 th September 2009 HCC 2009 Frequently Asked Questions 0v1 M. Zerlauth

2. Markus.zerlauth@cern.ch Machine Interlock Systems Outline  HCC campaign 2009 is progressing very well with >> number of tests performed / shift  Many changes and improvements of test plans, powering procedures, powering specifities, tools, … have/had to be done for this years campaign  LHC powering system + commissioning tools by now a rather complex construct  Presentation summarizes and answers/provides hints to some FAQ to further minimize number of failing tests and (avoidable) test repetition  Focus on operational procedure  Not included are implementation-issues: ie, errors in sequences, test- plan, LSA, etc…

3. Markus.zerlauth@cern.ch Machine Interlock Systems Conditions before starting a test  Next step not correctly proposed by sequencer (especially for PIC1 / PIC2 of mains)  Login as lhcop  SOC needs to be reserved for your console  Circuit must not be locked (ie due to previous failure)  Is previous test completed? Ie the analysis signed  Check correct short status in SOC editor and Simulation/normal mode of power converter

4. Markus.zerlauth@cern.ch Machine Interlock Systems Conditions for starting a test  Circuit not being ready for testing is most frequent reason for a ‘failed’ test  Multitude of conditions required, most frequent missing (>90%) are  Circuit still locked / super-locked -> PIC supervision  QPS controller not yet ready (ie a reset is needed) or 13kA switches are still open -> PIC supervision / Circuit synoptic  CRYO conditions (temporarily) lost -> PIC supervision  UPS conditions (temporarily) lost -> PIC supervision  What the sequencer is capable/allowed doing in 2009  Closing of 600A switches  Resetting of faults in PIC/PC

5. Markus.zerlauth@cern.ch Machine Interlock Systems Conditions for starting a test  To avoid unnecessary repetition (and follow-up) of failed tests, it is strongly recommended to verify the correct state of the circuit before launching ANY test, e.g. using Circuit Synoptic

6. Markus.zerlauth@cern.ch Machine Interlock Systems Conditions for starting a test  To avoid unnecessary repetition (and follow-up) of failed tests, it is strongly recommended to verify the correct state of the circuit before launching ANY test, e.g. using Circuit Synoptic

7. Markus.zerlauth@cern.ch Machine Interlock Systems Conditions for starting a test  To avoid unnecessary repetition (and follow-up) of failed tests, it is strongly recommended to verify the correct state of the circuit before launching ANY test, e.g. using PIC Supervision

8. Markus.zerlauth@cern.ch Machine Interlock Systems Conditions for starting a test  Be careful with Circuits that have already been frequently repeated (especially tests where heaters are fired, etc..)  ‘Dangerous tests’: PIC2 CIRCUIT QUENCH, PLI*.f*, PNO*.f*  Check in HCC pages history, now # of test repetition also in overall view

9. Markus.zerlauth@cern.ch Machine Interlock Systems  New version of HCC show now the current repetition number for the test currently under execution  By default, green as long as # rep <=2, red if repeated more often  New MPP strategy: 2 training quenches allowed, afterwards open NC and change I_PNO Conditions for starting a test

10. Markus.zerlauth@cern.ch Machine Interlock Systems Reasons for failures during sequence  In addition to quenches, trips, etc… the following things might make a sequence fail/time-out  Wrong login in sequencer, resulting in RBAC exception -> PIC + FGC is now under RBAC, only lhcop will work from CCC island  QPS controller not reset -> Error message like ‘QPS.PVSS ST_LOGGING not OK’ -> Reset QPS controller before test  Mismatch of parameters in LSA and power converter (out of limits exception) -> ask EPC expert to re-synchronize or CO to correct LSA  Exceeding I_ERR or I_EARTH limits: Considered as warning only (sequence will halt but not fail) -> Confirm with EPC expert and in case accepted skip step in sequence  CRYO conditions (only CRYO_MAINTAIN), UPS conditions (only HW signal) -> Can be checked in PIC supervision or PIC history buffer

11. Markus.zerlauth@cern.ch Machine Interlock Systems Reasons for failures during sequence  Asserting pop-ups too quickly (ie if action not yet completed) might result in timeouts at a later stage  Re-trip of agent after automatic reset (e.g. in RQ6.R7B1 ) -> Circuit synoptic  PM acquisition in agent blocks (RQ4.L8) -> Circuit synoptic

12. Markus.zerlauth@cern.ch Machine Interlock Systems Signing a test  Overall outcome of a HCC test is logical AND between successfully performed sequence + PMA comments  Thus if sequence failed for any reason, even a passed PMA will not alter the test outcome  For special cases (ie long sequences that failed with minor errors), outcome should be altered by CO expert BEFORE signing the test  Automated analysis  Automated path introduces additional 10 minute delay to make sure data is present  Currently only PIC2 for 120A and 600A signed automatically  Test should normally be signed 15 minutes after the end-time of test  If after this time test does not appear in PMEA nor in history -> Call LV- expert  If test is signed by experts (not visible in PMEA anymore) but not updated in HCC pages (ie next step proposed) -> Call LV-expert

13. Markus.zerlauth@cern.ch Machine Interlock Systems Unlocking of Circuits after Failing test  Test execution must be sequential, when UNLOCKING circuits in SOC editor after failure, make sure no PMA is pending  Test repetition before completion of analysis of previous test could be dangerous and breaks MTF upload (requires for sequential upload)

14. Markus.zerlauth@cern.ch Machine Interlock Systems Re-commissioning of circuit  Pointer defines commissioning status of a circuits = LAST_PASSED_TEST  If circuit should be re-commissioned (e.g. after HW change)  Request point owner to move back this pointed in the SOC editor to the required step (where to start re-commissioning)

15. Markus.zerlauth@cern.ch Machine Interlock Systems Cleaning-up  If sequences have been e.g. repeated by mistake (and step was already performed correctly)  Complete the test (if not long test, critical, etc..)  Abort test + immediately tell CO expert to delete test in LSA before signed and copied to MTF  Others…?  I’d be happy for your feedback of other reoccurring issues  Try to include new things + answers in these slide & publish on HCC pages and others?

16. Markus.zerlauth@cern.ch Machine Interlock Systems FIN