1. 1 15 April 2010: Post Mortem Analysis by M.Zerlauth Automatic POWERING EVENT Analysis Adam, Arek, Gert-Jan, Ivan, Knud, Hubert, Markus, Nikolai, Nikolay, Odd, Reiner, Roman, Zinur and many others…

2. 2 15 April 2010: Post Mortem Analysis by M.Zerlauth Preamble  1 st version of POWERING SERVERS ready for release early next week – Support work of MP3 through continuous online analysis of powering events (with and without beam) – Data driven mechanism, ie arriving PM RAW DATA FILES will be grouped into powering EVENTS, and are analyzed in SESSIONS – Analysis session finishes typically some 7-8 minutes AFTER circuit trip and might (depending on the analysis outcome) SUPERLOCK circuits in case of equipment malfunctioning – Analysis module included in initial release Single Circuit Event (SCE) module for dissection of event into individual circuit events 600A Energy Extraction 13kA Energy Extraction Quench Heater Discharge Analysis (QHDA) 600A Quench discard module FGC_ext analysis (for possible feedback issues) Automatic Powering Event Recognition (Summer module for final decision on event type, identification of fault source, FGC faults, MP features, circuit lock,…)

3. 3 15 April 2010: Post Mortem Analysis by M.Zerlauth Reminder: PM Server Architecture PM Data files PMA Scheduler Event Builder Data Collection PM Data Manager Libraries (references, DB Access Logging) GUI Modules Database LHC Logging PMA server PM buffers notifies triggers runs uses reads writes browses triggers Courtesy of Vito

4. 4 15 April 2010: Post Mortem Analysis by M.Zerlauth ONLINE and PLAYBACK servers  ONLINE server – Launched from CCM or http://abwww/ap/dist/pm/powering/pm-powering-gui/PRO/pmpow-online-gui.jnlp http://abwww/ap/dist/pm/powering/pm-powering-gui/PRO/pmpow-online-gui.jnlp – For real-time analysis in CCC – Completely automatic, including automatic confirmation & circuit locking – Keeps history of past 15 analysis sessions in memory  PLAYBACK server – Launched from CCM or http://abwww/ap/dist/pm/powering/pm-powering-gui/PRO/pmpow-playback-gui.jnlp http://abwww/ap/dist/pm/powering/pm-powering-gui/PRO/pmpow-playback-gui.jnlp – For offline analysis by equipment experts,… – Allowing replaying ANY event in the past – NO EFFECT ON LHC OPERATION, ie no circuit locking, etc…. – Manual confirmation/discard for potential saving of results – Keeps history of past 15 analysis sessions in memory

5. 5 15 April 2010: Post Mortem Analysis by M.Zerlauth What to do in case a circuit gets super-locked 1. Identify the super-locked circuit(s). Most likely the CCC will let you know, otherwise check the list of locked/super-locked circuits in the PIC SCADA – Circuits locked by POWERING SERVER will be marked in Comment

6. 6 15 April 2010: Post Mortem Analysis by M.Zerlauth What to do in case a circuit gets super-locked 2. Identify the EVENT during which the circuit was super-locked. If it is recent it will most likely still be present in ONLINE server, easiest to check PM Database (from CCM or https://cs-ccr-oas1.cern.ch/pls/htmldb_dbabco/f?p=117:1 ) https://cs-ccr-oas1.cern.ch/pls/htmldb_dbabco/f?p=117:1

7. 7 15 April 2010: Post Mortem Analysis by M.Zerlauth What to do in case a circuit gets super-locked 3. Open the according analysis SESSION, either in the ONLINE server (if still present) or in PLAYBACK In ONLINE: Use File -> Reconnect menu: In PLAYBACK: Use File -> Load Event menu:

8. 8 15 April 2010: Post Mortem Analysis by M.Zerlauth What to do in case a circuit gets super-locked 4. Identify event source + cause of locking in ‘Overview’ tab, eventually browse to details in ‘Modules Graph’ tab

9. 9 15 April 2010: Post Mortem Analysis by M.Zerlauth What to do in case a circuit gets super-locked 4. Identify event source + cause of locking in ‘Overview’ tab, eventually browse to details in ‘Modules Graph’ tab

10. 10 15 April 2010: Post Mortem Analysis by M.Zerlauth What to do in case a circuit gets super-locked 5. If applicable, document reason for failure in MP3 logbook, unlock circuit after authenticating in PIC SCADA

11. 11 15 April 2010: Post Mortem Analysis by M.Zerlauth Let’s try some examples….  14-OCT-10 12.05.41.315000 PM – RQX.R1 @ -1.493402958 – CIRCUIT_LOCK: Locked because of heater discharge  17-OCT-10 08.23.13.112000 PM – Sector 23 trip from 3.5TeV following quench test, Heater firing on MQ.14R2  23-OCT-10 06.52.25.085000 AM – Sector trip of A23 from injection current – Trip of RQ9.R2 (+ heater discharge) – RQTF.A23B2 @ 5.831202507 EE file not OK

12. 12 15 April 2010: Post Mortem Analysis by M.Zerlauth Conclusions  Server ready to be released mid next week (announcement will be sent to EIS’s, operators and MP3 members)  Limited number of modules included in initial version (a lot of tuning already done but tbc for e.g. 600A EE, 13kA EE, QPS_600….)  Expecting <1 circuit locks per day (depending on activities), please document in logbook cases where you think the circuit lock was unjustified  I’m available to help, don’t hesitate to call me if needed (I’ll receive as well automatic e-mails)  Next weeks essential for learning with new system, will add new modules starting early next year