1 15 April 2010: Post Mortem Analysis by M.Zerlauth Automated analysis of Powering Events – Progress update

2 15 April 2010: Post Mortem Analysis by M.Zerlauth Outline  Implementation is progressing well (even if not yet fully visible), will need few weeks more to tie together all pieces and release a 1 st ONLINE version (including interlocking of circuits)  Regular meetings taking place to follow the progress, workload shared between JAVA and LV developments  Currently focusing on most frequent tasks, e.g. EE openings, 600A trips, converter faults, etc… more elaborate use cases to follow later…  Reminder: Approach is purely data driven, no PM data -> potentially no analysis and no circuit locking

3 15 April 2010: Post Mortem Analysis by M.Zerlauth POWERING analysis servers up and running  POWERING event building and analysis servers are defined and up and running (ONLINE + OFFLINE) -> See later demo..  POWERING version of PMA GUI (1 st version only), only few modules included yet… – Including 1 st version of PM basket (Viewer part to be completed) – Dedicated summary screen defined but to be implemented by CO-AP

4 15 April 2010: Post Mortem Analysis by M.Zerlauth Interlocking and DB upload  (Super-)Locking mechanism on PIC via CMW implemented and deployed – Possibility to send additional comment for lock (analysis result description)  DB table + APEX interface (for web-access to DB contents and statistics) defined but to be implemented by CO-DM

5 15 April 2010: Post Mortem Analysis by M.Zerlauth Agreed minimum requirements for 1 st ONLINE version  JAVA modules: – Single Circuit Event module, to identify event source and failure type – DONE – Automatic Event Recognition Module, final module for determination of machine protection features, event summary for DB upload, circuit locking, etc…– DONE, (needs input of additional modules when available) – 600A EE/13kA verification to check switch opening – DONE – 600A EE/13kA verification to check switch opening – In progress by Arek (similar module and GUI structure as for 600A EE)  LV modules: – QH Discharge module: Mostly completed for MB and MQ magnets. To come: IPQ, IPD and IT (needs configuration DB/look-up table for discharge time, heater #, etc…) -> Zinur and Nuria – 600A quench (exclusion) module : Module to determine whether a 600A circuit really quenched or not + provide a potential fault cause (e.g. zero- voltage crossing, feedback, didt/acceleration,…) -> ongoing by Odd ? – FGC Faults + Discharge analysis to validate discharge curve + reveal the possible failure of the power converter (water fault, fan fault, etc..) -> Discussion started with Hubert

6 15 April 2010: Post Mortem Analysis by M.Zerlauth Some examples which are being developed but not yet released …

7 15 April 2010: Post Mortem Analysis by M.Zerlauth Some examples which are being developed but not yet released …

8 15 April 2010: Post Mortem Analysis by M.Zerlauth Demo – how a typical MP3 day looks in the PMA server

9 15 April 2010: Post Mortem Analysis by M.Zerlauth Demo – how a typical MP3 day looks in the PMA server – ctd…

10 15 April 2010: Post Mortem Analysis by M.Zerlauth Conclusions  Basic infrastructure mostly in place, needs some tying up (locking mechanism, DB upload), will be finished in 1-2 weeks  Module development well on it’s way, still loads of work to be done (anyone interested to contribute)?  Servers are available for playing here: postmortem/ postmortem/  First operational (and reliable) release hopefully in 4-5 weeks

11 15 April 2010: Post Mortem Analysis by M.Zerlauth

12 15 April 2010: Post Mortem Analysis by M.Zerlauth Examples where we might want to improve… (from our MP3 logbook)  Sector 3-4 SOC : - Feb 2010 CIRCUIT : many COMMENT : This concerns a provoked quench on dipole A19R3 a few days ago (on 20/2/2010 at 23h11) at a current of 6 kA. It did not cause secondary quenches in other dipoles but caused trips in many other circuits. Nothing is mentioned in the MP3 logbook about this (neither in QPS or OP logbooks)!!?? Circuits concerned: - RB.A34 - RQS.L4B1 - ROD.A34B2 - RCO.A34B2 - RSS.A34B1 - RQTD.A34B2 - RQTL11.R3B2 - RCS.A34B2 - RQTL9.R3B2 - RSD2.A34B2 - RQT13.R3B2 - RQ6.R3B2 - bus RQD.A34 - bus RQF.A34 Maybe a point of discussion for MP3. Do we have to analyze all trips/quenches (I think yes) or not. Quench signals on A19R3 look normal. I did not look at the others.  Major trip (All sectors but 56&81. - Feb kV failure in Meyrin Seen by the PC as external FPA About 400 PM files!! No heater discharged for the main circuits Could be interesting to have a check list for similar cases. Temperature of RB, RQF, RQD resistors not increasing (<50 deg) There are QPS files for many circuits (In 600 A QPS1 (2), 600 A QPS2 (2), 600 A QPS4 (about 50)) and also for dipole ; checked some of them and they are not quenches