1. Peter Chochula ALICE DCS Workshop, October 6,2005 PVSSII Alert Handling

2. Outline  PVSS Alert handling  Functionality  Performance  Future developments

3.  PVSSII offers a comprehensive alert handling strategy based on industrial standards:  DIN 3699 Process control using display screens  DIN 19235 Measurement and control; signaling of operating conditions

4.  Alerts are sent if the monitored DPE value changes from one alert range to another  Alerts can be visualized :  As text on alert screen  As state display (e.g. flashing graphical element)  Alerts might require acknowledgment on the AES  Automatic functions can be assigned to alerts  The function is executed when the alert is triggered

5. Priorities and Priority Levels in the PVSS  There are 255 priorities available to facilitate the optimization of process status handling  Priorities can be freely grouped into the priority levels  The example shows six default priority levels Priority levelPriority Information10-19 Advance Alarm 20-39 Warning40-69 Alert70-89 Danger90-100 Disturbance>100

6. Alert Ranges  Alert ranges must be assigned to priority levels  For each DPE at least two ranges must exist:  Valid range (Priority 0)  Alert range  Example: temperature probe with 5 defined ranges Alert RangeNumerical range Priority level Very high value above limit > 50 CDanger High value above limit 30-50 CAlert Valid20-30 C0 Low value below limit 10-20 CAlert Very low value below limit <10 CDanger

7. Alert Classes  Alert classes simplify the parameterization of alert handling  by grouping together the parameterizable properties of alert ranges of datapoint elements  Each alert has an alert class assigned  Alert class describes the acknowledgment mode for the alert  CTRL script can be used for example to change the status of graphical elements, start the horn, display a message etc.  The six default alert levels shown in the previous slide correspond to six default alert classes (information, advance alarm, warning, alert, danger, disturbance)  Users can define additional alert classes

8. Example of Alert Parametrization Using Ranges and Classes

9. Alert Statuses  Each alert can have one of 5 possible statuses:  No Alert – normal status  Incoming not acknowledged – alert active, not yet acknowledged  Incoming acknowledged – alert active, already acknowledged  Outgoing unacknowledged – status not pending, not yet acknowledged  Outgoing acknowledged – status not pending, already acknowledged

10. Acknowledgment types  Acknowledgment types are assigned to DPEs to allow for different handling algorithms 1. Not acknowledgeable – alert changes only as a result of status change 2. Acknowledgment deletes – pending alert is reset to No Alert 3. Incoming acknowledgeable – the message is acknowledgeable 4. Alert pair requires acknowledgement – same as 3, but outgoing alert also requires acknowledgment. New alert overwrites the old one 5. Incoming and Outgoing require acknowledgement – same as 4, but both alerts must be acknowledged (new alert does not overwrite the old one)

11. Group alerts  Several alerts can be OR linked to form a group alert  An alert is fired if any of the linked alerts is pending  Using the panel topology the group alerts can be linked across several panels  Only summary information Is brought to the attention of the operator (e.g. crate has a problem if any of the channels has a problem)

12. Suppression of alerts  Suppression function can activate/deactivate any of the alerts  Very useful feature if the monitored value is outside of limits, but this is irrelevant for the process (e.g. current fluctuations during ramping process or FERO configuration)  Suppressed alerts are considered as non- existent

13. Alert Hysteresis  Hysteresis is defined to avoid repeated alert generation for values fluctuating around the alert limit  Value related hysteresis uses dead zones around the alert limit  Time hysteresis for digital values – bit must remain in a given state for certain amount of time  Time hysteresis for analog values – adds time inertia to value related hysteresis

14. Value related Hysteresis Range 1 Range 2 Range 3 12321 Upper Limit 1 Lower Limit 2 Upper Limit 2 Lower Limit 3 Upper Limit 3 Priority levels defined in the PVSS

15. Hysteresis parametrization

16. Relation between the PVSSII Alert Handling and the FSM  Described alert handling is related to the low-level PVSSII operation  At the detector level, the alerts influence the global states of the detector  Filtering and masking mechanism can affect the reaction of the FSM to the alert status  If a pending alert changes the FSM state, this can be recovered only if the alert status changes or if the alert is masked

17. Example of the Relation between the PVSSI Alerts and the FSM  Imagine a situation when an overcurrent on a single channel puts the detector into Error condition  Scenario 1: the recovery procedure resets the channel (cures the overcurrent status) and FSM state can be recovered to READY  Scenario 2: the recovery procedure changes the alert range, so the overcurrent will not trigger an alert anymore (but the overcurrent condition will persist)  Scenario 3: the channel will be taken out from the FSM hierarchy, so it will not contribute to the resulting state computation (the error condition on the channel will persist)

18. Error handling  By default the PVSSII errors are passed to the Log Viewer  Error handler can be used to redirect the errors  To the standard output  To the database

19. Alert handling performance  Several tests performed within the SUP and FWWG  SUP (Paul Burkimsher) studied the load of the systems and alert digestion  FWWG (Oliver Holme) studied the efficiency of different alert filtering modes

20. What is the system load caused by the alert definition? (SUP test 38)  Question: does the alert definition affect the achievable DPE change rate?  Answer: No, if the alert is not provoked, the DPE change rate is not affected by the alert definition (with or without alert checking)  The test system could cope with 1400 DPE changes/s  What is the load if the alert is provoked?  If one level was passed the rate dropped to 1100 changes/s  If two levels were passed, the rate dropped to 725 changes/s (because both alerts were processed)

21. Memory consumption caused by alert definition (Test 39)  Tests were performed by defining the DPE and comparing the memory used by the EM  After adding alert definition, the memory consumption was recalculated  Declaration of alerts takes extra memory  5 level alert definition requires app. additional 2500 bytes per DPE  Activation of alerts take ~2% more memory

22. Absorption of alerts  Test Setup  2 systems  5 crates defined per system  256 channels per crate  40000 DPE per system  2 alerts defined per channel  5 alert ranges per voltage channel  3 alert ranges per current channel  Each channel was provoked to pass 2 levels  Test Results  10000 lines (from both systems) displayed in 30s  1000 came alerts absorbed in 15s  5000 came alerts cancelled in 45s  Acknowledgments of 10000 alerts takes 2 min 20s  Systems can cope with sustained rate of 200 alerts/s

23. Interpretation of test results  The described tests showed limits for alert absorbtion by the PVSSII system  Further tests were performed to show the difference between displaying alerts from the local and a remote system  For a typical CAEN crate, all local alerts are displayed within 2 sec  If display of additional alerts from remote system is required, the total time is ~3 sec

24. Present status abd developments  The alert definition and handling Is fully supported by all FW devices  Tools are available in the FW  Alert configuration is supported by the FW configuration database tool  FWWG is studying improved AES screen  ETM is working on improved alert filtering methods

25. Conclusions  PVSSII offers powerful alert handling mechanism based on industrial standards  A single system can cope with alert avalanche of 10000 alerts  A single system can support a sustained alert rate of ~200 alerts/sec  Group alerts and panel topology allow for efficient alert reporting  In distributed system each AES screen can retrieve alerts from a remote system without significant overhead  New developments are underway