Presentation is loading. Please wait.

Presentation is loading. Please wait.

15 Years of Expert Judgement at TUDelft Louis Goossens & Roger Cooke and Andrew Hale & Ljiljana Rodic WOS2006, Zeewolde 12-15 September 2006.

Similar presentations


Presentation on theme: "15 Years of Expert Judgement at TUDelft Louis Goossens & Roger Cooke and Andrew Hale & Ljiljana Rodic WOS2006, Zeewolde 12-15 September 2006."— Presentation transcript:

1 15 Years of Expert Judgement at TUDelft Louis Goossens & Roger Cooke and Andrew Hale & Ljiljana Rodic WOS2006, Zeewolde September 2006

2 !! A TRIBUTE TO ANDREW HALE Professor emeritus in spe in Safety Science at TUDelft

3 The Delft Method of Expert Judgement Elicitation Developed by Roger Cooke – early 90-ies Support from –Ministry of Housing, Physical Planning and the Environment (the Netherlands) –European Commission Expert Judgement Procedures Guide (EUR 18820, 2000) Main goal: RATIONAL CONSENSUS in Decision-Making

4 Applications of the Delft Method In total (we elicit mostly the 5, 50, 95 percentile assessments of unknown variables) –880 experts –4,339 variables (“the unknowns and knowns”) –82,585 elicitations (total number of questions)

5 SectorPanelsExptsVariablElicits Nuclear appl.10982,20320,461 Chem/gas ind ,386 Chem. toxicity ,105 Groundwtr/wtrpollut Movbarr/dikering ,217 Volcanos/dams ,079 Space/debris/aviat ,149 Health:bovine CARMA/SARS ,979 Banking ,324 Ladders/buildings In total475874,13767,759

6 ?? WHAT HAS ANDREW HALE GOT TO DO WITH EXPERT JUDGEMENT?

7 Applications of the PC method Separate assessments (pairwise comparisons) –293 experts –202 variables –14,826 elicitations 2 Major projects: SAFETY MANAGEMENT developments: ANDREW HALE RELIABILITY of LANDFILL LINERS LJILJANA RODIC

8 How does the PC Method work? If you want to compare 3 attributes (A, B, C) wrt the impact on safety improvement for a particular safety management strategy, you ask the experts to assess each pair of attributes possible: A and B, A and C, and B and C Which attribute has the highest safety improvement potential in each pair ? n attributes require n(n-1)/2 assessments

9 Safety Management and PC I-Risk project based safety impact on reducing the risk The risk was calculated with Master Logic Diagrams containing parameter failure rates, inspection intervals, and maintenance parameters Safety management can influence the parameter data in the risk calculation by introducing safety measures: and then, which safety measure should have priority? The PC Method is available to compare priorities for safety measures

10 One example from Andrew’s work Maintenance management (MM) in chemical plants Risk has the following parameters for MM: –Tm (time for maintenance) –Tr (time for repair) – Im (time of maintenance interval) – It (time of test interval) –L/L (probability of failure to replace like with like –RDe (respect of equipment design enveloppe) –Hem (human error in maintenance) –Hei (human error in inspection)

11 Safety management Safety management consists of generic management areas, which determine the quality of completion of safety critical tasks In I-Risk the generic management areas were: – Availability of suitable personnel – Competence of those personnel – their Commitment to safety – Communication and coordination – Conflict resolution (priority of safety vs other goals) – Interface design – Procedures and plans – delivery of correct spares and replacements

12 Ask the right questions To connect the 8 generic management areas with the 8 risk parameters a protocol is applied: – detail the 8 parameters – describe relevant scenarios for deviations in the optimal values of these parameters – define a detailed task list to manage these deviations – detail the 8 generic management areas

13 Example of task list Time for preventive maintenance is determined by the following main tasks: – decide maintenance concept – plan and prioritise maintenance work – schedule maintenance work – prepare maintenance work – prepare area – do maintenance – recommission plant – record experience Phrase these main tasks in questions, leading to 8 attributes to compare with expert judgement

14 Results for hardware failure influences Area of Management ControlPercentage of influence Change (configuration) management47 % Operations17 % Engineering design11.5 % Process11 % Maintenance execution (RDe)9 % Maintenance interval (Im)4 % Spares1 % Raw materials0.5 % Number of experts11

15 Conclusions Although the views on safety management have been changed gradually during the past years, the Paired Comparisons Protocol, developed mainly by Andrew Hale, does not need to change at all.

16 Performance measures Calibration (statistical likelihood) Information (wrt background measure) Range graphs expertwise

17 EU-USNRC Dry Deposition 03/07/2003 ________________________________________________________________________________ Results of scoring experts Bayesian Updates: no Weights: global DM Optimisation: yes Significance Level: Calibration Power: 1 ______________________________________________________________________________________ Nr.| Id |Calibr. |Mean relat|Mean relat|Numb|UnNormaliz|Normaliz.w|Normaliz.w | | | total |realizatii|real|weight |without DM|with DM ______|________|__________|__________|__________|____|__________|__________|__________ 1|Expert1 | 3.064E-5| | | 14| 0| 0| 0 2|Expert2 | | | | 14| | | |Expert3 | | 0.679| 0.41| 14| | | |Expert4 | | | | 14| | | |Expert5 | 2.054E-8| 0.789| | 14| 0| 0| 0 6|Expert6 | | 1.188| 1.341| 14| | | |Expert7 | | | | 14| | | |Expert8 | | | | 14| 0| 0| 0 9|perf wgt| | | 0.142| 14| | | |eq wgt | | | | 14| | | ______________________________________________________________________________________ ________________________________________________________________________________ (c) 1999 TU Delft

18 Dry Deposition Range Graphs: itemwise Item no.: 61 Item name: DD-E mu Scale: LOG Experts 1 [ * ] 2 [ * ] 3 [ * ] 4 [ * ] 5 [ * ] 6 [---*------] 7 [ * ] 8 [ * ] prf wgt [===================================*=====================] eq wgt [===========================*=============================] Real:::::::::::::::::::::::::::::::::::::::::::#:::::::::::::::::::::::::::::::


Download ppt "15 Years of Expert Judgement at TUDelft Louis Goossens & Roger Cooke and Andrew Hale & Ljiljana Rodic WOS2006, Zeewolde 12-15 September 2006."

Similar presentations


Ads by Google