WP3: D3.1 status, pending comments and next steps Leonardo Montecchi lmontecchi@unifi.it http://rcl.dsi.unifi.it Resilient Computing Lab

Outline Status overview ToC Major comments Next steps

D3.1: status overview Title: Three main chapters: Modeling and Evaluation: State-of-the-art Three main chapters: Model-based approaches Experimental measurement approaches Works combining different evaluation approaches Partners involved: ALL but UniRM All the expected contributions have been provided. NOTE: Some of them appears to be out of the scope of the deliverable (see the major comments) Still to be completed (by UniFI): Executive summary + Conclusions

ToC (version v06) – 1/3 Executive Summary 6 1. Introduction (UniFI) 7 2. Model-based approaches (UniFI) 8 2.1 Formalisms for modelling dependability 8 2.1.1 Combinatorial models 8 2.1.2 State-based models 9 2.2 Model construction and solution approaches 10 2.2.1 Compositional approaches 10 2.2.2 Decomposition and aggregation approaches 12 2.3 Dependability modeling and solution tools 14 2.4 Deriving dependability models from engineering models 15

ToC (version v06) – 2/3 3. Experimental measurement approaches (UniMORE) 18 3.1 SCADA-based LCCIs (UniPARTHENOPE) 18 3.1.1 Metrics (UniPARTHENOPE+UniMORE) 18 3.1.1.1 Security Metrics 19 3.1.1.2 Quality of a Metric 19 3.1.1.3 Security Metric Domains 21 3.1.1.4 Dependability Metrics 21 3.1.2 Dependability and Security Benchmarking of SCADA based LCCIs (UniPARTHENOPE) 25 3.1.3 Data filtering and analysis (UniMORE) 28 3.1.4 Anomaly detection (UniMORE) 34 3.2 Systems for LCCI security (UniMORE) 44 3.2.1 Protocol vulnerability (UniPARTHENOPE) 45 3.2.2 Intrusion Detection Systems for LCCIs (UniMORE) 55 3.3 Field Failure Data Analysis (FFDA) (UniNA) 59 3.4 On line Monitoring (UniNA) 62 3.5 Fault Injection (UniNA) 65

ToC (version v06) – 3/3 4. Works combining different evaluation approaches (UniFI) 70 4.1 Relationships between modeling and experimentation 70 4.2 Works combining modeling and simulation 71 4.3 An holistic evaluation framework 72 5. Conclusions (UniFI) 75

Major comments to be discussed - 1 Positioning of Section 3.1.1 (“Metrics”) -UniPARTHENOPE+UniMORE. Comments: Section 3.1.1 defines the possible objectives of the analyses, i.e., the dependability and security metrics to be evaluated. It is something shared between all the evaluation approaches, not only related to experimental measurement approaches. The last page of subsection 3.1.1.4 (“Dependability Metrics”) describes one specific work ([Romano 1999]): why only this work and not a survey? And why this discussion has been inserted in the dependability metrics section? Proposed action: Move section 3.1.1 at the beginning of the deliverable as a stand-alone chapter, or as a section within chapter 1. Find the right place where [Romano 1999] can be discussed, possibly adding other works as required for a state-of-the-art.

Major comments to be discussed – 2 Content of Section 3.1.2 (“Dependability and Security Benchmarking of SCADA based LCCIs”) – UniPARTHENOPE. Comment: The section provides an overview of DBench, but it is not clear what are the specificities of DBench with respect to SCADA based LCCIs. The section’s title suggests a s-o-t-a on dependability and security benchmarking, not just the description of DBench. It is not clear how the subsections “Static code review”, “dynamic vulnerability analysis” and “vulnerability scanners” are related to the main benchmark section they belong to. The content of subsections “static code review” and “vulnerability scanners” seems to be out of the scope of the deliverable. Proposed actions: Complete the section adding works dealing with dependability and security benchmarking analysis for LCCIs. Clarify the links between the subsections and the main section. Move to “D2.1 Diagnosis and reconfiguration: state-of-the-art”? or revise the two sections that seem to be outside the scope of D3.1.

Major comments to be discussed – 3 Content of Section 3.2.1 (“Protocol vulnerability”) – UniPARTHENOPE. Comment: The whole section appears to be outside the scope of the deliverable. D3.1 should be a survey on modeling and evaluation aspects the section first presents a set of protocols for SCADA systems and then lists the related vulnerabilities and weaknesses. Proposed action: Move the section to “D2.1 Diagnosis and reconfiguration: state-of-the-art”? or rewrite it, e.g. describing the modeling and evaluation techniques used to identify the protocol vulnerabilities and weaknesses.

Major comments to be discussed – 4 Content of Section 3.2.2 (“Intrusion detection systems for LCCIs”) – UniMORE. Comment: Some works surveyed within this section actually deal with evaluation techniques used for intrusion detection, while others are more focused on the systems that use such techniques. This second set of works appears to be outside the scope of the deliverable. Proposed actions: Remove from the section the works focusing on the system definition (possibly moving it to “D1.2 Architectures, algorithms and middleware”?). In general, review the section to better fit the scope of the deliverable. Change the section’s title e.g. “Evaluation methods for Intrusion detection in LCCIs”.

Major comments to be discussed – 5 Works combining different modeling formalisms Comment: A section that explicitly surveys the works combining different modeling formalisms is currently missing, but it is an important topic. Proposed action: Insert a new section within Chapter 4 e.g. titled “Approaches combining different modeling formalisms”, which should also include the contribution by UniNA currently inserted at the end of section 4.3 (holistic evaluation approaches). The title of Chapter 4 should change as well, like “Works combining different modeling formalisms and evaluation approaches”

Next steps (to be agreed) Feb 28: ALL the partners send the agreed modifications; Mar 2: UniFI distributes a new version of D3.1 for last check and (minor) modifications; Finalizing exec summary & conclusions; Addressing minor (typo and editorial) comments; Emphasizing in the doc the (minor) pending issues still to be addressed by the partners; March 9: Partners send final (minor) modifications March 11: UniFI distributes the final version of D3.1

Thank you for your attention!