1. Page 1 Building Reliable Component-based Systems Chapter 10 - Predicting System Trustworthiness Chapter 10 Predicting System Trustworthiness

2. Page 2 Building Reliable Component-based Systems Overview qIntroduction qWhat else can be done? qPredicting component interoperability qSummary

3. Page 3 Building Reliable Component-based Systems Introduction q Functional Composability (FC) and functional correctness: l FC is concerned with whether f(a) x f(b) = f(a x B) is true. l These concerns stem from the problem of composing "ilities".  Reliability  Safety  Security

4. Page 4 Building Reliable Component-based Systems The Problem q The problem stems from our inability to know a priori, l For example, that the security of a system composed of two components, A and B, can be determined from knowledge about the security of A and the security of B. q Why? l Because the security of the composite is based on more than just the security of the individual components.

5. Page 5 Building Reliable Component-based Systems An Example q As an example, suppose that: l A is an operating system and B is an intrusion detection system. l Operating systems have some level of built-in authentication security. l Intrusion detection systems have some definition of the types of event patterns that warn of a possible attack. q Thus, the security of the composition clearly depends on the security models of the individual components.

6. Page 6 Building Reliable Component-based Systems The Example Continued q But even if A has a worthless security policy or flawed implementation, the composite can still be secure. q How? l IF A has poor performance l THEN no one can log in OR l IF A's security mechanism not reliable l THEN security is increased q While these last 2 examples are clearly not a desirable way to attain higher levels of system security, both do actually decrease the likelihood that a system will be successfully attacked.

7. Page 7 Building Reliable Component-based Systems Another Example q A as an operating system and B as an intrusion detection system, l AND We assume that A provides excellent security and B provides excellent security, l WE MUST still accept the fact that the security of B is also a function of calendar time. q So the question then comes down to: which "ilities", if any, are easy to compose? l The answer is that there are no "ilities" easy to compose and that some are much harder to compose than others.

8. Page 8 Building Reliable Component-based Systems What Else Can Be Done? q If a piece of software fails only once after 100 tests, l DO NOT calculate quantitative score based on the result! l DO consider it to be the result of the testing.

9. Page 9 Building Reliable Component-based Systems Isolating Potential Contributors q Parties that have contributed software functionality (whether COTS or custom) to the system. q Potential contributors to the system failure include: l Defective software components l Problems with interfaces between components l Problems with assumptions between components l Hidden interfaces and non-functional component behaviors that cannot be detected at the component level.

10. Page 10 Building Reliable Component-based Systems Interface Propagation Analysis q Interface propagation analysis (IPA): l Perturbs the states that propagate through the interfaces that connect COTS software components to other types of components. l Note that software fault injection is also a form of accelerated testing.

11. Page 11 Building Reliable Component-based Systems Reliability Testing  Operational profile testing  test-cases  Test for defects occuring in operational phase  Many insignificant experiments  Time consuming Component/System Input

12. Page 12 Building Reliable Component-based Systems IPA at Work q To modify the information (states) that components use for inter-communication l write access to those states is required (in order to modify the data in those states). l This is obtained by creating a small software routine named PERTURB which replaces, during system execution, the original output state with a different (corrupted) state. Component A Component B Input

13. Page 13 Building Reliable Component-based Systems PERTURB q An Example using: double cos(double x) … if (cos(a) > THRESHOLD) {do something} … if (PERTURB(cos(a)) > THRESHOLD) {do something} q The value added by having a utility such as PERTURB is, in general, dependent on how well PERTURB mimics corruptions that the utility under consideration.

14. Page 14 Building Reliable Component-based Systems Technique 1 q The first technique: l Involves the deliberate inversion of the operational profile originally anticipated by the system designers. l This technique is most beneficial when the description of the expected profile is accurate. Component/System Input Inversed operation al profile

15. Page 15 Building Reliable Component-based Systems Technique 2 q The second technique: l Is simply a combination of the previous technique with IPA. l This is a situation in which the software is operating in an unusual input mode while being bombarded with corrupt information. Inversed operation al profile Component A Component B Input

16. Page 16 Building Reliable Component-based Systems Summary  Non-functional behaviors are difficult to handle in composition  Ordinary (reliability) testing is not enough  SWIFI can be used for testing non-functional behaviors  IPA is a technique for predicting interoperability  IPA is not the answer, but a complement to other (traditional) testing techniques q.