Towards an Effective Software Component Certification Process Advisor Silvio Lemos Meira Student Alexandre Alvaro

Agenda Introduction –Reuse –RiSE {Reuse in Software Engineering} –CBD {Component-Based Development} –Component –Quality –Component Certification Component Certification History Future

Introduction {Reuse} [Frakes, 1995] “Software reuse is the use of existing software knowledge or artifacts to build new software artifacts.” “Everything that was done, should not be done again“

Introduction {Reuse} Advantages Increase Quality Productivity Decrease Time to market Manutenability Reduce life-cycle of development Software Reuse… –Not simple –Systematic Software Reuse

Introduction {RiSE} [Almeida et al., 2004] The IEEE International Conference on Information Reuse and Integration.

Introduction {CBD} Software reuse idea’s is not new [McIlroy, 1969] “Mass Produced Software Components” “To (re)use”, instead of “to develop” To keep a set of reused components Repository systems [Mili, 1998] 50 proposed solutions to this problem

Introduction {Component} “ A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be independently deployed and is subject to third-party composition”. [Szyperski, 2002] The exactly concept of component in CBD is not yet a consensus...

Introduction {Component} [Bass et al., 2000] CMU/SEI’s report Inhibitors: Lack of available components; Lack of stable standards for component technology ; Lack of certified components ; Lack of an engineering method to consistently produce quality systems from components.

Introduction {Component} Besides CMU/SEI... [Heineman, 2000] [Councill, 2001] [Crnkovic, 2001] [Wallnau, 2003]

Introduction {Quality} ISO 9000 CMM [Weber & Nascimento, 2002]

Introduction { Component Certification} Concept... “Third-party certification is a method to ensure that software components conform to well- defined standards; based on this certification, trusted assemblies of components can be constructed.” [Councill, 2001]

Introduction {Component Certification}

[Frakes et al., 1996]

History of Component Certification History of the Software Component Certification Decade of 90 –Mathematical models –Test-Based models

[Poore et al., 1993] “Planning and certifying software system reliability” –Three mathematics model Test cases Report the failures Data are analyzed to achieve a reliability index –Reliability of system Considering how the components affects this reliability History of Component Certification

[Wohlin & Runeson, 1994] “Certification of Software Components” –Method that consist: Usage model Usage profile –Test cases based on this models –Collection the failure data –Certification of reliability –Hypothesis certification Certify a specific reliability level -> given degree of confidence –Reutilization degree of the models History of Component Certification

[Rohde et al., 1996] “Certification of Reusable Software Components” –Rome Laboratory of the Air Force, NY History of Component Certification

[Rohde et al., 1996] “Certification of Reusable Software Components” –Certification process: 1.Readiness Assessment –Compile without error and execute the code 2.Static Analysis –Automatic tool 3.Code Inspection –Manual technique 4.Testing –Other tests –Analysis of the certification process… History of Component Certification

[TCI Initiative, 1998] 1 –Affiliation of researchers Formal interface specification –Supports compositional reasoning A restricted set of behavioral properties of assemblies –Difficult to find real contributions… 1 History of Component Certification

[Voas, 1998] “Certifying Off-the-Shelf Software Components” –Automated technologies Black-Box testing and fault injection –Methodology: Black-box component testing System-level fault injection Operational system testing –Certify components to a determined environment... History of Component Certification

[Wohlin & Regnell, 1998] “Reliability Certification of Software Components” –Extend [Wohlin & Runeson, 1994] work –Certification process 1.Usage specification (usage model and usage profile) 2.Certification procedure Three approaches: –Certification Process –Reliability Certification of Component and Systems –Certify or Derive System Reliability History of Component Certification

[Wohlin & Regnell, 1998] “Reliability Certification of Software Components” –Extend [Wohlin & Runeson, 1994] work History of Component Certification

[Wohlin & Regnell, 1998] “Reliability Certification of Software Components” –Extend [Wohlin & Runeson, 1994] work History of Component Certification ?

[Voas & Payne, 2000] “Dependability Certification of Software Components” –Metrics framework –Create a tests methodology… Component testability score –Mathematical models –Statistics approaches Estimates the number of test cases necessary Consider: –The number of tests that a component received; –The “fault revealing” ability of those test cases. History of Component Certification

[Morris et al., 2001] “Software Component Certification” –Four steps: Tests Specification Specification Document Specified Results Test-Pattern Verificator –Limitations… History of Component Certification

However... –Testing is not enough... [Sametinger, 1997] –Component certification levels Level 1: A component is described with keywords and a summary and is stored for automatic search. No tests are performed; the degree of completeness is unknown; Level 2: A source code component must be compiled and metrics are determined; Level 3: Testing, test data, and test results are added; Level 4: A reuse manual is added. History of Component Certification

[Heineman et al., 2000] –Panel presented in ICSE’2000 –Discuss the necessity of trust assurance in component –Considerable CBD researchers participate: Heineman {organizations} Councill {software development} Flynt {benefits to the customers} Shaw {reutilization of the components} History of Component Certification

Workshops… –4th ICSE Workshop on Component-Based Software Engineering (CBSE): Component Certification and System Prediction, –5th ICSE Workshop on Component- Based Software Engineering (CBSE): Benchmarks for Predictable Assembly, History of Component Certification

Long time considering just test... [Stafford & Wallnau, 2001] “Is Third Party Certification Necessary?” –Define a process model… Support prediction of system properties prior to component selection –Introduce “credentials” concept –Active component dossier A dossier is an abstract component that defines certain credentials History of Component Certification

Long time considering just test... [Stafford & Wallnau, 2001] “Is Third Party Certification Necessary?” History of Component Certification

[Stafford & Wallnau, 2001] “Is Third Party Certification Necessary?” –Some open questions: What level of trust is required? Are there other mechanisms that might be used to support trust? How to certify measurement techniques? History of Component Certification

Other authors… How certification should be carried out? (Goulao & Abreu, 2002) What does it mean to trust a component? (Hissam et al., 2003) What characteristics of a component make it certifiable, and what kinds of component properties can be certified? (Wallnau, 2003) History of Component Certification

[Councill, 2001] “Third-Party Certification and Its Required Elements” –Other aspect of component certification… Human Industrial Business –Certification is the components future… History of Component Certification

[Woodman et al., 2001] “Issues of CBD Product Quality and Process Quality” –Analyze some process in various CBD approaches –Examine 11 potential CBD quality attributes ReusabilityMaintainabilityAccuracyClarity ReplaceabilityInteroperabilityScalabilityPerformance FlexibilityAdaptabilityReliability History of Component Certification ReusabilityMaintainabilityAccuracyClarity ReplaceabilityInteroperabilityScalabilityPerformance FlexibilityAdaptabilityReliability

[Hissam & Wallnau, 2003] “Enabling Predictable Assembly” –Extends the [Stafford & Wallnau, 2001]work –Introduced Prediction-Enabled Component Technology (PECT) –Component technology with analysis technology Prediction of assembly properties Identify required component properties Certifiable properties History of Component Certification

[Hissam & Wallnau, 2003] “Enabling Predictable Assembly” –Extends the [Stafford & Wallnau, 2001]work –Component technology and analysis technology Component model Component runtime environment Assembly environment History of Component Certification Defines a property theory Parameters of this theory Component properties Increased accuracy prediction both More abstract, less acurate Increased accuracy prediction

[Hissam & Wallnau, 2003] “Enabling Predictable Assembly” –Extends the [Stafford & Wallnau, 2001]work –Validation: Empirical –Predictions made, conform to observations –Limitations Two prediction technology may be incompatible How are non-resource attributes, such as security, to be empirically validated? Industrial component certification ? History of Component Certification

[Meyer, 2003] “The Grand Challenge of Trusted Components” –Two complementary roads: Low Road –Qualification of existing components High Road –Production of components with fully proved correctness properties. History of Component Certification

[Meyer, 2003] “The Grand Challenge of Trusted Components” History of Component Certification

[McGregor, 2003] “Measuring Component Reliability” –Support prediction of assemblies reliabilities based on properties of the components –Method to measuring and communicating the reliability of the component Component’s services –Component’s documentation Test plan is created, based on component’s services Provide the reliability of each service –This method is a fundamental element of PECT History of Component Certification

[Wallnau, 2003] “Volume III: A Technology for Predictable Assembly from Certifiable Components” CMU/SEI’s report –How component technology can be extended in order to achieve Predictable Assembly from Certifiable Components (PACC). Runtime behavior of software components assemblies Component ’ s property –Component ’ s proprieties need rigorously defined and trusted; and –It can be certified by independent third-party developers History of Component Certification

[Wallnau, 2003] “Volume III: A Technology for Predictable Assembly from Certifiable Components” CMU/SEI’s report –SEI’s approach to PACC is PECT. History of Component Certification

[Wallnau, 2003] “Volume III: A Technology for Predictable Assembly from Certifiable Components” CMU/SEI’s report –Status: On going work –PECT is relatively immature One or more certification properties… Tools are being developed Functional certification complements the PECT [Meyer, 2003] –Precondition to PECT Non-functional properties History of Component Certification

Two failures case.... –National Information Assurance Partnership (NIAP) Together with NIST and NSA From 1993 until 1996 Defines criteria for certifying security features of components Restricted set of behavioral assembly properties. –IEEE 1997 The initiative was suspended, in this same year… History of Component Certification

Summary

Future – RiSE Context [Almeida et al., 2004] The IEEE International Conference on Information Reuse and Integration.

Future work 1.Key CBD Requirements What are the requirements for a certification process? [Woodman et al., 2001] 11 CBD requirements [Simao,2003] 124 CBD requirements [Larson, 2004] 72 CBD requirements 2.Component Quality Model What requirements are more important ? [Meyer, 2003]

Future work 3.Certification Method How certify components ? 4.A Metrics Framework How to measure the component certification processes ?

Future work Write a Paper –“On the Software Component Certification Process” The history The proposal

Future work

References [Frakes, 1995] Frakes, W., B., Fox, C., J. Sixteen Questions about Software Reuse. Communications of the ACM, June, [Szyperski, 2002] Szyperski, C., Component Software: Beyond Object-Oriented Programming. Addison-Wesley, USA. ISBN [Mcllroy, 1968] Mcllroy, M. D., Mass Produced Software Components. NATO Software Engineering Conference Report, October, pp [Mili et al., 1998] Mili, A., Mili, R., Mittermeir, R., A Survey of Software Reuse Libraries. Annals Software Engineering, Vol. 05, pp. 349–414. [Heineman & Councill, 2001] Heineman, G. T., Councill, W. T., Component- Based Software Engineering: Putting the Pieces Together. Addison-Wesley, USA. ISBN: [Heineman et al., 2000] Heineman, G. T., Councill, W. T., Flynt, J. S., Mehta, A., Speed, J. R., Shaw, M., Component-Based Software Engineering and the Issue of Trust. The IEEE Proceedings of the 22 nd International Conference on Software Engineering (ICSE), Canada, pp [Crnkovic, 2001] Crnkovic, I., Component-based software engineering - new challenges in software development. Software Focus, Vol. 2, No. 4, pp

References [Wallnau, 2003] Wallnau, K. C., Volume III: A Technology for Predictable Assembly from Certifiable Components. Software Engineering Institute (SEI), Technical Report, Vol. III, April. [Frakes & Terry, 1996] Frakes, W., Terry, C., Software Reuse: Metrics and Models. ACM Computing Survey, Vol. 28, No. 2, June, pp [Poore et al., 1993] Poore, J., Mills, H., Mutchler, D., Planning and certifying software system reliability. IEEE Computer, Vol. 10, No. 1, January, pp [Wohlin & Runeson, 1994] Wohlin, C., Runeson, P., Certification of Software Components. IEEE Transactions on Software Engineering, Vol. 20, No. 6, June, pp [Rohde et al., 1996] Rohde, S. L., Dyson, K. A., Geriner, P. T., Cerino, D. A., Certification of Reusable Software Components: Summary of Work in Progress. The IEEE Proceedings of the 2nd International Conference on Engineering of Complex Computer Systems (ICECCS), Canada, pp

References [Voas, 1998] Voas, J. M., Certifying Off-the-Shelf Software Components. IEEE Computer, Vol. 31, No. 6, June, pp [Wohlin & Regnell, 1998] Wohlin, C., Regnell, B., Reliability Certification of Software Components. The IEEE Proceedings of the 5th International Conference on Software Reuse (ICSR), Canada, pp [Voas & Payne, 2000] Voas, J. M., Payne, J., Dependability Certification of Software Components. Journal of Systems and Software, Vol. 52, No.2-3, June, pp [Morris et al., 2001] Morris, J., Lee, G., Parker, K., Bundell, G. A., Lam, C. P., Software Component Certification. IEEE Computer, Vol. 34, No. 9, September, pp [Sametinger, 1997] Sametinger, J., Software Engineering with Reusable Components. Springer Verlag, USA. ISBN

References [Stafford & Wallnau, 2001] Stafford, J., Wallnau, K. C., Is Third Party Certification Necessary?. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May, pp. 13–17. [Councill, 2001] Councill, B., Third-Party Certification and Its Required Elements. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May. [Woodman et al., 2001] Woodman, M., Benebiktsson, O., Lefever, B., Stallinger, F., Issues of CBD Product Quality and Process Quality. The IEEE Proceedings of the 4th ICSE Workshop on Component-Based Software Engineering (CBSE), Canada, May. [Hissam et al., 2003] Hissam, S. A., Moreno, G. A., Stafford, J., Wallnau, K. C., Enabling Predictable Assembly. Journal of Systems and Software, Vol. 65, No. 3, March, pp [Meyer, 2003] Meyer, B., The Grand Challenge of Trusted Components. The IEEE Proceedings of 25 th International Conference on Software Engineering (ICSE), USA, pp. 660–667.

References [McGregor et al., 2003] McGregor, J. D., Stafford, J. A., Cho, I. H., Measuring Component Reliability. The IEEE Proceedings of the 6th ICSE Workshop on Component- Based Software Engineering (CBSE), USA, May, pp [Schmidt, 2003] Schmidt, H., Trustworthy components: compositionality and prediction. Journal of Systems and Software, Vol. 65, No. 3, March, pp [Simão, 2003] R. Simao, A. Belchior, Quality Characteristics for Software Components: Hierarchy and Quality Guides. Lecture Notes in Computer Science, pp , June. Springer-Verlag [Larson, 2004] M. Larson, Predicting Quality Attributes in Component-based Software Systems, PhD Thesis, Malardalen University, 2004.