Which role might model-based engineering play in software certification? Selo Sulistyo

2 1.Software Certification 2.Model-based Engineering (MBE) 3.Roles of MBE in Software Certification 4.Summary Outline

3 Software certification is the term used to certify the quality of a software application/product against the functional and non- functional specifications before its release to production/ end-user. (Eushiuan Tran, 1999) Definition Software Certification

4 The software certification process establishes the level of confidence in a software system in the context of its functional and safety requirements. Why is a certification needed? Software Certification

5 Virtual Machine byteCode Generates is executed by What is being certified? Software Certification Operating System Programming Languages Source Code Is used to produce application Running Application Machine Code is executed and managed by Is compiled using Compiler Generates Assembly Assembler Development process Computer Hardware Product

6  Product based Example: ISO IEC This international standard provides guidance for the practical implementation of software product evaluation when several parties need to understand, accept and trust evaluation results.  Process-based Example: IEC (Functional safety of electrical/electronic/ programmable electronic safety-related systems) DO-178B (Software Considerations in Airborne Systems and Equipment Certification). Certification Approaches Software Certification

7 Verification and validation are part of the certification process for software system (Eushiuan Tran, 1999) Certification process Software Certification Verification Validation Certification Developer Functional Expert Certification Authority

8 Development Testing Was the product built right? Was the right product built? iteration When and how the verification and validation are done? Software Certification artifacts requirements Requirement analysis design implementation verification deployment Validation Testing Deve- lopment

9 The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. Was the product built right? Verification does not ensure the application: – Solves an important problem – Correctly reflects the workings of a real world process Verification Software Certification

10 The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. Was the right product built? Validation Software Certification

11 Verification and Validation Techniques Software Certification

12 An example : DO-178B Software Certification

13 An example : DO-178B Software Certification

14 1.Software Certification 2.Model-based Engineering (MBE) 3.Role of MBE in Software Certification 4.Summary Outline

15 An approach to engineering that uses models as an integral part of the technical baseline that includes  the requirements,  analysis,  design,  implementation, and  verification of a capability, system, and/or product throughout the acquisition life cycle. Definition Model-based Engineering Models requirements Requirement analysis design implementation verification deployment Validation

16 A physical, mathematical, or otherwise logical representation of a system, entity phenomenon or process Models are scoped to purpose Models are appropriate to the context (e.g., application domain, life cycle phase) Models integrated or interoperable across domains and across lifecycle What is a model? Model-based Engineering

17 Software Model Model-based Engineering model system Modeling language is written in describes

18 Jos B. Warmer and Anneke Kleppe defined Modeling Maturity levels (MML)  MML 0: No Specification  MML 1: Textual Specification  MML 2: Text with Models  MML 3: Models with Text  MML 4: Precise Models  MML 5: Models only The use of model in software development Model-based Engineering requirements Requirement analysis design implementation verification deployment Validation Testing Deve- lopment

19 Testing Deve- lopment MML 0 No Specification My software … No specification Model-based Engineering requirements Requirement analysis design implementation verification deployment Validation

20 Testing Deve- lopment Specification of software is written down in one or more natural language documents. MML 1 Textual Specification Model-based Engineering requirements Requirement analysis design implementation verification deployment Validation

21 Testing Deve- lopment Specification of software in one or more natural language documents plus several high-level diagrams to explain the overall architecture. + MML 2 Text with Models Model-based Engineering requirements Requirement analysis design implementation verification deployment Validation

22 Testing Deve- lopment Specification of software is written down in one or more models. Additional natural language text is used to explain the background and motivation of the models. + MML 3 Models with Text Model-based Engineering requirements Requirement analysis design implementation verification deployment Validation

23 Testing Deve- lopment  Specification of software is written down in one or more models.  Natural language text is used to explain the background and motivation of the models.  The models are precise enough to have a direct link with the actual code. MML 4 Precise Models Model-based Engineering requirements Requirement analysis design implementation verification deployment Validation

24 Testing Develop ment  The models are precise and detailed enough to allow complete code- generation.  The code is invisible (as assembler is today).  Modeling language  High level programming language. MML 5 Models only Model-based Engineering requirements Requirement analysis design implementation verification deployment Validation

25  MML 0: No Specification  MML 1: Textual Specification  MML 2: Text with Models  MML 3: Models with Text  MML 4: Precise Models  MML 5: Models only Modeling Maturity Levels (MML) MBE ! Model-based Engineering

26 OMG’s Model-driven Architecture Model-based Engineering The current state of the art in MDE is much influenced by the ongoing standardisation activities around the OMG Model Driven Architecture ® (MDA ® ).  MDA is a framework which defines a model- driven approach to software systems development.  Focus on the solution of the problem rather than the implementation problem  Raising the level of abstraction, dealing with complexity  An automatic code generation

27 Models in MDA (Model-Driven Architecture) Model-based Engineering CIM: Computational-independent models PIM: Platform-independent Models PSM: Platform-specific Models Requirement Analysis Implementation Low-level design High-level design PIM Code PSM TT Code PSM TT Code PSM TT CIM TT TT: Transformation Tool

28 1.Software Certification 2.Model-based Engineering (MBE) 3.Role of MBE in Software Certification 4.Summary Outline

29 How does model-based engineering produce applications? Modeling Language Models Tools and Production processes Transformation tool e.g Code Generator ?? Role of MBE in Software Certification Running Application Products

30 Role of MBE in Software Certification Certification One central goal of MBE is to enable analysis of the system, thus ensuring the quality of the system already on the model level.  early verification and validation

31 requirements Requirement analysis design implementation Role of MBE in Software Certification When and how the verification and validation are done? How to verify these software models? Certification Authority verification deployment Validation Model Verification Model Validation Verified and Validated Software Models ….. Models Was the model built right? Was the right model built? Was the model built right?

32 Role of MBE in Software Certification Model verification  Verification is done to ensure that: – The model is developed correctly and conformance with standard – The algorithms have been implemented properly – The model does not contain errors or bugs  Verification does not ensure that the model: – Solves an important problem – Meets a specified set of model requirements – Correctly reflects the workings of a real world process

33 Role of MBE in Software Certification Verification techniques Verification techniques that can be used: – Model checking – proof of correctness – consistency checking This will ensure that the models are  Consistent  Correct against to language syntax of the language

34 Role of MBE in Software Certification Model validation  Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained.  Compliance with requirements  The ultimate goal of model validation is to make the model useful in the sense that the model addresses the right problem, provides accurate information about the system being modeled, and to make the model actually used.

35 Role of MBE in Software Certification B B B A A A CIM PIM Code PSM TT Model verification and validation Code PSM TT B Product family B A A: Verification -Consistency -Conformance with standard B: Validation -Compliance with requirements -Traceability TT: Transformation Tool

36  Models provides a view to a complex problem and its solutions.  Models are less risky, cheaper to develop and easier to understand than the implementation of a genuine target system  By assuming that the transformation tools is verified then the verification process can be done in model level, that obviously it saves time and cost.  It also improves the quality of design and software code by better assuring the traceability between artifacts. Role of MBE in Software Certification Benefits MBE in software certification

37 MBE can play role on:  Process-based certification.  Early verification process. Activities for verification, validation, and test with Model-Based Engineering can be applied at every stage of the development process.  The model traceability will minimize the certification efforts, especially for product families.  reducing certification and re-certification times,  Saving time and cost since the verification is done on higher abstraction levels, but it may be a complicated process with the current MDE technology. Summary

38 Thank You

39 1.Jos Warmer, The role of OCL in the MDA, Klasse Objecten, 2.Eushiuan Tran, Verification/Validation/Certification, Carnegie Mellon University, b Dependable Embedded Systems, Spring Rajwinder Kaur Panesar-Walawege, Mehrdad Sabetzadeh, Lionel Briand, Using Model-Driven Engineering for Managing Safety Evidence: Challenges, Vision and Experience, Simula Research Laboratory, University of Oslo, Lysaker, Norway 4.Ewen Denney and Bernd Fischer, Software Certification and Software Certificate Management Systems, USRA/RIACS, NASA Ames Research Center, Moffett Field, CA 94035, USA 5.OSMAN BALCI, A Methodology for Certification of Modeling and Simulation Applications, ACM Transactions on Modeling and Computer Simulation, Vol. 11, No. 4, October 2001, Pages 352– Damian Dechev, Bjarne Stroustrup, Model-based Product-Oriented Certification, Texas A&M University College Station, TX References