1. A Embedded software component quality framework Fernando Ferreira de Carvalho Advisor: Silvio Romero de Lemos Meira Informatics Center - Federal University of Pernambuco C.E.S.A.R. - Recife Center for Advanced Studies and Systems ffc@cin.ufpe.br 12-Junho-2008

2. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework Introduction / Motivation Embedded system is at the heart of many systems So, embedded system industry needs, Low production cost Short time to market High Quality to be more efficient and competitive (Brown, 2000) The CBD with reuse technique had been a nice direction to reach this objectives… But, Component reuse without quality assurance give catastrophic results [ariane].

3. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Introduction / Motivation The CBD technologies do not take into account the specifics needs of embedded- systems development: Timing, Memory, power, hardware constraints and others.

4. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Introduction / Solution o Certification is the future of software components [Wallnau, Heineman, Councill, Shaw] o According to Weber et al. (Weber et al., 2002), the need for quality assurance in software development has exponentially increased in the past few years

5. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Introduction / Motivation Benefits o More reliability o Short time-to-market; o More product quality; o higher quality levels, o reduced maintenance time, o investment return, o reduced time-to-market, among others o Wide used, where certification is mandatory (aircraft)

6. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Rise Framework

7. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process

8. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process focused in embedded systems An Embedded Computer System: A computer system that is part of a larger system and performs some of the requirements of that system; for example, a computer system used in an aircraft or rapid transit system. (IEEE,1992).

9. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process focused in embedded systems Embedded systems : Used for specifics propose Used to control systems (ex: mechanical machines) Ultra small devices with simple specific functionality Small systems with sophisticated functions Produced in large scale

10. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process focused in embedded systems Embedded systems has a specific requirements: - real-time - hi reliability - low power consumption - low data and code memory - low resources - low CPU capabilities - others

11. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Problem Formulation The CBD is an efficient and effective way for design of simple and complex embedded systems. However, quality assurance of components is must be done to take advantage of CBD. The Software industry still far to reach the maturity level the hardware industry which it have catalogues and datasheets available for its components. For this reason, a well-defined and consistent embedded software component quality assurance is essential for CBD and reuse adoption.

12. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Proposed solution An Embedded Software Component Quality Verification Framework It is composed of four inter-relation module:, based on a set of activities, metrics and guidelines. Embedded software component Quality Model (EQM) Maturity Level evaluation Techniques Metrics Approach Component Certification based on a set of activities, metrics and guidelines.

13. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Proposed solution This Framework is based in the standards ISO/IEC 9126, 2001 - Quality Model for Software Product ISO/IEC 14598, 1998 - Software Product Evaluation Process This two standards converged to: ISO/IEC 25010, 2005 - Software product quality - requirements and evaluation the Framework adapted the quality model and evaluation to component context and embedded domain.

14. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Out of scope This Framework is part of broad context, some aspects were expected since initial definition. Nevertheless, other process can be added in the future. Cost Model Formal Proof Prediction of the component assembly

15. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Embedded System Design Embedded system design comprise: Ultra-small device x simple functionality Small system x sophisticated functions Large systems and distributed systems Systems produced in large quantities x low production cost Systems produced in low volume x important features

16. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Embedded System Design The different requirements of embedded systems have a impact on feasibility, on use of CBD for it. A common characteristic in different area of embedded domain is increasing importance of software [Crnkovic, 2003]. Example, the software cost in embedded systems: in industrial robots constitute about 75% of total cots in car industry it is about 30% Fifteen year ago: 25% of total cots in industrial robots Negligible for cars

17. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Embedded System Design Properties that involves embedded software component is divided in: Functional property (component interface) Non-functional or Extra-functional property, so called Quality attributes, fox example: Timing Performance Consumption Resource Behavior, and others. This properties can be classified in run-time and life-cycle.

18. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Specific Requirements for Embedded System In the most of case, embedded system is real-time with limited resource. So, it has specifics characteristics which depends on domain application, but it have strong implication on requirements. The REQUIREMENTS are related Extra-functional property or Quality attributes, and its priority depends on the domain application.

19. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Specific Requirements for Embedded System Industrial Automation Automotive Medical Electronic consumer Other domain There has been developed a research in order to find the most important characteristics in different areas in embedded domain.

20. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Specific Requirements for Embedded System – Industrial Automation At low level: a.Availability b.Timeliness c.Reliability The most important characteristics, following the research: Industrial Automation was classified by research’s Larsson, [Larsson, 2002] At high level: a.Performanc e b.Usability c.Integrabilit y

21. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Specific Requirements for Embedded System - Automotive 1.Safety 2.Reliability 3.Predictability 4.Usability 5.Extendibility 6.Maintainabilit y 7.Efficiency 8.Testability 9.Security 10.Flexibility The resulting list of characteristics is presented below Akerholm [Akerholm, 2005] executed a research in vehicle industry.

22. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Specific Requirements for Embedded System - Medical The resulting list of characteristics is presented below Wijnstra [Wijnstra, 2001] describe their experience with characteristics in the development of medical imaging family. 1.Reliability 2.Safety 3.Functionality 4.Portability 5.Modifiability a.Configurability b.Extensibility and Evolvability 6.Security 7.Serviceability

23. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2 – Specific Requirements for Embedded System – Others Domain The table show the results. Crnkovic [Crnkovic, 2003] summarized the main characteristics and sub- characteristics in the CBD apply embedded system. CharacteristicsSub-characteristics Real-time propertiesResponse time or latency execution time worst case execution time Deadline DependabilityReliability Availability integrity confidentiality safety Resource consumptionPower consumption computation (CPU) power memory Consumption execution (CPU) time, Life cycle propertiesmaintainability

24. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 But the specifics requirements depends on the application (domain) - Industrial Automation - Automotive - Medical - Consumer electronics - Aircrafts - others … Component Certification Process focused in embedded systems

25. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process focused in embedded systems So, embedded component certification process must be different that general propose component, because the component evaluation is realized focused in other requirements We divided in two groups: General propose sw component certification process o desktops, servers, x86 architecture Specific propose sw component certification process (embedded systems, others architectures)

26. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process focused in embedded systems (requiriments) Industrial Automation (Crnkovic et al., 2005) In the lowest levels: availability, timeliness, and reliability At higher levels: performance, usability, and integrability

27. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process focused in embedded systems (requirements) Automotive (Åkerholm e.t. al, 2004) Extendibility Maintainability Usability Predictability Security Safety Reliability Testability Flexibility Efficiency

28. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Component Certification Process focused in embedded systems (requirements) Medical (Wijnstra e.t. al,2001) Reliability Safety Functionality Portability Modifiability Configurability Extensibility and Evolvability Testability Serviceability

29. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Certification is the future of software components [Wallnau, Heineman, Councill, Shaw] RiSE Approach 1.Embedded Component Quality Model (ECQM)  Embedded Software Component Maturity Model (ESCMM) 2.Certification Techniques Framework  Defining techniques for evaluate quality attributes of ECQM 3.Metrics Framework  Track the properties of the ECQM, certification techniques and process. 4.Embedded Software Component Certification Process  Defining the steps for certify a component Embedded Software Component Quality Process

30. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 1.Embedded Component Quality Model (ECQM) o Embedded Software Component Maturity Model (ESCMM) o Based ISO/IEC 25010 (*) standard, with some news and extended characteristics. Embedded Component Certification Process *ISO/IEC 9126 - Software Products Quality ISO/IEC 14598 - Evaluation software product Development time Cost Time to market Targeted market Affordability Licensing Marketability Deployability Replaceability Adaptability Reusability Portability Analyzability Stability Changeability Testability Maintainability Time Behavior Resource behavior Scalability Efficiency Understandability Configurability Learnability Operability Usability Maturity Recoverability Fault Tolerance Reliability Suitability Accuracy Interoperability Security Compliance Self-contained Functionality Sub-CharacteristicsCharacteristics Changes in the Proposed Component Quality Model, in relation to ISO/IEC 25010

31. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 1.Embedded Component Quality Model (ECQM) Embedded Component Certification Process Changes in the Proposed Component Qualit Model, in relation to ISO/IEC 25010 Development time Cost Time to market Targeted market Affordability Licensing Marketability Deployability Replaceability Adaptability Reusability Portability Analyzability Stability Changeability Testability Maintainability Time Behavior Resource behavior Scalability Efficiency Understandability Configurability Learnability Operability Usability Maturity Recoverability Fault Tolerance Reliability Suitability Accuracy Interoperability Security Compliance Self-contained Functionality Sub-CharacteristicsCharacteristics

32. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2. Certification Techniques Framework o Defining techniques for evaluate quality attributes of ECQM Guidelines for selecting evaluation level. Financial disasterMany people killedUnrecoverable environmental damage ESCMM V Large economic gross Threat to human livesRecoverable environment damage ESCMM IV Significant economic loss Large number of people disabled Damage properlyESCMM III Few economic lossFew people disabledSmall/Medium damage properly ESCMM II Negligible economic loss Few material damage; No specific risk No damageESCMM I EconomicSafety/SecurityEnvironment Level

33. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 2. Certification Techniques Framework o Defining techniques for evaluate quality attributes of ECQM Guidelines for selecting evaluation level. Programming Language Facilities (Best Practices) Maturity analysis ESCMM II Fault tolerance analysis Error Manipulation analysis ESCMM III Formal Proof Reliability growth model Suitability analysis Reliability ESCMM VESCMM IVESCMM ILevel

34. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 3. Metrics Framework o Track the properties of the ECQM, certification techniques and process. 0 <= x <= 1; which closer to 1 is betterInterpretation Precision on results / Amount of testsMetric Based on the amount of tests executed, how much test results return with precision? Question Evaluates the percentage of the results that were obtained with precision Goal CorrectnessQuality Attribute AccuracySub-Characteristic Functionality

35. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 4. Embedded Software Component Certification Process o Defining the steps for certify a component  4.1 Establish Evaluation Requirements activity  4.2 Specify the Evaluation activity  4.3 Design the Evaluation activity  4.4 Execute the Evaluation activity

36. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 4. Embedded Software Component Certification Process

37. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 4.1 Establish Evaluation Requirements activity

38. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 4.2 Specify the Evaluation activity

39. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 4.3 Design the Evaluation activity

40. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 4.4 Execute the Evaluation activity

41. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Submissão de artigo dia 05/06/2008 - SBCARS

42. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1

43. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Thank you ! Questions ? Fernando F. de Carvalho o ffc@cin.ufpe.br

44. http://www.rise.com.br Fernando Carvalho, PhD Candidate A Embedded software component quality framework SAAP 2008.1 Embedded Component Certification Timeline