1. www.eu-etics.org INFSOM-RI-026753 A Quality Certification Model for Grid Research Projects the ETICS feasibility Study Adriano Rippa (adriano.rippa@eng.it) adriano.rippa@eng.it - Engineering Ingegneria Informatica s.p.a. - Engineering Ingegneria Informatica s.p.a. On behalf of the ETICS Project

2. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 2 Summary Grid Quality Certification Model (GQCM) - UPDATEGrid Quality Certification Model (GQCM) - UPDATE Comparing GQCM and other QA standardsComparing GQCM and other QA standards TimelineTimeline Gotten feedbacksGotten feedbacks PublicationsPublications A possible first implementationA possible first implementation

3. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 3 In Madison we left here… The Grid Quality Assurance Certification Process (GQACP) was organized in three perspectives looking at Code, Platforms and Standards quality aspects.The Grid Quality Assurance Certification Process (GQACP) was organized in three perspectives looking at Code, Platforms and Standards quality aspects. Now it asses the quality against the same macro- aspects but…Now it asses the quality against the same macro- aspects but… Code Platforms Standards

4. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 4 …Now we are here GQCM has been reorganized as follows:GQCM has been reorganized as follows: –A new name! –GQCM has been described according to several ISO standards (e.g. ISO/IEC 25000, 14598) –GQCM has been restructured according to ISO 25041 –Quality attributes has been named using the same terminology as ISO 9126 –More metrics has been added including bounds and target values

5. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 5 GQCM: the structure GQCM is structured in Evaluation Modules (EM) according to ISO 25041GQCM is structured in Evaluation Modules (EM) according to ISO 25041 The set of evaluation techniques are grouped in families. Every family is an EMThe set of evaluation techniques are grouped in families. Every family is an EM 5 EMs:5 EMs: –EM: Static analysis –EM: Coding style –EM: Structural testing –EM: Functional testing –EM: Standards compliance

6. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 6 Evaluation Modules 1/5 EM Static analysisEM Static analysis –Quality characteristics: –Reliability – maturity –Maintainability – analysability –Maintainability – changeability –Maintainability – testability –Technique: –Static analysis of classes. Statistics on measures are used as predictor of quality characteristics. –Applicability: –Object oriented programming languages. –Input: –source code, each class of the code is analysed

7. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 7 Evaluation Modules 2/5 EM Coding styleEM Coding style –Quality characteristics: –Maintainability – analysability –Technique: –Static analysis of the source code. –Applicability: –Most programming language –Input: –source code

8. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 8 Evaluation Modules 3/5 EM Structural testingEM Structural testing –Quality characteristics: –Functionality – accuracy –Reliability – maturity –Technique: –Structural testing. The intention of this EM is to test specific classes that are identified by static measures as being statistically more likely to have many errors. –Applicability: –Object oriented programming languages –Input: –source code

9. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 9 Evaluation Modules 4/5 EM Functional testingEM Functional testing –Quality characteristics: –Functionality – accuracy –Functionality – interoperability –Reliability – maturity –Portability – adaptability –Portability - installability –Technique: –Functional testing –The purpose of this EM is both to check platform compliance and to check to functional abilities of the software product. –Applicability: –General –Input: –compiled code –user documentation

10. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 10 Evaluation Modules 5/5 EM Standards complianceEM Standards compliance –Quality characteristics: –Functionality – standards compliance –Technique: –Functional testing –This EM has a good potential for automating the test. –Applicability: –Standard specific –Input: –compiled code

11. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 11 GQCM: Final Score Final score should be provided according to the following schema. The items which should be available for the users are:Final score should be provided according to the following schema. The items which should be available for the users are: –A table summarizing the results –A list of passed and non passed tests –All the important information as: –Software product (e.g. name, version, executable code, documentation..) –Platform (name, version, date) –Quality characteristics (name, evaluation result, evaluation module identification) –Standard compliance (for each standard: name, version, date) –Identification of evaluation report (organization, report number, date) –Identification of certification body (organization, contact information) –Certification data (dates, certification number) –Electronic signature of certification record

12. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 12 GQCM: Summarizing Table example Consolidated eval. result EM Static EM Coding EM structural EM Functional EM Std compliance EM eval. resultMMGGGY FunctionalityGGG AccuracyGGG InteroperabilityGG ComplianceY(Y) ReliabilityMMGG MaturityMMGG MaintainabilityMMG AnalyzabilityMMG ChangeabilityMM TestabilityMM PortabilityGG AdaptabilityGG InstallabilityGG G = Good M = Medium P = Poor

13. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 13 GQCM integration in an ISO 9126 certified organization Facts:Facts: –ISO 9126 define quality attributes –GQCM support the measure and evaluation of quality metrics Why they are compatible each other?Why they are compatible each other? –Many ISO9126 aspects can be evaluated by GQCM  GQCM apply what ISO 9126 asks –Any tool implementing GQCM and can be integrated as supporting ISO9126 adoption SoSo –GQCM can be both an independent quality model and a “tool” to be easily compliant with ISO 9126

14. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 14 GQCM integration in a CMMi certified organization FactsFacts –CMM is process oriented (it gives guidelines). –GQCM is product oriented. Hypothesis: Hypothesis: –CMM2 –asks to “control and value” –GQCM proposes the measures and metrics to be collected, hence controlling and evaluating specific sw related items –CMM3 –asks to “use internal standards and have a reference model” –GQCM proposes code and implementation conventions (how to organise and structure code, how to define tests, etc.)

15. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 15 GQCM integration in a CMM certified organization CMM4CMM4 –asks to “have measured and controlled processes using quantitative and statistical techniques” –GQCM proposes the continuous collection of quality data to perform trends analysis. CMM5CMM5 –asks to have an “improvement process and quality based on measurement” –GQCM provides numerical reference to set the improvement processes. –Our Thesis –GQCM is not in contrast with CMMI and can be integrated in organization as a tool to support CMMI adoption

16. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 16 The timeline OCTOBER DECEMBER ECHOGRID/EUChinagrid Conference 24-25 April Beijing MAY FEBRUARY OGF 20/EGEE UF 7-11 May Manchester Belief Conference 25-28 June Rio de Janeiro OGF 21 15-19 October Seattle QUALIPSOConference 16-17 January 2008 Rome Now EGEE ’07 1-5 October Budapest EELA 3° Conference 3-5 December Catania AHM 23-25 May Madison

17. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 17 GQCM: publications 2 papers submitted2 papers submitted 2 papers accepted2 papers accepted When/WhereWhen/Where EELA third conferenceEELA third conference 3-5 December 20073-5 December 2007 Catania/ItalyCatania/Italy QUALIPSO first conferenceQUALIPSO first conference 16-17 January 200816-17 January 2008 Rome/ItalyRome/Italy The first quality oriented conference!The first quality oriented conference!

18. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 18 GQCM: feedbacks The model has got good feedbacks and acknowledgementsThe model has got good feedbacks and acknowledgements People are asking for more informationPeople are asking for more information Some issues about the name of the model (solved)Some issues about the name of the model (solved) One issue about ISO9000 compliance (outside the scope?)One issue about ISO9000 compliance (outside the scope?) Requests for metrics about the process  outside the scope?Requests for metrics about the process  outside the scope?

19. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 19 GQCM: future and open issues Within ETICS 2Within ETICS 2 ImplementationImplementation ImprovementsImprovements Pilot projects to test itPilot projects to test it ISO standardisationISO standardisation Open issueOpen issue First implementation?First implementation?

20. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 20 What we have now? Programming languages  Java/PhytonProgramming languages  Java/Phyton –Others? JDepend can calculateJDepend can calculate –Number of classes indicator of the extensibility of the package –Afferent Couplings indicator of the package’s responsibility –Efferent Couplings … independence –Abstractness 0/1  package concrete – abstract –Instability 0/1 –Distance  indicator of the package bilance between abstractness and stability PMD scans Java source code and looks for potential problems like:PMD scans Java source code and looks for potential problems like: –Possible bugs - empty try/catch/finally/switch statements –Dead code - unused local variables, parameters and private methods –Suboptimal code - wasteful String/StringBuffer usage –Overcomplicated expressions - unnecessary if statements, for loops that could be while loops –Duplicate code - copied/pasted code means copied/pasted bugs Cobertura/EmmaCobertura/Emma –Coverage: class, method, lines, block FindBugs reportsFindBugs reports –high/medium/low priority bugs for java JUnitJUnit –Java unit tests PyUnitPyUnit –Python unit tests PyCyclomaticComplexityPyCyclomaticComplexity –Python Cyclomatic Complexity

21. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 21 A possible first “implementation”? We can implement partially three evaluation modulesWe can implement partially three evaluation modules –EM: Static Analysis (Only one measure..) –WMC: looks at the complexity of classes –EM: Coding Style –Only some measurement but we can have a realistic value from it –EM: Structural Testing –it is expected that a small part of the code contain a relative large number of the defects in the code. Some metrics are used to identify potential problematic classes. –Use other test cases than already used for statement coverage. –The first implementation will aim at the programming languages Java and Python. […]

22. INFSOM-RI-026753 ETICS All Hands Meeting - Geneva 21-23/11/2007 22 Q&A http://www.eu-etics.org