1. Simultaneous use of process-oriented software quality approaches
Supervisors:
Katalin Balla, Budapest University of Technology and Economics
Jos Trienekens, Rob Kusters, Eindhoven University of Technology
Simultaneous use of process-oriented software quality approaches
Zádor Dániel Kelemen, Budapest University of Technology and Economics

2. Contents Terminology Multi-model process improvement
Background (available solutions)
Research focus, questions and objectives
Structure of process-oriented quality approaches
Mapping quality approach elements
Example: tailoring a process from multiple process-
oriented quality approaches

3. Terminology “If you wish to converse with me, define your terms.”
Voltaire

4. Terms used
Approach
a standard, model or method is an approach (gives a view of a solution) to one or more problem
Quality Approach
an approach connected to quality
Software Quality Approach
a quality approach used in software industry

5. Terms used Process-Oriented (Software) Quality Approach
a (software) quality approach which focuses on processes (POSQA)
Simultaneous Use of Process-Oriented Software Quality Approaches
~ subset of multi-model process improvement
process improvement based on multiple software quality approaches

6. Multi-model process improvement
Situation, benefits, problems

7. Multi-model environment
Several SPI methods, standards and models are available
Companies use these approaches simultaneously
Some examples:
CMMI, ISO/IEC/IEEE 12207:2008, ISO/IEC 15939:2002, ISO 9001:2008, ISO 9004:2000,ISO 90003:2000, IEEE 1028:2008, Agile methods, ITIL, SPICE

8. Focusing on the same process from different perspectives
ISO 12207
Peer Review
CMMI
Organizational peer review needs
Organizational Peer Review Process
Peer Review
IEEE 1028
Peer Review

9. Benefits of Using Multiple Approaches
Quality approaches are often focusing
on the same problems from different perspectives
A multi-model solution therefore
coulde be more complete
uses the best practices
supports combined appraisals
has higher ROI …

10. Problems of Using Multiple Approaches
Approaches have different
structure
terminology
level of detail
point of view on quality
Difficult to tailor organizational processes from multiple quality approaches

11. Backgorund, Background (available solutions)
mappings, standardization, combined
models, formal methods, further solutions

12. Mappings of Quality Approaches
Wide literature on comparing CMMI to other approaches
ISO 9001 (Mutafelija)
TSP
SPICE (T. Route et al.)
Agile methods
SixSigma
PMBOK
etc…

13. Combined models Available combined models: CITIL Appraisal Assistant
Developed by Wibas
CMMI-ITIL
Appraisal Assistant
Developed at Griffith University
CMMI – SPICE
Combined appraisal of CMMI and SPICE

14. Standardization
International Organization for Standardization (ISO) harmonizes its most widely used standards
Eg. ISO 9001, and has been developed with the same structure and terminology
IEEE 1028:2008 and IEEE 12207:2008 are also harmonized
performing 1028 satisfies requirements of 12207

15. Formal methods Other researchers create more formal solutions
using natural language processing know-how for processing standards and models
or building ontologies.
Eg. Malzahn briefly presents an ontology based solution for multi-model process improvement

16. Further multi-model solutions
QMIM (Quality through Managed Improvement and Measurement)
shows how the quality standards and models are connected to quality objects
PrIME (Process Improvement in Multimodel Environments)
A SEI project for maximizing the ROI

17. Characteristics of available solutions 1
Solution type
Solution example
Main strengths
Main weaknesses
Standardization
ISO – – 14401
Common terminology, structure,
Provides guidance
Must follow the structure and terminology of iso 9001
IEEE 1028 – 12207
Common terminology
1028 can be used only to satisfy 2 requirements of 12207
Mapping
ISO9001-CMMI
Concrete (maps each requirement, terminology, gives guideline)
Different terminology, structure, content

18. Characteristics of available solutions 2
Solution type
Solution Example
Main Strengths
Main Weaknesses
Com-bined model
CITIL
Combined appraisal,
terminology, structure, content
Applicable only for two approaches (CMMI and ITIL)
SPICE-CMMI, Appraisal Assistant
combined appraisal, terminology,
Structure, content
Could be applicable only for CMMI and SPICE structured approaches
Formal method
Natural language processing,
precise, formal,
easy to process the model with computers
still not available for wider use, Often needs special mathematical knowledge
ontologies
fuzzy logic

19. Characteristics of available solutions 3
Solution type
Solution Example
Main Strengths
Main Weaknesses
Multi-model
QMIM
gives a starting point for multi-model-oriented companies
TBD
PrIME
increase ROI
process-based approach (with possibility to represent it graphically)
A continuation of QMIM, identifies main elements and level of QAS

20. Summary of available solutions
Significant effort has been spent on multi-model problem
Mappings are useful in using two different models or standards
Combined appraisal models reduce the cost of the appraisals
Formal methods could serve good solutions in the future
Process modelling facilitate the better understanding and maintenance of processes
QMIM framework helps companies to start their process improvement
Multi-model process improvement is still an issue
when companies want to use more than two
or still not mapped approaches

21. Our solution
Research focus, questions and objectives

22. Research focus Our goal: to create a process based approach for
supporting the use of multiple process-oriented
(software) quality approaches
Excluding approaches in which the main focus is not
on the process but on other quality objects, such as
the product or resource.
Using this approach, software companies can tailor their
own quality system from different process-oriented
software quality approaches

23. Research Questions
Is a process model based approach (PBA) usable to support the use of multiple process-oriented software quality approaches?
Does the outcome of the PBA provide sufficient information to unify selected parts of different process-oriented software quality approaches?
Objective: To determine the completeness of the material developed using the process based approach.
Is the outcome of PBA traceable?
To identify the main element types to be extracted from quality approaches
To determine the level of traceability of derived elements back to the selected quality approaches
(eg. to verify that a process can be built on the peer review material developed using our process based approach)

24. Research Questions Is the outcome of PBA adaptable and expandable?
To identify possible reactions to the changes in and number of quality approaches.
 Is the outcome of PBA usable in practice?
To use the process based approach and its outcomes in practice.
Does the outcome of the PBA will use a harmonized terminology and content?
To provide a unified terminology and content for the selected parts of the used process oriented quality approaches.
Does the outcome of the PBA will support combined appraisals?

25. The structure of quality approaches
Searching for convertible quality approach elements…

26. The structure of quality approaches
In order to quickly create process models, would be useful to know, what kind of elements could be converted directly from textual information to graphical representation.
Aspects considered important:
to analyse the structure of widespread approaches,
to describe the structure of approaches analysed in a very simple and understandable format. ->
UML class seems suitable for describing structures

27. Selected approaches /based on surveys made at Hungarian companies/
The most used and “mandatory” quality approach:
ISO 9001:2008 Quality management systems – requirements
Besides ISO 9001:2000, most used approaches are:
CMMI & SPICE
Further, well known approaches at Hungarian companies:
ISO 9004:2000 Quality management systems – Guidelines for performance improvements
ISO/IEC 90003:2000 Software Engineering – Guidelines for the application of ISO9001:2000 to computer software
ISO/IEC – “Information technology - Software measurement process”
IEEE/ISO/IEC – “Information technology - Software life cycle process”
A peer review specific approach:
IEEE 1028:2008

28. The structure of ISO 9001:2008 Elements: Chapter Subchapter
Sub-subchapter
Requirement in (simple) sentences
ISO 9004 and have similar structure

29. The structure of CMMI-DEV v1.2
Elements: Process area, Specific Goal, Specific Practice, Typical Work Product, Subpractice, Generic Goal, Generic Practice, Generic Practice Elaboration

30. The structure of ISO-IEC 12207-2008 Information Technology – Software Lifecycle Processes
Elements: Chapter, Process category, Process Description, Activity, Task, Purpose, Outcome, …

31. The structure of ISO-IEC 15939-2002 Software engineering
The structure of ISO-IEC Software engineering. Software measurement process
Elements:
Chapter / process activity
Subchapter /sub-activity
Sub-sub-activity
Activity description

32. The structure of IEEE 1028:2008

33. Mapping quality approach elements
mapping to process elements and process model elements

34. Some elements of process-oriented quality approaches
Chapter
Subchapter
Sub-subchapter
Requirement (in /simple/ sentences)
Guideline
Process
Process Description
Activity
Task
Option
Condition
Documentation requirement
Responsibility
Chapter / process activity
Subchapter /sub-activity
Sub-sub-activity
Activity description
Process area
Specific Goal
Specific Practice
Typical Work Product
Sub Practice
Generic Goal
Generic Practice
Generic Practice Elaboration
Process
Mission Statement
Goal
Activity
Benefit
Critical Success Factor
Key Performance Indicator
Further Specific Informative Components
Method
Practice
Value
Feature
Similar elements are marked with same color

35. Process vs. Quality Approach Elements
QAP Elements:
Chapter
Subchapter
Sub-subchapter
Requirement (in /simple/ sentences)
Guideline
Process
Process Description
Activity
Task
Option
Condition
Documentation requirement
Responsibility
Chapter / process activity
Subchapter /sub-activity
Sub-sub-activity
Activity description
Process area
Specific Goal
Specific Practice
Typical Work Product
Sub Practice
Generic Goal
Generic Practice
Generic Practice Elaboration
Mission Statement
Goal
Benefit
Critical Success Factor
Key Performance Indicator
Further Specific Informative Components
Method
Practice
Value
Feature
Process Elements:
Purpose
Inputs
Entry criteria
Activities
Roles
Measures
Verification steps
Outputs
Exit criteria

36. Mapping quality approaches to processes
Knowing the structure of quality approaches, their
elements can be mapped to
Process elements
Process modeling language elements
BPMN elements
Elements of UML Activity diagrams
EPC elements
… other process modeling language elements

37. Eg: mapping CMMI elements 1
REQ. level
Process elements
BPMN elements
Process area
Calculated
Process
Specific Goal
REQUIRED (Calculated) -
Specific Practice
EXPECTED
(Sub)process
Typical Work Product
INFORMATIVE
Input & Output
Artefact/Data Object

38. Eg: mapping CMMI elements 2
REQ. level
Process elements
BPMN elements
Sub Practice
INFORMATIVE
Task
Generic Goal
REQUIRED (Calculated) -
Generic Practice
EXPECTED
(Sub)process
Generic Practice Elaboration

39. Observations 7 quality approaches (QAP) were analysed
Similar elements are present in the QAPs and processes
Process, process description, activity, process activity, activity description and task are proportional to the activity element of processes.
Documentation requirements and typical work products are proportional are similar to inputs and outputs.
...
Element which have no similarities to process elements are also present.
Such elements are eg. benefits, critical success factors, features or key performance indicators in ITIL.
Difficult to convert ISO chapters, sentences...

40. The peer review process
An example of tailoring a process from
multiple process-oriented quality approaches

41. Quality approaches used
CMMI, SPICE
IEEE 1028:2008
ISO 12207
Fagan’s Inspections
Volere Requirements Review Checklist
Templates of Karl E. Wiegers
Templates of the team of R. Kusters
Personal Software Process template for code review
Articles connected to reviews/inspections(mainly from Wiegers)
Software Testing Books

42. General steps of building the graphical notation
2nd case:
1st case:
1b case:
Textual POQAs
Textual POQAs
Textual POQAs
Textual General Process
Developed and practically performed process
Textual
Graphical
Graphical General Process
Textual General Process
Graphical General Process
Tailored Organizational Textual and Graphical Process
Tailored Organizational Textual and Graphical Process
Tailored Organizational Textual and Graphical Process

43. Peer Review specific steps of building the graphical notation (1st case)
Organizational level description (CMMI, SPICE, ISO12207)
Process, procedure level description (IEEE 1028, …)
Practical level description (Temlates of Karl E. Wiegers, Kusters, …)
Textual description
Graphical description
Tailored Textual description for Org. needs
Tailored Graphical description for Org. needs

44. Quality elements connected to peer reviews on a high process level
One instance of a peer review (session)
CMMI GGs and GPs are also included, represented
Complete CMMI and/or ISO/IEC/IEEE 12207
compatibility can be achieved!

45. Mapped elements on a lower process level…

46. Summary Situation of multi-model process improvement
A review of available solutions
Research Questions
Structure of process-oriented quality approaches
emphasis on convertible elements
An example of a tailored process from multiple
process-oriented quality approaches
the peer review process

47. Acknowledgement This project has been supported by
SQI Hungarian Software Quality Consulting Institute Ltd.
BME (IT)2 (BME Innovation and Knowledge Centre of Information Technology) within the frame of Peter Pázmány Programme, National Office for Research and Technology, Hungary.
BUTE project “Szoftverminőség-biztosítási szolgáltatás-csomag kidolgozása a nyílt dokumentumformátum alkalmazásaira”

48. Thanks for your attention!
We are pleased to collaborate with you
from the industry to carry out a case study on using multiple quality approaches
from the academic field to cooperate with similar research projects
Send mail to: Zádor Dániel Kelemen
Project webpage: