GQM – an example from Fjellanger - Widerøe Hans J. Lied Tor Stålhane IDI / NTNU

Environment Medium sized company making tailor-made software systems Fixed price projects with little room for rework caused by introduction of errors Two development projects, same project team and process model. – Project 1: standard with a given requirements specification – Project 2: prototyping without requirements specification

The projects Project 1 had – structure given from the start – customer defined quality assurance requirements – requirements specification from the customer Project 2 – depended on prototyping. – started with an idea from the customer – the requirements specification was a co- production between customer and developers.

Methods We used a combination of three methods: GQM, to decide what and how to measure Pareto with a robustness analysis, to analyse and assess collected data RCA: brainstorming and Ishikawa diagrams to identify root causes

GQM method – 1 Analysefor the purpose of with respect toas seen fromin the context of Focus Q1: Q2: Environment Qa: Qb: Qc: What do we believe Q1: Q2: How does the environment influence focus Increased Qa values will reduce Q1 Low Qb values will increase Q2 Comments Interesting info that surfaces during the GQM process but is not directly relevant for the current GQM goal. GQM abstraction sheet without the lower part

GQM method – 2 The GQM abstraction sheet was used to: manage and structure the brainstorming sessions identify problem areas for the RCA benefit from its strong focus on developer participation to get a set of basic root causes to analyse

GQM method – 3 The GQM method helps us to obtain: reliable data. The developers are more interested in obtaining correct data if they felt that they had ownership to the goal and purpose of the data collection more interest in the results. One of the factors that can kill an SPI initiative is lack of interest. By involving the developers from day one, we hoped to create interest and keep the improvement program alive.

GQM - process Two hour workshop to introduce the method Establishing the GQM goal: – Analyse the reported failures in order to understand causes for failures seen from the developers in the X project. Produced a GQM abstraction sheet, which was converted to a data collection form

GQM - Data collection form

Pareto – 1 The 20/80 distribution, known as Pareto’s Law, is applicable in software development Pareto diagrams is a way of visualising the main problem areas We used Pareto analysis on our collection of data to identify areas of improvement

Pareto – 2 Pareto identified the following main problem areas: Project1 – standard project: Incorrect use of language features (3) Incorrect use of library components (4) Wrong code logic (14) Project2 – prototyping : Incomplete or insufficiently detailed req. spec. (13) Errors in HW-SW communication (15) Wrong code logic (14)

RCA - Brainstorm We presented the development team with the results from the Pareto analysis and introduced them to RCA. We made an Ishikawa diagram, using the categories from the data collection form as a starting point. The strong point of a brain-storming session is that it maximises the use of available knowledge

The RCA diagram

RCA - Results Improvement actions identified from this session: The need for sharing competence internally Checklist on Intranet Common routines/solutions for standard tasks (best practice experience) Common dynamic document templates Start the planning, design and development of an experience database

Conclusions Individual experience for the team members Measurements gave knowledge of how things really are in the company Common understanding of what we learned from this knowledge Explicit shared knowledge about the process that the company can reuse