1. Lecture 17 “Does every inspection need a meeting?” FOR0383 Software Quality Assurance 9/19/20151Dr Andy Brooks Source: Votta, L G (1993) Does Every Inspection Need a Meeting? Proceedings of the first ACM symposium on the Foundations of Sofware Engineering, SIGSOFT’93, pp 107-114. ©ACM n = 4

2. 19.9.2015Dr Andy Brooks2 The survey data ReasonAuthors & Managers (29) Synergy79% (23) Education62% (18) Schedule48% (14) Competition34% (10) Requirement28% (8) Qu: Why do you think inspection meetings are held? Responses coded from answers. All other reasons given occurred with a frequency of less than 20%. Always provide data about what a % means.

3. 19.9.2015Dr Andy Brooks3 Top five reasons why meetings are held. 1.Synergy The idea is that working together as team produces a better performance than the sum of the performance of individual team members. Synergy has been clearly demonstrated in many areas. Planting a tree as an individual is difficult, but not if you work in pairs. However, synergy has not been clearly demonstrated for inspection meetings. “If this effect occurred in the software inspection process, we would see many inspection faults found only by holding a meeting.” samvirkni

4. 19.9.2015Dr Andy Brooks4 2.Education The idea is that new employees learn from experienced reviewers how the software development process works, and what makes a good reviewer. However, this might be “the blind leading the blind”. Sending employees on proper training courses about inspections might well be more efficient and effective. Top five reasons why meetings are held.

5. 19.9.2015Dr Andy Brooks5 3.Schedule deadlines People have a planned meeting to work towards. Scheduling events such as inspection meetings helps with the time management of a project. Top five reasons why meetings are held.

6. 19.9.2015Dr Andy Brooks6 4.Competition Employees make their contribution to defect finding in public and so can earn the respect of their peer group. Employees try to improve their inspection performance. However, managers can easily destroy teamwork by rewarding individual performance. “Why should I devote a lot of time to inspecting, if it is always Joe who gets the salary bonus?”. Top five reasons why meetings are held.

7. 19.9.2015Dr Andy Brooks7 5.Requirement Holding a meeting to collect individual reviewer defects is required by the inspection process. Having a well-defined process helps with the management of a project. Top five reasons why meetings are held.

8. Figure 1. Typical Time Line for Inspection Process ©ACM 19.9.2015Dr Andy Brooks8 “Typical values in a large software development for a feature of 600 NCSL are 10, 10, and 10 working days for T inspection, T collection, and T repair, respectively.” L G Votta

9. 19.9.2015Dr Andy Brooks9 The development interval The longer money is borrowed, the more difficult it is to realise a profit. You get less and less responsive to customer needs and marketing conditions as you take longer and longer to develop something. Waiting for the collection meeting (T collection ) to happen can be very costly. –hazard cost of being late to market –carrying cost if the author is not working on the product –rework cost if the author continues working on the product only to have this work invalidated by the results of the inspection meeting T inspection T collection T repair T inspection T collection T repair T inspection T collection T repair..................

10. 19.9.2015Dr Andy Brooks10 When can n people have a meeting? Suppose –each day comprises 4 two-hour slots –m is the average number of meetings per day –p is the probability a two hour slot is occupied by a meeting (= m/4) –q is the probability that a two hour slot is unoccupied (= 1-p) –assume p is the same for each person and independent The probability of any slot being available for n people is: The average number of days that elapse before the meeting occurs is: n = 5

11. 19.9.2015Dr Andy Brooks11 © ACM Keep the number of meetings to a minimum and the number attending small. Exception: architectural design meetings for a large system. Average number of meetings per day.

12. 19.9.201512 Is there enough synergy? Each point represents the number of faults undetected in individual preparation divided by the total number recorded at an inspection meeting. 4% average © ACM Always provide information about what an error bar means. actual measurements

13. 19.9.2015Dr Andy Brooks13 The severity of a defect is important. Even if the inspection meeting yields only another 4%, the meeting will be invaluable if you find a showstopper. –A showstopper is a defect that will cause a system crash in the field (such as null pointer dereference). Holding inspection meetings is, however, unnecessary if only additional minor defects are found. –This is especially true if static analysis tools can detect all the minor defects (violations of coding conventions, for example). Inspection data needs to be recorded and analysed to help make decisions about the value of holding an inspection meeting. Common sense can also be applied: – “...this part of the requirements, design or code is so important that we should have an inspection meeting.”

14. 19.9.2015Dr Andy Brooks14 False positives and lost defects. During individual preparation, inspectors can soemtimes report issues that are not defects (false positives). –An inspection meeting usually eliminates most of the false positives. During the inspection meeting, some issues found in individual preparation or at the meeting do not get recorded (lost defects). –Lost defects typically represent a small percentage of the total. (L G Votta quotes a figure of 4%.)... the benefit from synergy was measured at 4%.

15. 19.9.2015Dr Andy Brooks15 What are the alternatives to holding an inspection meeting? Deposition Meetings The moderator and author meet with single reviewers. (3 person meetings.) There is no reading activity: a deposition meeting collects the defects found from the reviewer. The moderator acts as the recorder. The moderator and author have to attend more meetings. There is no meeting schedule delay as depositions can be taken in a day.

16. 19.9.2015Dr Andy Brooks16 What are the alternatives to holding an inspection meeting? Direct deposition hand over The moderator collects all the individual logs. The moderator can simply hand over the logs to the author. Or, the moderator creates a master defect report which is then handed over to the author.

17. The author and reviewers never physically meet and can even be located in different countries. Reviewers can use e-mail to report defects. Reviewers can use general or special purpose groupware to report defects. 19.9.2015Dr Andy Brooks17 What are the alternatives to holding an inspection meeting? Correspondence IBIS - Internet Based Inspection System http://sourceforge.net/projects/ibis/ Adobe Acrobat 9 http://www.adobe.com/products/acrobat/

18. 19.9.2015Dr Andy Brooks18 Inspection data needs to be recorded and analysed to help make decisions about the value of holding an inspection meeting. What is the synergy rate? Are showstoppers being found? What is the lost defect rate? How big is the meeting schedule delay? Common sense can be applied: “...this part of the requirements, design or code is so important that we should have an inspection meeting.” almenn skynsemi