1. SE 501 Software Development Processes Dr. Basit Qureshi College of Computer Science and Information Systems Prince Sultan University Lecture for Week 9

2. Contents The Personal Software Process PSP1.1 Overview of PSP2 Performance of PSP SE 501 Dr. Basit Qureshi: Lecture for Week 9

3. Bibliography Humphrey, Watts (1995). A disciple for Software Engineering. The Personal Software Process. http://www.sei.cmu.edu/library/abstracts/reports/00tr022.cfm http://www.sei.cmu.edu/library/abstracts/reports/00tr022.cfm Ferguson, P., Humphrey, W., Khajenoori, S., Macke, S., and Matvya, A. "Introducing the Personal Software Process: Three Industry Case Studies," Computer, pp. 24-31, May 1997. Software Engineering Measurement and Analysis, Process Maturity Profile: Software CMM, CBA IPI and SPA Appraisal Results, 2003 Mid-Year Update, Software Engineering Institute, Carnegie Mellon University September 2003, http://www.sei.cmu.edu/sema/pdf/SW-CMM/2003sepSwCMM.pdf http://www.sei.cmu.edu/sema/pdf/SW-CMM/2003sepSwCMM.pdf SE 501 Dr. Basit Qureshi: Lecture for Week 9

4. THE PSP1.1 SE 501 Dr. Basit Qureshi: Lecture for Week 9

5. PSP1.1 The objectives of PSP1.1 are to introduce and practice methods for – making resource and schedule plans – tracking your performance against these plans – judging likely project completion dates What’s new? – Project Plan Summary form – Task Planning Template – Schedule Planning Template What didn’t change – Time Logs – Defect Logs SE 501 Dr. Basit Qureshi: Lecture for Week 9

6. © 2006 by Carnegie Mellon University January 2006 PSP I - Using PSP1.1 - 6 PSP 1.1 New Process Elements There are two new process elements. task planning template schedule planning template These are typically used for projects that take several days or weeks. They are not required for the PSP exercises. The project plan summary has been expanded to include basic process statistics.

7. © 2006 by Carnegie Mellon University January 2006 PSP I - Using PSP1.1 - 7 PSP1.1 Project Plan Summary The summary section has been expanded to include Planned time Actual time CPI %Reused %New Reusable These values are calculated automatically.

8. © 2006 by Carnegie Mellon University January 2006 PSP I - Using PSP1.1 - 8 Measurement Definitions -1 Planned Time Plan: planned development time for this project To-Date: sum of planned development time for all completed projects + this project Actual Time Actual: actual development time for this project To-Date: sum of actual development time for all completed projects + this project Cost Performance Index (CPI) To-Date planned time / To-Date actual time

9. © 2006 by Carnegie Mellon University January 2006 PSP I - Using PSP1.1 - 9 Measurement Definitions -2 %Reused: percentage of reused size for this project Reused / Total * 100 %New Reusable: percentage of new reusable code developed for this project New Reusable / Added and Modified * 100

10. © 2006 by Carnegie Mellon University January 2006 PSP I - Using PSP1.1 - 10 Task and Schedule Plans Task and schedule plans are used to estimate when work will be completed. Tasks are entered on the task plan. each task in order planned hours actual hours date task completed Schedule hours are entered on the schedule plan. planned hours for each schedule week actual hours for each schedule week

11. PSP1.1 SE 501 Dr. Basit Qureshi: Lecture for Week 9

12. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 12 Organizing Proxy Data -1 To make an estimate break the planned product into parts relate these planned parts to parts that you have already built use the size of the previously-built parts to estimate the sizes of the new parts To do this, you need size ranges for the types of parts that you typically develop. For each product type, you also need size ranges to help you to judge the sizes of the new parts.

13. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 13 Organizing Proxy Data -2 To determine the size ranges, start with the part data. Assume that you have the following data. class A, three items (or methods), 39 total LOC class B, five items, 127 total LOC class C, two items, 64 total LOC class D, three items, 28 total LOC class E, one item, 12 LOC class F, two items, 21 total LOC The LOC per item is 13, 25.4, 32, 9.333, 12, 10.5. The objective is define size ranges that approximate our intuitive feel for size.

14. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 14 Organizing Proxy Data -3 To produce the size ranges, sort the data as follows. The sorted LOC per item data: 9.333, 10.5, 12, 13, 25.4, 32. Arrange these data as follows. Pick the smallest item as very small: VS = 9.333. Select the largest item as very large: VL = 32. Pick the middle item as medium: M = 12 or 13. For the large and small ranges, pick the midpoints between M and VS and M and VL: 10.9, and 22.25. While these may be useful ranges, they are probably not stable. That is, additional data points will likely result in substantial size-range adjustments.

15. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 15 Intuitive Size Ranges -1 In judging size, our intuition is generally based on a normal distribution. That is, we think of something as of average size if most such items are about that same size. We consider something to be very large if it is larger than almost all items in its category. When items are distributed this way, it is called a normal distribution. With normally distributed data, the ranges should remain reasonably stable with the addition of new data points.

16. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 16 Intuitive Size Ranges -2 A normal distribution

17. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 17 Intuitive Size Ranges -3 With a large volume of data, you could calculate the mean and standard deviation of that data. For the size ranges Medium would be the mean value. Large would be mean plus one standard deviation. Small would be mean minus one standard deviation. Very large would be mean plus two standard deviations. Very small would be mean minus two standard deviations. This method would provide suitably intuitive size ranges if the data were normally distributed.

18. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 18 The Distribution of Size Data Program size data are not normally distributed. many small values a few large values no negative values With size data, the mean minus one or two standard deviations often gives negative size values. The common strategy for dealing with such distributions is to treat it as a log-normal distribution.

19. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 19 A Log-Normal Distribution

20. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 20 The Log-Normal Distribution To normalize size data, do the following: 1.Take the natural logarithm of the data. 2.Determine the mean and standard deviation of the log data. 3.Calculate the average, large, very large, small, and very small values for the log data. 4.Take the inverse log of the ranges to obtain the range size values. This procedure will generally produce useful size ranges.

21. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 21 Organizing Proxy Data -4 A mathematically precise way to determine the proxy size ranges is described in the text (pages 78-79). This simple way to determine these size ranges will work when you have lots of data. Otherwise, it can cause underestimates. Comparative estimating ranges

22. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 22 Estimating with Limited Data -1 Even after using PSP for many projects, you will have to make estimates with limited data when you work in a new environment use new tools or languages change your process do unfamiliar tasks Since estimates made with data are more accurate than guesses, use data whenever you can. Use the data carefully since improper use can lead to serious errors.

23. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 23 Estimating with Limited Data -2 Depending on the quality of your data, select one of the four PROBE estimating methods. Method Aregression with estimated proxy size Bregression with plan added and modified size Cthe averaging method Dengineering judgment To use regression method A or B, you need a reasonable amount of historical data data that correlate reasonable β 0 and β 1 parameter values

24. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 24 Method A (Regression): Estimated Proxy Size Method A uses the relationship between estimated proxy size (E) and actual added and modified size development time The criteria for using this method are three or more data points that correlate (R 2 > 0.5) reasonable regression parameters (table 6.6 on pg. 96) completion of at least three exercises with PSP1 or higher

25. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 25 Estimating Accuracy Planning is a skill that must be developed. The PSP helps to build planning skills. Even simple plans are subject to error. -unforeseen events -unexpected complications -better design ideas -just plain mistakes The best strategy is to plan in detail. Identify the recognized tasks. Make estimates based on similar experiences. Make judgments on the rest.

26. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 26 Combining Estimates To combine multiple estimates made by a single developer add the estimates for the separate parts make one linear regression calculation calculate one set of prediction intervals Multiple developers can combine independently-made estimates by making separate linear regression projections adding the projected sizes and times adding the squares of the individual ranges and taking the square root to get the prediction interval

27. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 27 Estimating Error: Example When estimating in parts, the total error will be less than the sum of the part errors. Errors tend to balance out. This assumes no common bias. For a 1000-hour job with estimating accuracy of ± 50%, the estimate range is from 500 to 1500 hours. If the estimate is independently made in 25 parts, each with 50% error, the total would be 1000 hours, as before estimate range would be from 900 to 1100 hours

28. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 28 Combining Individual Errors To combine independently-made estimates Add the estimated values. Combine the variances (squares) of the errors. With 25 estimates for a 1000-hour job Each estimate averages 40 hours. The standard deviation is 50%, or 20 hours. The variance for each estimate is 400 hours. The variances add up to 10,000 hours. The combined standard deviation is the square root of the sum of the variances, or 100 hours. The estimate range is 900 to 1100 hours.

29. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 29 Class Exercise -1 Start with three estimates. A = 45 hours, + or - 10 B = 18 hours, + or - 5 C = 85 hours, + or - 25 What is the combined estimate?

30. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 30 Class Exercise -2 Start with three estimates. A = 45 hours, + or - 10 B = 18 hours, + or - 5 C = 85 hours, + or - 25 What is the combined estimate? total = 45 + 18 + 85 = 148 hours What is the combined estimate range?

31. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 31 Class Exercise -3 Start with three estimates. A = 45 hours, + or - 10 B = 18 hours, + or - 5 C = 85 hours, + or - 25 What is the combined estimate? total = 45 + 18 + 85 = 148 hours What is the combined estimate range? variance = 100 + 25 + 625 = 750 range = square root of variance = 27.4 hours What is the combined UPI and LPI?

32. © 2006 by Carnegie Mellon University January 2006 PSP I - Estimating with PROBE II - 32 Class Exercise -4 Start with three estimates. A = 45 hours, + or - 10 B = 18 hours, + or - 5 C = 85 hours, + or - 25 What is the combined estimate? total = 45 + 18 + 85 = 148 hours What is the combined estimate range? variance = 100 + 25 + 625 = 750 range = square root of variance = 27.4 hours What is the combined UPI and LPI? UPI = 148 + 27.4 = 175.4 hours LPI = 148 – 27.4 = 120.6 hours

33. PSP1.1 Summary Task and schedule planning templates are not required for small programs, but are helpful for longer tasks. PSP1.1 gives you insight into your historical performance (CPI-Cost Performance Index). PSP1.1 allows you to make effective proxy based estimates. Estimates can be used to plan tasks (e.g. LOC) and resources (e.g. time) SE 501 Dr. Basit Qureshi: Lecture for Week 9

34. THE PSP2 SE 501 Dr. Basit Qureshi: Lecture for Week 9

35. We have done PSP0->PSP1.1. What's Next? PSP Part I: Planning – Introduction to PSP – Size measurement – Estimating with PROBE I – Estimating with PROBE II – Using PSP data PSP Part II: Quality – Software quality – State-machine design and verification – Design – Design verification – Using the TSP PSP0, PSP0.1 PSP1.0, PSP1.1 PSP2, PSP2.1, PSP3 SE 501 Dr. Basit Qureshi: Lecture for Week 9

36. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 36 What is Quality? Basic definition: meeting the users’ needs needs, not wants true functional needs are often unknowable There is a hierarchy of needs. Do the required tasks. Meet performance requirements. Be usable and convenient. Be economical and timely. Be dependable and reliable.

37. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 37 The PSP Quality Focus -1 To be useful, software must install quickly and easily run consistently properly handle normal and abnormal cases not do destructive or unexpected things be essentially bug-free Defects are not important to users, as long as they do not affect operations cause inconvenience cost time or money cause loss of confidence in the program’s results

38. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 38 The PSP Quality Focus -2 The defect content of software products must be managed before more important quality issues can be addressed. Low defect content is an essential prerequisite to a quality software process. Since low defect content can best be achieved where the defects are injected, engineers should remove their own defects determine the causes of their defects learn to prevent those defects

39. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 39 The Economics of Quality Software is the only modern technology that relies on testing to manage quality. With common quality practices, software groups typically spend 50+% of the schedule in test devote more than half their resources to fixing defects cannot predict when they will finish deliver poor-quality and over-cost products To manage cost and schedule, you must manage quality. To get a quality product out of test, you must put a quality product into test.

40. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 40 Testing Alone is Ineffective Safe and secure region = tested (shaded) Unsafe and insecure region = untested (unshaded)

41. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 41 Removing Defects in Test When performing a task thousands of times, economics would suggest that you use the most efficient methods. A 50,000 LOC system with traditional development methods would have 25+ defects/KLOC at test entry - 1250 defects take 12,500+ programmer hours to test be late and over budget At the typical rate of 10+ hours per defect, this is 6 programmer years.

42. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 42 Quality and Productivity Managed quality = 60% increased team productivity

43. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 43 Defect-removal Methods -1 The principal ways to find and fix defects are by compiling unit testing integration and system testing team inspections personal reviews Since you will likely have to remove lots of defects, you should use the most efficient methods.

44. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 44 Source: Xerox Defect-removal Times

45. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 45 Defect-removal Methods -2 In a personal review you privately review your product your objective is to find and fix defects before test Reviews are most effective when they are structured and measured. Use reviews for requirements, designs, code, and everything else that you develop. Also continue to use inspections, compiling, and testing.

46. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 46 Defect-Removal Filters

47. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 47 0 1 2 3 4 5 6 7 8 9 10 0 20304050 Compile Defects Test Defects Compile vs. Test Defects - Student 20

48. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 48 Defect Prevention Defect prevention is important because it is always expensive to find defects if the defects can be prevented, you can avoid the costs of finding and fixing them defect prevention analysis costs are incurred once, but the savings apply to every project Defect prevention should follow an orderly strategy and a defined process. For the PSP, defect prevention actions include gathering defect data, improving design methods, and prototyping.

49. © 2006 by Carnegie Mellon UniversityJanuary 2006PSP II - Software Quality - 49 Messages to Remember Improve product quality and accelerate development work by doing reviews and inspections to remove defects before test using testing to check product quality, not to remove volumes of defects Design and code reviews improve the quality of your programs save development time To do effective reviews, you must establish review goals follow a disciplined review process measure and improve your review practices

50. PERFORMANCE OF PSP SE 501 Dr. Basit Qureshi: Lecture for Week 9

51. PSP Courses The “full” PSP course – industry course: 3 intensive weeks (60 hours per week) – academic course: 12 to 15 weeks (3 hours per week) – includes 10 programming assignments and 5 reports – course results similar in industry and academia – uses A Discipline for Software Engineering [Humphrey 1995] A “introductory” PSP course – does not teach the complete PSP – is often offered as part of a project management course – may start in the 1 st year of a computing degree – uses Introduction to the Personal Software Process [Humphrey 1997]

52. SE 501 Dr. Basit Qureshi: Lecture for Week 9 PSP Structure 1 PSP0 Current process Basic measures PSP1 Size estimating Test report PSP2 Code reviews Design reviews PSP3 Cyclic development Team Software Process Requirements Configuration management PSP2.1 Design templates PSP1.1 Task planning Schedule planning PSP0.1 Coding standard Process improvement proposal Size measurement PSP components - forms, logs and templates - process scripts - standards - benchmarks

53. SE 501 Dr. Basit Qureshi: Lecture for Week 9 PSP Structure 2 PSP0: establish a measured performance baseline PSP1: make size, resource, and schedule plans PSP2: learn defect and yield management PSP3: scale up PSP methods to larger projects The PSP can be extended to team development of large-scale software systems. – The PSP is a pre-requisite for the Team Software Process(TSP).

54. SE 501 Dr. Basit Qureshi: Lecture for Week 9 Software Engineering and the PSP The PSP offers an opportunity and framework for incorporating software engineering best practices into an individual engineer’s process. The PSP address’s 12 of the SW-CMM 18 KPA’s. For this reason, PSP sometimes referred to as a “Level 5” process. Data about PSP training show that students and engineers can improve their planning capabilities and produce higher quality products without an increase in cost.

55. SE 501 Dr. Basit Qureshi: Lecture for Week 9 1997 SEI Study Assignment Average 10987654321 0.3 0.4 0.5 0.6 0.7 0.8 0.9 PSP Level Average Assignment Number Size Estimation Accuracy PROBE size estimation begins Size Estimating Accuracy

56. SE 501 Dr. Basit Qureshi: Lecture for Week 9 1997 SEI Study Assignment Average PSP Level Average 10987654321 0.2 0.3 0.4 0.5 0.6 0.7 Assignment Number Effort Estimation Accuracy Linear regression for effort estimation begins Effort Estimating Accuracy

57. SE 501 Dr. Basit Qureshi: Lecture for Week 9 Total Defects/KLOC Removed 10987654321 40 60 80 100 120 140 Assignment Number Mean Defects Per KLOC Design review and code review introduced Total Defects/KLOC Removed

58. SE 501 Dr. Basit Qureshi: Lecture for Week 9 Test Defects/KLOC Removed

59. SE 501 Dr. Basit Qureshi: Lecture for Week 9 Defect Removal Comparisons 10987654321 0% 10% 20% 30% 40% 50% 60% Design Review Code Review Compile Test Assignment Number Mean Percent of Defects Removed 1997 SEI Study

60. SE 501 Dr. Basit Qureshi: Lecture for Week 9 Test Defects/KLOC vs. A/F R CMU 94 data

61. SE 501 Dr. Basit Qureshi: Lecture for Week 9 Productivity 1997 SEI Study

62. SE 501 Dr. Basit Qureshi: Lecture for Week 9 PSP Results for Industrial Use Although there have been some notable successes [Ferguson 1997], replicating the results in PSP training in an industrial setting has not been easy. – In most cases, the classroom PSP has to be adapted/tailored to the development environment and the domain of application. – PSP requires a great deal of discipline and most people need a supportive environment to maintain such discipline. – PSP does not fit well in an organization with a low maturity level. More positive results have been achieved when the PSP is coupled with the Team Software Process SM (TSP SM ) SM Team Software Process and TSP are service marks of Carnegie Mellon University.

63. SE 501 Dr. Basit Qureshi: Lecture for Week 9 Messages to Remember The PSP implements CMM key practices at the individual and team level respectively. – The PSP is a “level 5” process. The PSP is flexible and tailor-able, but requires discipline and commitment. The PSP can – reduce schedule deviation – improve the quality of the delivered product – improve productivity (or cause not decrease)