1. 1 Disciplined Software Engineering Watts S. Humphrey Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213-3890 Sponsored by the U.S. Department of Defense © 2001 by Carnegie Mellon University Carnegie Mellon University Software Engineering Institute

2. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 2 A War Story - 1 The team had 9 software and 4 hardware engineers. They were charged with building a new hardware-software system. This was to be a new communications line testing device. Management told the team the product was critically needed in 9 months.

3. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 3 A War Story - 2 Development of the prior product had taken two years. Would this be another crisis project? late over cost months of testing lots of defects Or could the team somehow succeed? The key question: Is this an engineering team?

4. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 4 Agenda The nature of engineering The Personal Software Process (PSP) The Team Software Process (TSP) TSP costs and benefits TSP introduction Conclusions SM Personal Software Process, PSP, Team Software Process, and TSP are service marks of Carnegie-Mellon University. SM

5. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 5 Definition Engineering is the practice of optimally using science to build or apply products to the needs of humankind While engineering is concerned with technology, engineers must provide solutions that are timely cost-effective and of high quality

6. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 6 Why Plan? Planning is like design. Without a plan, engineers waste time deciding what to do next can’t tell where they are focus on the wrong tasks consistently produce late, poor-quality, and over- budget products With a plan, engineers can negotiate rational schedules and resources react to changing conditions keep management informed take the time to do the job the right way

7. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 7 Why Use a Defined Process? A process is much like a plan. It defines the job steps the principal measures the framework for planning Without a process, engineers do not have a framework for making plans a structure for measuring the work a basis for analyzing their work

8. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 8 Why Measure and Track the Work? Measurements provide a factual basis for analyzing the work the historical data for planning an objective assessment of job status Without measurements, we are forced to rely on opinion mythology power and politics

9. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 9 Why Measure Quality? There are no good measures of quality. Without numbers, however, quality plans are just talk. Measured quality programs work. Software quality can be measured. defect levels usability performance function user satisfaction, etc.

10. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 10 Why Improve? People grow and mature. The technology is evolving. The world is changing faster than ever before. Our choices are to make change happen watch change happen wonder what happened

11. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 11 What Do We Need? To produce consistently good work, we need 3 things. We need engineers who do disciplined personal work. We need engineers who can work effectively in teams. We need engineering teams who act professionally, even under pressure.

12. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 12 PSP and TSP Principles - 1 The basic PSP and TSP principles are that Software work is challenging. Without proper methods, it is much harder. Sound software methods are not obvious. Engineers must be taught these methods to use them properly. Unless management supports the use of these methods, engineers will be able to use them. The PSP and TSP define and teach a family of software engineering, team, and management practices.

13. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 13 PSP and TSP Principles - 2 Plan the work before committing to or starting the job. Use a defined process. Measure and track development time, size, and defects. Plan, measure, and track program quality from the beginning of the job. Analyze every job and use the results to improve the process.

14. Version # Course or Lecture or Module Info. - page 14 How PSP and TSP Build Engineering Teams PSP Skill-building TSP Team-building TSP Team-working Team Management Team Members Team Disciplines Personal measures Process discipline Estimating & planning Quality management Project goals Team roles Team process Project plan Balanced plan Risk analysis Team communication Team coordination Status tracking Project reporting Disciplined Engineering Teams

15. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 15 Introducing Personal Disciplines Until they try it, most software professionals do not believe disciplined engineering will work for them. They won’t try it without evidence. They can’t get evidence without trying it. The PSP course provides this experience. Engineers write 10 programs. They start with their current methods. They advance in steps to the full PSP. They gather data on every program. They see from their own data that, with disciplined personal methods, they do better work.

16. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 16 The PSP Is an Evolving Process PSP0 Current process Basic measures PSP1 Size estimating Test report PSP2 Code reviews Design reviews PSP3 Cyclic development Team Software Process Teambuilding Risk management Project planning and tracking PSP2.1 Design templates PSP1.1 Task planning Schedule planning PSP0.1 Coding standard Process improvement proposal Size measurement

17. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 17 Effort Estimation Results 11109876543210 0.2 0.3 0.4 0.5 0.6 0.7 Mean Time Misestimation PSP Level Average Effort Estimation Accuracy Trend Program Number Estimated Minutes - Actual Minutes / Estimated Minutes

18. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 18 Quality Results 11109876543210 0 20 30 40 50 60 70 80 90 100 110 120 Mean Compile + Test PSP Level Mean Comp + Test Defects Per KLOC Removed in Compile and Test Program Number Mean Number of Defects Per KLOC

19. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 19 Productivity Results 11109876543210 20 22 24 26 28 30 Mean LOC/Hour PSP Level Average Lines of (New and Changed) Code Produced Per Hour of Total Development Time Program Number Mean LOC / Hour

20. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 20 Design Time Results 11109876543210 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Design Code Compile Test Time Invested Per (New and Changed) Line of Code Program Number Mean Minutes Spent Per LOC

21. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 21 Working In Teams Most software work is done by teams of engineers. The size and complexity of these teams will likely grow in the future. Teamworking methods are known and well understood, but not obvious. Teamworking can be taught. Effective teams provide the support framework engineers need to maintain their personal disciplines.

22. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 22 The TSP The TSP builds and guides cross-functional teams. It provides a defined teambuilding process a teamworking framework a supportive management environment It is designed for development and maintenance teams of 2 to 20 engineers. It includes a full set of process forms, scripts, and standards standard team-member roles a structured launch and tracking process a team and engineer support system

23. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 23 TSP Overview TSP projects can start on any phase. Each phase starts with a launch or relaunch step. The strategy is to overlap phases develop in increments balance team workload accelerate tasks wherever possible Postmortem Requirements Launch High-Level Design Relaunch Implementation Relaunch Integration and Test Relaunch

24. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 24 The TSP Launch Every TSP project starts with a launch. The launch takes 4 days. It is part of the project. It is led by a trained team coach. It immediately follows PSP training. In the launch, the engineers select personal roles define their own processes produce team and individual plans balance these plans assess and assign project risks

25. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 25 The Launch Process Meetings Day 1 1. Establish Product and Business Goals 2. Assign Roles and Define Team Goals Day 2 4. Build Top- down and Next-Phase Plans 5. Develop the Quality Plan 6. Build Bottom- up and Consolidated Plans Day 3 7. Conduct Risk Assessment 8. Prepare Management Briefing and Launch Report Launch Postmortem Day 4 9. Hold Management Review 3. Produce Development Strategy

26. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 26 Planning a TSP Project In the TSP, planning is done at three levels. The team first makes overall project and quality plans. Then, the team makes a detailed plan for the next project phase. The engineers next produce detailed personal plans for the following phase. Finally, the engineers balance their plans to evenly distribute the work and minimize the schedule.

27. Version # Course or Lecture or Module Info. - page 27 TSP Planning Produce Conceptual Design Make Size Estimates Define Process and Tasks Estimate Tasks Estimate Weekly Time Estimate Defects Injected Estimate Yields Make the Engineers’ Plans Balance Team Workload Conceptual Design Overall Project Plan Quality Plan Detailed Engineer Plans Balanced Team Plans

28. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 28 Tracking a TSP Project The detailed team and individual plans facilitate precise project tracking. The team members regularly reassess project risks and devise mitigation actions. In weekly team meetings, the engineers report on task status review the key risks rebalance the workload The team produces weekly management status reports.

29. Version # Course or Lecture or Module Info. - page 29 TSP Project Tracking Quality Summary Schedule Status Engineer A Product Summary Enter Defects by Component and Phase Enter Size by Component Enter Week Task Completed Enter Time by Task Updated Team and Engineer Task, Schedule, and Quality Plans Team Task and Schedule Summary Task Status Engineer A

30. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 30 Management Support The TSP will not work unless the engineers are fully supported by all management levels. During the project launch the team reviews their plan with management. answers questions resolves issues explores alternatives To sustain the TSP, management must periodically review the project probe the team’s data focus on quality

31. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 31 Engineering Reactions With proper support, TSP engineers can produce extraordinary results. Engineers like to use the TSP: This really feels like a tight team. The TSP forces you to design, to think the whole thing out. Design time is way up but code time decreased to compensate. Tracking your time is an eye opener. Really good teamwork on this project - no duplication of effort. I’m more productive. Gives you incredible insight into project performance. Wonderful to have team members assigned specific roles. Team really came together to make the plan. I feel included and empowered.

32. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 32 A TSP Industrial Project - 1 Communications test equipment 12 software and 3 hardware engineers not all software engineers PSP trained Plan Size - KLOC 110 Effort - hours 16,000 Schedule - weeks 77 Defects per KLOC Integration and system test 1.1 Field trial 0.0 Customer use 0.0

33. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 33 A TSP Industrial Project - 1 Communications test equipment 12 software and 3 hardware engineers not all software engineers PSP trained Plan Actual Size - KLOC 110 89.9 Effort - hours 16,000 14,711 Schedule - weeks 77 71 Defects per KLOC Integration and system test 1.1 0.6 Field trial 0.0 0.02 Customer use 0.0 0.0

34. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 34 A TSP Industrial Project - 2 Comparison with a prior project of 9,500 LOC. Engineers not PSP trained. Communications system controller TSP non-TSP Size - KLOC 89.9 9.5 Field testing duration engineering trial 2 weeks 3 months acceptance trial 3 weeks 6 months Trial defects/KLOC 0.02 5+

35. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 35 TSP Benefits # of Defect Detected Release # 6 Release # 7 Release # 8 Release # 9 75% lower Defect PSP/TSP trained (Boeing Pilot #1) 2.36X more Sloc count Software Size

36. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 36 TSP Benefits System Test time Release # 6 Release # 7 Release # 8 Release # 9 PSP/TSP trained (Boeing Pilot #1) 2.36X more Sloc count 32 days 41 days 28 days 4 days 94% less time

37. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 37 A TSP Government Project Air Force flight planning system 6 software engineers 5 were PSP trained Integration and system test time was reduced from an average of 22% of schedule to 2.7%. Productivity was 2.33 times the same team’s prior project. Projects A B C TSP Size - LOC 67,291 7,955 86,543 25,820 Test Days 63 23 92 6 System test defects/KLOC 2.21 4.78 2.66 0.54 Acceptance test defects/KLOC N.A. 1.89 0.07 0.15

38. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 38 A TSP Maintenance Project A company applied PSP/TSP with 2-engineer teams on 14 maintenance projects. Average results showed. TSP non-TSP Time estimate errors +- 7.5% Not tracked Size estimate errors +- 10.0% +-23.4% Defects/page 0.14 0.35 Review yield 57.1% Not tracked Productivity - pages/hour 2.28 2.08

39. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 39 Getting Started with PSP/TSP Sprinkling a few PSP-trained engineers around an organization will not produce noticeable results. Installing PSP/TSP in an organization requires a team-based improvement focus careful planning senior management involvement and sponsorship a change in the behavior of both the engineers and their managers

40. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 40 Conclusion The PSP shows software professionals how to use engineering methods in their personal work. The TSP shows organizations how to build and manage effective engineering teams. When software professionals use sound engineering methods, their teams produce extraordinary results.

41. © 2001 by Carnegie Mellon University Version 010130 Disciplined Software Engineering - page 41 For More Information Visit the PSP or TSP web sites http://www.sei.cmu.edu/psp/ http://www.sei.cmu.edu/tsp/ Contact a PSP transition partner http://www.sei.cmu.edu/collaborating/partners/trans.part.psp.html Contact SEI customer relations Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213-3890 Phone, voice mail, and on-demand FAX: 412/268-5800 E-mail: customer-relations@sei.cmu.edu