Applying The CMMI/SPI In A Cost Controlled Environment Dennis J. Frailey

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 2 Agenda How to Get There Lessons Learned Tangible Results

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 3 A Long Journey Become Repeatable Define SW Processes Optimize Processes Manage by Fact Continuous Improve- ments 1989 Today This presentation will discuss how we got where we are today, and give a glimpse into the future.

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 4 Status Report Software was a universal problem throughout the company Common symptoms: –Cost overruns –Schedule overruns –Quality problems Customers were unhappy with the industry as a whole and we were no exception OUR MISSION - FIX IT

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 5 Typical Comments “We are running by the seat of our pants.” “We build requirements and plans by throwing engineers at the project and seeing what they come up with.” “Those of us who are lucky enough to attend training are often not allowed to use it.” “Software is the last thing we think about.” “Testing is the last thing we think about -- after software.”

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 6 Management Reacted

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 7 Important Early Lessons - We Learned About Ourselves Not as good as we thought we were Not as consistent as we thought we were Much more software than we thought we had Many pockets of excellence But not much transfer of knowledge

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 8 Most Of Our Software Was Not Where We Thought it Was Where we had concentrated our efforts

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 9 Organizational Barriers To Improvement No central oversight of software product development Several independent business entities within the organization “Cottage industry” mentality among software developers –“Not Invented Here” –Perception of uniqueness –Customer-specific or project-specific standards, policies, and objectives –Dependence on heroes

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 10 Organizational Bright Spots A Cooperative Team of Software Leaders (Software Improvement Team) –had been meeting to resolve mutual problems A Centralized Software Technology Department –had many of the necessary resources in place A Corporate Process Group was Forming –developed a corporate wide process framework & goals –nurtured top management support A Reorganization was Pending –to put more emphasis on coordination within engineering functions

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 11 SEI was beginning to promote its maturity model and assessment process External Forces »CUSTOMERS were beginning to demand use of the SEI model and assessments

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 12 We Were Not Alone! Software Process Maturity Distribution (from SEI)

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 13 A Process For Improving The Process BIG PICTURE VIEW - FOUR STEPS Level 1 -> Level Level 4 -> Level Level 3 -> Level Level 2 -> Level Assess CMM CMMI

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 14 Why Does It Take So Long To Move Up One CMM Level? Each level represents a major cultural change in the organization –Attitudes –Organizational Structure –Mission of Support Organizations And each level requires a lot of work –Five major sub-steps –The business has other priorities as well –Overhead constraints limit process effort

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 15 We Don’t Control All Of The Variables Especially at higher CMM levels, both the other parts of our organization AND our customers and suppliers must be part of the solution

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 16 The Process Within Each Step Level 1 -> Level Level 4 -> Level Level 3 -> Level Level 2 -> Level Establish the Goal Vision Definition Buy-in and Resources Develop Training Process & Methods Just in Time Languages Pilot Efforts Early Adopters Lessons Learned Organize for Deployment Organization Support Resources Full Deployment Buy-in Training Motivation Support Monitor You must have buy- in at All Levels before deploying process improvement

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 17 Establish the Goal Resources Definition (What You Want to Do) Goal Buy-In / Motivation Vision Define the Process Definition Standard Procedure Operating Instructions Develop Training Process & Methods Just in Time Languages Pilot Efforts Various Programs Lessons Learned Organize for Deployment Organization / QITs SEPG Process Engineers Full Deployment Site Based Action Plans Sponsorship Assessments & Training Execution of Plans

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 18 Training Process Training for All New Hires Focused Training for Specific Functions Software Project Management Inspections Process Tailoring and many more Program Manager Training Sessions for Each Project A Work Force Capable of Applying the Process “Just in Time” Planning Sessions for Each Project Project Planning Requirements Analysis Preliminary Design etc. Define the Process Definition Standard Procedure Operating Instructions Develop Training Process & Methods Just in Time Languages Pilot Efforts Various Programs Lessons Learned Organize for Deployment Organization / QITs SEPG Process Engineers Full Deployment Site Based Action Plans Sponsorship Assessments & Training Execution of Plans

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 19 Pilot Programs Identify changes needed for deployment on real programs Give new ideas to eager, early adopters Provide extra support to help them and to learn from them Develop a small core of vocal, influential promoters Define the Process Definition Standard Procedure Operating Instructions Develop Training Process & Methods Just in Time Languages Pilot Efforts Various Programs Lessons Learned Organize for Deployment Organization / QITs SEPG Process Engineers Full Deployment Site Based Action Plans Sponsorship Assessments & Training Execution of Plans

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 20 May require significant organizational changes –establish oversight bodies that really do pay attention to process improvement progress and results –shift resources from organization-wide SEPG to site-specific and project- specific counterparts –reassign influential and experienced personnel to key process deployment roles Plan For Deployment WHY CHANGE ? Define the Process Definition Standard Procedure Operating Instructions Develop Training Process & Methods Just in Time Languages Pilot Efforts Various Programs Lessons Learned Organize for Deployment Organization / QITs SEPG Process Engineers Full Deployment Site Based Action Plans Sponsorship Assessments & Training Execution of Plans

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 21 Deployment Define the Process Definition Standard Procedure Operating Instructions Develop Training Process & Methods Just in Time Languages Pilot Efforts Various Programs Lessons Learned Organize for Deployment Organization / QITs SEPG Process Engineers Full Deployment Site Based Action Plans Sponsorship Assessments & Training Execution of Plans % with the pro- gram time Early Adopters Bulk of the Population Laggards The Deployment Learning Curve

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 22 Deployment Elements Buy-in Training Motivation Support Monitor Effective deployment is perhaps the most significant difference between successful and unsuccessful process improvement initiatives

Lessons Learned

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 24 Leaders Must Lead Organizational leaders must be highly visible at the head of the parade –Participating actively in all phases from goal setting to the nitty-gritty of deployment –Communicating that they know and care Foster an attitude of success and unity It must be clear up and down the organization that the improvements are not going away

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 25 Set Clear Expectations Who is expected to do What, When, Where, How and Why Distinguish between requirements, desires, and options Document what is expected Communicate and enforce the consequences Clarify to whom the expectations apply

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 26 Tie to Organizational Initiatives Synergy with organizational programs for overall corporate improvement Support for what you want to do instead of competition for resources Acknowledgement and appreciation of what you are doing from top management

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 27 Don’t Take On Too Much Higher Maturity Levels 100% Deployment Different Types of Software Mission Critical Test Maintenance Simulation etc. Higher CMM Level 100%100% A l l T y p e s »The magnitude of the effort is sobering

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 28 Educate Every person playing a role should be competent and knowledgeable They should understand the principles behind the methods –So they know when NOT to do it as prescribed When they know what to do and why, people will dare to achieve and improve

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 29 Deploy Behaviors, Don’t Teach Models (CMM, CMMI, etc.) –One behavior every few weeks Frequent, short briefings With plenty of buy-in up the line –Over a period of time –With strong reinforcement Set expectations Reporting and monitoring Behaviors become integrated parts of the process –And training available NOW

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 30 Frequent Feedback Surveys to understand how things are going Measurements and monitoring Be willing to change if it isn’t working Walk the talk

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 31 The SEPG Must Not be an Ivory Tower Populated with people from real projects –Job rotation into and out of the SEPG –Project people participate in defining the processes and procedures Actively out there, supporting projects –Playing a role in the day to day activity –Providing subject matter experts who get involved Their world view must include the real world, not be limited to writing procedures and processes

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 32 The Process Must be Efficient and Appropriate Large, thick documents are never read - and they cost a lot of money It takes time to mature a process Tailoring is an essential part of an effective process - and it saves money –Tailoring can be reviewed, approved, justified, etc.

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 33 The WEB Can Be Wonderful or Horrible You must have a systematic approach to using web resources –Plan it, don’t just tell people to use it –Standard architectures, formats, etc. –Consistent look and feel –Systematic methods of developing and maintaining web pages and sites

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 34 Just-in-Time Training Short training modules on individual topics Taught right before people need to know Taught by people who know how it works in practice May be integrated with project planning or discussion activities

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 35 Keep Your Eye on the Ball Regular reporting and feedback mechanisms Track progress on deployment Project Improvements ABC… Alpha Beta … Nobody wants to be “red” on a report card

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 36 Outside Consultants See What You Overlook Bring them in early! –Lack of adequate evidence of good practices –Inadequate access to data –Ad-hoc, inconsistent behaviors –Misunderstandings –… Dry run your assessments

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 37 Evolving Role Of The SEPG Initially, it was to define the process for software development and the process for improving the organization As time went on, it became a support organization to help with the rest of the steps of the improvement process And now it provides many process maintenance functions: –Software Metrics Data Base - collection & analysis –Software Process Assets Library –Lessons Learned and Problem Reports –Integrated with Tool Support Organization

Tangible Results

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 39 Some Of The Tangible Signs Consistency in measurements and processes It begins to be possible to collect meaningful data at the organizational level Quantifiable results: –Measurable improvements in quality, productivity and cycle time –Measurable improvements on customer satisfaction surveys

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 40

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 41

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 42

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 43 Alignment Of Methods Common measurements Consistent terminology Compatible reward systems VS.

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 44 Reductions in Rework Rework = Cost and Schedule Slips Process improvement helped us –See the rework –Quantify the rework –Identify the causes of rework –Motivate spending to reduce rework –Reap the benefits of less rework

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 45 What Process Does For Us Codifies the expertise of our predecessors and ourselves Deals with the details so we can concentrate on what is important Enables us to stand on the shoulders of giants Gives us the freedom and the support we need to do our job in a professional manner

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 46 Misuse of Process Process should be a Guideline, not a Mandate -- like a map for an explorer It should be like a high level language for the process of developing software Processes should depend on the knowledge and expertise of practitioners -- not substitute for it

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 47 Other Fruits of our Labor Variation between best and worst projects has been reduced substantially Overall performance on cost, schedule, and quality have improved –It helps to have a way to measure these! Our processes are followed more uniformly and are respected by our colleagues in other disciplines

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 48 Final Remarks The journey has been tough but the results are showing in very quantitative ways A long term outlook is required for success Never underestimate the power of a supportive and insistent customer We are well past the “point of no return” - turning back is not an option

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 49 References – Slide 1 of 3 Bate, Roger R. (1991), STEP - Software Engineering Process, Guest Editorial, Texas Instruments Technical Journal, Vol. 8, no. 3, ISSN , 2-4. Benno, Steve and Dennis Frailey (1994). “Software Process Improvement in DSEG, ,” Texas Instruments Technical Journal, Vol. 12, no. 2, Department of Defense (1988), Defense System Software Development, Dod-STD 2167A, 29 Feb. 1988, Department of Defense, Washington D.C., Fortin, Pauline with David Carter and Dennis Frailey (1994), Who's On First - Covering All the Bases for Software Process Improvement, Proceedings, Fourth IEEE Workshop on Technology Transfer, April, 1994.

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 50 References – Slide 2 of 3 Frailey, Dennis J. (1993), "Concurrent Engineering and the Software Process," Proceedings, 2nd International Conference on the Software Process, IEEE Computer Society Press (ISBN , IEEE catalog no. 93TH0509-0), Frailey, Dennis J. (1991a), "Defining a Corporate-wide Software Process," Proceedings, 1st International Conference on the Software Process, IEEE Computer Society Press (ISBN ; order no. 2490), Frailey, Dennis J. (1991b), "Activities of the STEP Basic Software Development Cycle," Texas Instruments Technical Journal, Vol. 8, no. 3, ISSN , Humphrey, Watts S. (1989), Managing the Software Process, Addison-Wesley, Reading, MA, 1989.

November 13, 2003 SPI in a Cost Controlled Environment Copyright © , Dennis J. Frailey, All Rights Reserved Slide 51 References – Slide 3 of 3 Humphrey, Watts S. (1987a), Characterizing the Software Process: A Maturity Framework, Software Engineering Institute, CMU/SEI-87-TR-11, ADA182895, June Humphrey, Watts S. and W.L. Sweet (1987b), A Method for Assessing the Software Engineering Capability of Contractors, Software Engineering Institute, CMU/SEI-87- TR-23, ADA187320, September IEEE (1992), IEEE Standard for Developing Software Life-Cycle Processes, IEEE-STD IEEE Computer Society, New York. Paulk, Mark, et. al. (1993), Capability Maturity Model for Software, Version 1.1, Software Engineering Institute, CMU/SEI-93-TR-2, February, SEI (1991), State of the Practice: A Software Process Maturity Perspective. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA (presentation at the 1991 SEI Affiliates Symposium, August 27-29, 1991.