1. ________________________________________________________________________ Jonsson School of Engineering and Computer Science Dr. Mark C. Paulk 2015 ASEE Annual Conference, 21 February 2015 Adoption of Agile Methods by High Maturity Organizations

2. Software CMM v1.1 (1987-2005) Competent people (and heroics) Defect Prevention Technology Change Management Process Change Management Continuous process improvement Product and process quality Engineering processes and organizational support Project management processes Quantitative Process Management Software Quality Management Organization Process Focus Organization Process Definition Training Program Integrated Software Management Software Product Engineering Intergroup Coordination Peer Reviews Requirements Management Software Project Planning Software Project Tracking & Oversight Software Subcontract Management Software Quality Assurance Software Configuration Management LevelFocusKey Process Areas Initial Optimizing 1 Repeatable 2 3 Managed 4 5 Defined 2

3. Implications of Maturity Initial Repeatable Defined Process is informal and unpredictable Project management system in place; performance is repeatable Software engineering and management processes defined and integrated Product and process are quantitatively controlled Time/$/... Optimizing Process improvement is institutionalized Time/$/... LevelProcess CharacteristicsPredicted Performance Managed 1 2 3 4 5 Better predictability… Less variability… Improved performance… 3

4. Empirical Data on Improvement By Maturity Level Quality (reliability, defects) improves by roughly a factor of 2 (or more) -C. Jones, “Software Benchmarking,” IEEE Computer, October 1995. -K.D. Williams, "The Value of Software Improvement… Results! Results! Results!" SPIRE97, June 1997. -R. Yacobellis, “Panel: Does SEI Level 5 Lead to High Quality Software?” COMPSAC 2001. Effort for a given product decreases 15-21%, productivity increases, cycle time decreases -B.K. Clark, “Quantifying the Effects on Effort of Software Process Maturity,” IEEE Software, November/December 2000. -D.E. Harter, M.S. Krishnan, and S.A. Slaughter, “Effects of Process Maturity on Quality, Cycle Time, and Effort in Software Product Development,” Management Science, April 2000. -L.H. Putnam, “Linking the QSM Productivity Index with the SEI Maturity Level,” QSM, 2000. 4

5. CMMI-DEV v1.3 Process is unpredictable, poorly controlled, and reactive Process is characterized for projects and is often reactive Process is characterized for the organization and is proactive Process is measured and controlled Focus is on quantitative continuous process improvement Level Process Characteristics Requirements Management Project Planning Product & Process Quality Assurance Configuration Management Project Monitoring & Control Supplier Agreement Management Quantitative Project Management Organizational Process Performance Causal Analysis & Resolution Process Areas Requirements Development Technical Solution Product Integration Validation Verification Organizational Process Focus Integrated Project Management 1 Initial 2 Managed 3 Defined 4 Quantitatively Managed 5 Optimizing Measurement & Analysis Organization Process Definition Organizational Training Risk Management Decision Analysis & Resolution Organizational Performance Management 5

6. A Scrum Adoption Survey 184 distinct organizations responded to the 2011 Scrum adoption survey 24 ML5 2 ML4 44 ML3 61 ML2 125 ML1 and unknown 128 projects adopting Scrum M.C. Paulk, “A Scrum Adoption Survey,” ASQ Software Quality Professional, March 2013. 6

7. Organizational Size 7

8. Organizational Size by Level Org SizeML5ML4ML3ML2Unknown ≤25 100215 26-50 000114 51-75 00005 76-100 00029 101-200 202310 201-300 200210 301-500 41019 501-1000 30326 1001-2000 30419 >2000 00000 8

9. Software Engineering Methods 9

10. Methods by Level A high percentage of high maturity organizations use Scrum. Many high maturity organizations use XP, FDD, and the Unified Process. 10 MethodML5ML4ML3ML2Unknown Scrum170121686 Extreme Programming (XP)506616 Feature Driven Development (FDD)502411 Crystal methods (including Crystal Clear)00105 Team Software Process (TSP)10210 Unified Process (including RUP, AUP, OUP)724618 Other methods8110887

11. Requirements Volatility Per Month 11

12. Requirements Volatility by Level 12 Requirements VolatilityML5ML4ML3ML2Unknown <1% per month406413 1-3% per month30315 3-5% per month30015 5-10% per month403317 10-20% per month710617 20-50% per month100014 >50% per month50117

13. Use of Scrum 13

14. Use of Scrum by Level Few high maturity organizations are unaware of Scrum or uninterested in it. 14 Scrum AdoptionML5ML4ML3ML2Unknown We never heard of Scrum before 10104 We are aware that Scrum exists 212132 We are currently piloting Scrum 50019 We have piloted Scrum but no decision about adoption has been made 11419 We are currently deploying Scrum across the organization 202316 Scrum is one of the standard methods we use 6031130 Scrum is the normal way we build software 403025

15. Scrum Team Size by Level Team SizeML5ML4ML3ML2Unknown ≤301436 4 to 6502222 7 to 9501427 10 to 1220036 13 to 1540112 ≥16602214 15

16. Quality of the Software 16

17. Quality by Level QualityML5ML4ML3ML2Unknown much lower00001 lower10123 about the same212414 higher1205537 much higher502215 17 Quality tends to be higher for organizations adopting Scrum in general.

18. Cost 18

19. Cost by Level CostML5ML4ML3ML2Unknown much lower00003 lower703522 about the same1315529 higher00119 much higher00027 19 Cost does not appear to be significantly affected for most organizations adopting Scrum. lower for a noticeable percentage

20. Meeting Schedule Expectations 20

21. Schedule Expectations by Level Schedule ExpectationsML5ML4ML3ML2Unknown much lower00004 lower20013 about the same311412 higher1206633 much higher302318 21 Meeting schedule expectations appears to be higher for organizations adopting Scrum in general.

22. Customer Satisfaction 22

23. Customer Satisfaction by Level Customer SatisfactionML5ML4ML3ML2Unknown much lower00001 lower00022 about the same10149 higher1115328 much higher803526 23 Customer satisfaction tends to be higher for organizations adopting Scrum in general.

24. Concluding Thoughts It appears that most high maturity organizations are aware of, have piloted, and have adopted agile methods. Most particularly Scrum. With good success in terms of customer satisfaction, quality, and meeting schedule expectations… and some impact on improving cost. This survey is only a first step in exploring the adoption of agile methods by high maturity organizations… 24

25. 25 Questions and Answers