University of Southern California Center for Systems and Software Engineering A Model for Estimating Agile Project Schedule Acceleration Dan Ingold, USC-CSSE SSCM/COCOMO Forum 17 October 2012

University of Southern California Center for Systems and Software Engineering Project Goals Research Question: Can we quantify the schedule acceleration to be expected from employing agile techniques, given a range of development project characteristics? Goal is not to estimate what a team using poor SE/architectural practices & processes can achieve Can always cut corners to reduce schedule… for a while, at least Goal is to examine what effects the various characteristics of a project using good practices have on achieving schedule compression October 17, 2012Copyright © USC-CSSE2

University of Southern California Center for Systems and Software Engineering COCOMO for Agile Projects? COCOMO II calibrated against larger projects –Larger projects typically optimized to minimize cost –Agile projects typically optimized to minimize schedule Over-estimates schedule for smaller projects –Estimates schedule varies with cube-root of effort –Smaller projects vary with square-root of effort Optimizes 27-PM project as 2.45 persons / 11 mos. –Minimizes communication overhead, optimizes effort –But… 11 months is too long under competitive pressure October 17, 2012Copyright © USC-CSSE3

University of Southern California Center for Systems and Software Engineering (Re)introducing CORADMO Constructive Rapid Application Development Model Observations of early-agile projects completing 27- PM projects in 5 months with 5.4 persons, and even 9 persons to complete in 3 months Derivative of COCOMO II, introduced ~2000 –Implemented as COCOMO II / COPSEMO post-processor –Derived six drivers through initial two-round Delphi Lacked critical mass of data to calibrate model October 17, 2012Copyright © USC-CSSE4

University of Southern California Center for Systems and Software Engineering New CORADMO Drivers SERC RT-34 tasked to study “expediting SE” –Identified candidate firms and agencies that were successfully compressing project development time –Conducted series of onsite visits and in-depth interviews Derived expanded set of factors common across these entities, good candidates for new drivers –Product: describes nature of system to be developed –Process: characterizes the development methodology –Project: describes execution of the development effort –People: characterizes capabilities of development staff –Risk: describes stakeholder willingness to accept risk October 17, 2012Copyright © USC-CSSE5

University of Southern California Center for Systems and Software Engineering General CORADMO Structure CORADMO depends on the existence of a good baseline effort estimate; it does not estimate effort Estimated duration D is proportional to square-root of estimated effort PM Like COCOMO, CORADMO uses product of multiplier factors, rated according to project characteristics October 17, 2012Copyright © USC-CSSE6

University of Southern California Center for Systems and Software Engineering Product Factors Simplicity: simple products are easier to develop Reuse: reuse saves work (or does it?) Deferrals: postpone features to fit schedule Modeling: working models vs complete documents Maturity: fewer technologies needing development October 17, 2012Copyright © USC-CSSE7

University of Southern California Center for Systems and Software Engineering Process Factors Concurrency: serial waterfall / concurrent iteration Streamlining: bureaucracy requires “just so”? Tool support: integrated development, continuous integration, automated testing, model-to-code, etc. October 17, 2012Copyright © USC-CSSE8

University of Southern California Center for Systems and Software Engineering Project Factors Staff size: more people ≈ higher communication overhead (are factors large enough for big staff?) Collaboration: how well does team share data? MMPTs: tool support within and across domains October 17, 2012Copyright © USC-CSSE9

University of Southern California Center for Systems and Software Engineering People Factors KSAs: how senior is team? How agile is team? Single vs Multi-domain: how well do team skills cross domain boundaries (analogous to MMPTs) Compatibility: can’t we all just get along? October 17, 2012Copyright © USC-CSSE10

University of Southern California Center for Systems and Software Engineering Risk Acceptance Factors How willing are stakeholders to accept risk? –Tolerance of chaotic, evolving processes –“We’ve always done it this way.” “The manual says these are the required processes and artifacts.” –Adaptive, oriented toward product completion Many of the accelerated-development teams reviewed had compliant, risk-tolerant customers October 17, 2012Copyright © USC-CSSE11

University of Southern California Center for Systems and Software Engineering Commercial Calibration Midwest software development firm using agile Supplemented agile with additional SE processes –Detailed business process analysis –Delphi estimates of software testing effort –Risk-based situation audits –Componentized architectures –… Makes this firm reasonably comparable to complex aerospace/defense projects from which CORADMO factors derived October 17, 2012Copyright © USC-CSSE12

University of Southern California Center for Systems and Software Engineering Commercial Calibration (cont’d) Copyright © USC-CSSE13October 17, 2012

University of Southern California Center for Systems and Software Engineering Commercial Calibration (cont’d) Projects varying from 10 KSLOC to 400 KSLOC Varying levels of complexity and technology Selected rating factors based on reported project characteristics, and of firm as a whole –Product: C++ projects received Low ratings; HTML/VB projects received Very High ratings –Process: Most projects reported as “highly concurrent,” received Very High ratings –Project: Variation in staff sizes results in different ratings –People: Very senior staff rated as Very High –Risk: Consistent, rigorous and balanced approach yielded Nominal risk ratings October 17, 2012Copyright © USC-CSSE14

University of Southern California Center for Systems and Software Engineering Calibration Discussion Acceptable results for first-cut –One outlier discarded, described as having high requirements churn –Tends to over-estimate schedule compression Outlier suggests Product may require sub-factor for requirements churn: perhaps “stability?” Process within narrow range Project within narrow range People factor has single rating –May not extend to wider range or less capable staff –But… rapid projects often employ most-senior staff October 17, 2012Copyright © USC-CSSE15

University of Southern California Center for Systems and Software Engineering Decision Support Case Study Hypothetical company, composite of real affiliates of USC CSSE Illustrates use of CORADMO tool to support decision to move to more agile approach Evaluate as-is and to-be conditions, as rated by model sub-factors Determine potential schedule compression through adoption of new strategy October 17, 2012Copyright © USC-CSSE16

University of Southern California Center for Systems and Software Engineering Case Study—As-is Evaluate current state against factors Use to inform decision on change to more rapid development Overall acceleration factor of current state = 1.01 Copyright © USC-CSSE17October 17, 2012

University of Southern California Center for Systems and Software Engineering Case Study—Initial To-Be Produce artifacts more concurrently Causes reductions in –Technology maturity –SE tool support –General SE KSAs –Team compatibility Expected 5% schedule improvement Saw 4% schedule increase Copyright © USC-CSSE18October 17, 2012

University of Southern California Center for Systems and Software Engineering Case Study—Final To-Be Restore reduced factors to baseline, by being aware of the potential problems Choose to –Perform more activities concurrently –Improve bureaucratic internal and external processes Schedule improves by 8% Copyright © USC-CSSE19October 17, 2012

University of Southern California Center for Systems and Software Engineering Case Study Shortcomings Case study illustrates some problems with using the factors in the model We really would have expected some more noticeable change in schedule The expected improvements and discovered shortfalls are so small as to be lost in the noise –Suggests either that the factors are too small –Or that the method of combining sub-factors (in this example, by averaging them) is incorrect So, more work to be done… October 17, 2012Copyright © USC-CSSE20

University of Southern California Center for Systems and Software Engineering Further Issues How to handle contribution of sub-factors –Average, additive, multiplicative, preponderance? –Offsetting: does a very-low complement a very-high? Factors complete and correct? Too many factors (too complex)? Accuracy, consistency of how factors would be coded by potential users Overall range of factor multipliers, , (3.4:1) seems consistent with reported range of schedule compression in agile projects, but… October 17, 2012Copyright © USC-CSSE21

University of Southern California Center for Systems and Software Engineering Next Steps Need additional data on wider range of projects that would be characterized as “rapid” or “agile” So, looking for volunteers who would be willing to share project performance data Conducting workshop here to discuss factors –Delphi to uncover omitted (or superfluous) factors –Effect of contravening factors –Sufficient range of multiplier factors October 17, 2012Copyright © USC-CSSE22

University of Southern California Center for Systems and Software Engineering Questions? Copyright © USC-CSSE23October 17, 2012