3 Different Cultural Orientations Science EUROPE:Software as a Science –Formal Methods, Object-Oriented Design Production JAPAN:Software as Production –Software Factories, Zero-Defects Service INDIA:Software as a Service –Infosys, Tata, Wipro, Satym, Cognizant, Patni Business The USA:Software as a Business –Windows, Office, Navigator, $$$$
4 Problems in Software Development Similar problems recurring since the 1960s 1969 NATO Report on Software Engineering: Documented problems in –requirements, design vs. coding separation –estimates, monitoring progress, communication –productivity (26:1), metrics, reliability (bugs) –hardware dependencies, reuse –maintenance costs Sound familiar??
5 Proposed Solutions Many attempts –IBM-style software engineering (1960s, 1970s) –Japanese “software factories” (1970s, 1980s – stable teams, standard process & tools, reuse) –SEI Capabilities Maturity Model (1980s to present) –“Iterative” and “Agile” methods (US, Europe) No one process perfect for all projects –Variations: business models, customer requirements, application domain, competition, pace of change, local culture??, etc. Common Theme: How balance quality, features & design flexibility, with cost & speed
6 Different Process Philosophies Waterfall-style (sequential, “Stage-gate”) versus Iterative-style (agile, incremental) –Spiral –Rapid Prototyping –Synch-and-Stabilize (Microsoft, PC makers) –IBM Rational’s Unified Process & Toolkit –HP’s Evo Process (short cycles of mini-waterfalls) –Extreme Programming (XP), SCRUM, AGILE –Many other variations at companies
7 Traditional Waterfall Model (One development cycle) Requirements Functional design Detailed module design Module construction Integration/system test Module rework (debug) Re-test, debugging Product release
8 Frequent Waterfall Process Result Requirements Functional design Detailed module design Module construction Integration/system test If modules change a lot, integration fails and you can experience an infinite defect loop.
9 Reality: Spectrum of Approaches Process Choices for Different Projects User Feedback During Project Uncertainty in Requirements Early/Often High Late/Occasional Low # of Cycles, Releases1 Many Traditional Waterfall, Older “Factory-like” Approaches Rapid Prototyping XP Agile or Iterative Methods Incremental Adapted from Bill Crandall (HP)
13 “Newer” Iterative Practices IndiaJapanUSAEurope etcTotal No. of Projects24273122104 Subcycles -- Yes79%44%55%86%64% Beta tests -- Yes67% 77%82%73% Pair Testing -- Yes54%44%35%32%41% Pair Programmer -- Yes58%22%36%27%35% Daily Builds at project start17%22%36%9%22% In the middle13%26%29%27%24% At the end29%37%36%41%36% Regression test each build92%96%71%77%84%
14 “Crude” Output Comparisons IndiaJapanUSAEurope etc. TOTAL Projects24273122104 LOC/ Month median209469 cf. 389 in 1990 270 cf. 245 in 1990 436374 Bugs/ 1000 LOC median.263.020 cf..20 in 1990.400 cf..80 in 1990.225.150
15 Observations from Global Survey Most projects (64%) not pure waterfall; 36% were! Mix of “conventional” and “iterative” common -- use of functional specs, design & code reviews, but with subcycles, regression tests on frequent builds Customer-reported defects improved -- over past decade in US and Japan; LOC “productivity” may have improved a little Japanese still report best quality & productivity -- but what does this mean? Preoccupation with “zero defects”? Need more lines of code to write same functionality per day as US & Indian programmers? Indian projects strong in process and quality -- but not as strong as their dominance of CMM Level 5 suggests??
16 Hewlett Packard Pilot Study (2003 IEEE Software, “Tradeoffs” article) Managers – When use iterative or waterfall? –Survey: 35 responses, 29 projects with complete data –Median – 170K LOC, with 70K new code; 9-person team, 14 month projects –59% applications, 38% systems, 28% embedded 74% of variation in defects explained by early prototypes, design reviews, and integration/regression testing on builds –Median project: 40% of functionality complete when first prototype released and 35.6 defects per million (.04/1000) LOC, reported by customers in 12 months after release, and 18 LOC per person day (360/month)
17 Multivariate Regression Analysis Some striking results, compared to median: Releasing prototype earlier with 20% of functionality 27% reduction in defect rate Integration/regression testing at each code check-in 36% reduction in defect rate Design reviews 55% reduction in defect rate Releasing prototype with 20% of functionality 35% rise in LOC output/programmer Daily builds 93% rise in LOC output/programmer
18 Observations from HP Survey Best “nominal” quality from traditional “waterfall” (fewer cycles & late changes = less bugs, of course!!) Best balance of quality, flexibility, cost & speed from combining conventional & iterative practices BUT: Differences in quality between waterfall & iterative disappear if use a bundle of techniques: –Short development subcycles (subprojects/milestones) –Early prototypes to get customer feedback –Frequent builds to incorporate feedback, changes –Design/code reviews (check quality continuously) –Regression tests on each build (check for errors, late changes, integration problems)
19 Where Iterative May Work Better Fast-paced product markets where technologies, competition, and product features may have to change a lot during a project. Product or custom projects where customers place very high emphasis on leading-edge features. Custom systems where customers place high emphasis on responsiveness to their input during a project. Product or custom projects that require experimentation as in lots of short design, build, and test cycles. Other?
20 Where Waterfall May Work Better Extremely high-reliability systems (product or custom projects), where functions are very well understood and no changes in requirements during a project are desired Embedded products with hardware constraints that cannot be easily changed Contract software where client requires a detailed proposal upfront and not possible to limit scope/phases Complex projects (product or custom) where parts of design & coding are outsourced, off-shored, or done in multiple sites, AND there are weak mechanisms to synchronize and manage distributed teams
21 High-Level Process Innovation & Design Architecture Team Management Project Management Testing & QA Some Comments on Good Practices
22 High-Level Process Strategy “You can’t do anything that’s complex unless you have structure.” For creative people, that structure should be as subtle as possible.
23 Dave Maritz Test Mgr, Windows 95 Everything that I do here I learned in the military.... You can't do anything that's complex unless you have structure.... And what you have to do is make that structure as unseen as possible and build up this image for all these prima donnas to think that they can do what they like. Who cares if a guy walks around without shoes all day? Who cares if the guy has got his teddy bear in his office? I don't care.... [But if] somebody hasn't checked in his code by five o'clock, then that guy knows that I am going to get into his office.” From Microsoft Secrets
24 High-Level Process cont’d Structure: – Process must fit the product & market/customer Nature of the Product or Service –Mission critical vs. not; new vs. derivative –Infrastructure vs. applications, embedded, etc. –Packaged vs. custom, or multi-system solution Nature of the Market or Customer: –Speed & innovation for leading-edge markets –Quality & support for enterprise & mass mkts
25 Innovation & Design Strategy Being “slightly out of control” can stimulate innovative thinking and creativity. But too few controls chaos or “rocket science” Iterative a middle approach: Vision statements, outlines of functional specs, prototypes, early beta releases, multi-version release mentality, etc. Late design changes can be good: help improve the product -- respond fast to user feedback, competitors’ moves, or unforeseen changes in the market & technology.
26 Architecture Strategy Definition: –How to divide a product into subsystems and modules/components, and with what interfaces Strategy: – degree of “modular” vs. “integral” – degree of “open” vs. “closed” (proprietary) – consider/beware of “Open but not Open”! Modular & “horizontal” (feature-oriented) –de-couples small development teams –facilitates adding or cutting functionality –facilitates reuse of components
27 Architecture Strategy Tradeoff: Investment in architecture for the future versus new features for the present Try to design architectures that will last. –Rush-jobs “spaghetti” & hard to modify Or, plan to evolve architectures incrementally
28 Bob Lisbonne, Netscape VP Client Development When our teams grew beyond a certain point, they began to resemble a 200-person three- legged race … That’s why the componentization or the modularization of the product is so key, so that ultimately we can get back to lots of small teams each doing their own thing… and not getting caught up in one another’s efforts. From Competing on Internet Time
29 Team Management Strategy Small teams of great people work best Large teams can work like small teams (within limits – not Longhorn!) What you need: –One strong person in charge! –Map project architecture to product architecture feature & subsystem teams –Keep feature teams small 3-8 people. –Overlap functional responsibilities (vision/scope, requirements, QA) –Focus everyone on shipping the product!
30 Chris Peters, VP Microsoft Office Everybody in a business unit has exactly the same job description and that is to ship products. Your job is not to write code, your job is not to test, your job is not to write specs. Your job is to ship products.... When you wake up in the morning and you come to work, you say what is the focus? Are we trying to ship? Are we trying to write code? The answer is we are trying to ship.... You're trying not to write code. If we could make all this money by not writing code, we'd do it. From Microsoft Secrets
32 Project Management Strategy Avoid sequential “waterfall” schedules (usually). Divide long projects into multiple sub-projects or milestones, of a few weeks or months duration. Evolve requirements incrementally: do spec design, development & testing concurrently. Impose a few rigid rules to force frequent synchronizations and periodic stabilizations. Synchronize-and-Stabilize!
33 Project Management cont’d Prioritize Rigorously: most important features first Schedule “backwards” – people & time Let engineers schedule their own tasks Managers keep historical data on estimates Prototypes and early beta releases = rapid feedback on designs and quality Use historical data to schedule buffer time (for projects, not individuals) for changes & unknowns
35 Synch-Up and Debug Daily Microsoft Secrets: Few rules, but “military-like” discipline to force coordination & communication –Developers can check in when they want –Each project must build daily –Check-in at set times; can’t leave until build is OK –If you break the build, you must fix your code now! –Penalties for those who break the build Minimize build overhead through build team and automated tools Objectives: –Synching-up: 5-20 minutes; Quick test: 30 minutes –Total Check-in: 5-60 minutes
36 Testing and QA Strategy Try to build-in quality continuously –design and code reviews –gates or check points –continuous customer feedback, etc. But continually integrate and test –Frequent builds (daily, weekly) –Especially check late design changes Test what? –unit/feature testing, of course –system integration testing -- from early on!
37 Testing and QA cont’d Automate to test & retest design changes quickly and frequently. But automation never eliminates the need for people. Someone has to write and rewrite (update) the automated tests. Also need human testers to probe real user behavior
38 Testing and QA cont’d Post-mortems: what went well, what went poorly, and what the team should do next time. “Eat your own dog food”: first-hand feedback on products as quickly as possible. A few quantifiable metrics: to control and improve quality as well as monitor key product and process characteristics. Early beta releases: provide feedback on design as well as quality.
39 Concluding Comments No one “best” software development process –But “waterfall” less responsive to change –But using a “bundle” of practices eliminates differences in quality between waterfall and iterative! Process should depend on business, context, and strategy –Type of software, customer requirements, team experience and culture, contract needs, etc. –Product vs. custom, mass-market vs. niche, individual vs. enterprise, leading-edge vs. follower, etc.
40 Main References The Business of Software by M. Cusumano (Free Press/Simon & Schuster, 2004) Microsoft Secrets by M. Cusumano and R. Selby (Free Press/Simon & Schuster, 1995 and 1998) Competing on Internet Time by M. Cusumano and D. Yoffie (Free Press/Simon & Schuster, 1998) Michael Cusumano, Alan MacCormack, Chris Kemerer, and Bill Crandall, “Software Development Worldwide: The State of the Practice”, IEEE Software, November-December 2003. (International Comparisons) Alan MacCormack, Chris Kemerer, Michael Cusumano, and Bill Crandall, “Trade-offs between Productivity and Quality in Selecting Software Development Practices”, IEEE Software, September-October 2003. (HP Survey)