INNOV-9: Adventures in Development Methodologies there’s no subtitle like a good subtitle Gus Björklund Wizard, Progress Software Corporation

© 2007 Progress Software Corporation 2 INNOV-9: Adventures in Development Methodologies Adobe’s (old) Development Process  Waterfall method Specify desired features, set ship date Work on features to “feature complete” date Fix bugs Ship beta Revise features based on beta tests Fix more bugs Release PhotoShop development

© 2007 Progress Software Corporation 3 INNOV-9: Adventures in Development Methodologies the Klingon way What is this talk of 'release'? Klingons do not make software 'releases'. Our software 'escapes' leaving a bloody trail of designers and quality assurance people in its wake. Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are!

© 2007 Progress Software Corporation 4 INNOV-9: Adventures in Development Methodologies Adobe’s results  Death-march to get features in by date work nights and weekends as deadline looms  Lots of bugs at feature complete  Fix worst bugs before beta ship  No time to revise features based on feedback  Death-march to get enough bugs fixed by final freeze date work nights and weekends as deadline looms  Buggy product PhotoShop development reference: blogs.adobe.com

© 2007 Progress Software Corporation 5 INNOV-9: Adventures in Development Methodologies

© 2007 Progress Software Corporation 6 INNOV-9: Adventures in Development Methodologies This Just In … “The digital signs along the 98 B-line between downtown Vancouver and Richmond are supposed to let people waiting at the bus stop know when the next bus will arrive. The signs, which are linked to a GPS system on the buses, haven't been working for the past week, freezing up and requiring frequent reboots.... Siemens has basically thrown up its hands and say they can't make it work.” CBC News, March 8, 2007

© 2007 Progress Software Corporation 7 INNOV-9: Adventures in Development Methodologies Topics History Failures Agile Methods Results More Important Things Call to Action

© 2007 Progress Software Corporation 8 INNOV-9: Adventures in Development Methodologies A Bit of History “Those who fail to learn the lessons history teaches are doomed to repeat the mistakes of the past.”

© 2007 Progress Software Corporation 9 INNOV-9: Adventures in Development Methodologies Early days of programming: “Cut and Try”  code some  test some  fix  code some more  test, fix  lather, rinse, repeat

© 2007 Progress Software Corporation 10 INNOV-9: Adventures in Development Methodologies Programs got bigger. Chaos ensued

© 2007 Progress Software Corporation 11 INNOV-9: Adventures in Development Methodologies COBOL was invented. Business people could create their own applications and programmers were no longer needed. order was restored.

© 2007 Progress Software Corporation 12 INNOV-9: Adventures in Development Methodologies Fred Brooks: “The Mythical Man Month”

© 2007 Progress Software Corporation 13 INNOV-9: Adventures in Development Methodologies Harlan Mills: Top-Down Programming

© 2007 Progress Software Corporation 14 INNOV-9: Adventures in Development Methodologies Edsger Dijkstra: “Goto considered harmful”

© 2007 Progress Software Corporation 15 INNOV-9: Adventures in Development Methodologies 1970: Winston Royce: The “Waterfall Model” which became US DoD STD-2167 and other standards

© 2007 Progress Software Corporation 16 INNOV-9: Adventures in Development Methodologies The “waterfall model” Requirements DesignCodingIntegrationTest / DebugInstallationMaintenanceAnalysis

© 2007 Progress Software Corporation 17 INNOV-9: Adventures in Development Methodologies Order was restored.

© 2007 Progress Software Corporation 18 INNOV-9: Adventures in Development Methodologies Digression: A Survey  Who uses a formal development process?  Who uses waterfall?  Who uses DOD-2167?  Who uses something else?

© 2007 Progress Software Corporation 19 INNOV-9: Adventures in Development Methodologies Back to our regularly scheduled program

© 2007 Progress Software Corporation 20 INNOV-9: Adventures in Development Methodologies Order was restored.

© 2007 Progress Software Corporation 21 INNOV-9: Adventures in Development Methodologies Waterfall assumes that  A reasonably well defined set of requirements if we take the time to understand them  Change will be small and manageable  Integration will go well  The schedule can be met

© 2007 Progress Software Corporation 22 INNOV-9: Adventures in Development Methodologies Waterfall assumes that  A reasonably well defined set of requirements if we take the time to understand them  Change will be small and manageable  Integration will go well  The schedule can be met

© 2007 Progress Software Corporation 23 INNOV-9: Adventures in Development Methodologies Development “Failures”  : OS/360  : US Air traffic control system  1995: Denver airport baggage handling system  : MS SQL Server 2005  1988: Dbase IV  1994: Progress Version 7  2004: Sainsbury PLC supply chain system  2007: TurboTax online tax filing  : Vista  Many, many more.

© 2007 Progress Software Corporation 24 INNOV-9: Adventures in Development Methodologies Facts  $ 89 Billion spent on cancelled software projects  $ 59 Billion more on cost overruns  Of the challenged or cancelled projects, the average project was 189 % over budget was 222 % behind schedule had 61 % of planned features delivered source: Standish Group CHAOS Chronicles That was in The situation has not improved.

© 2007 Progress Software Corporation 25 INNOV-9: Adventures in Development Methodologies Why Is Software Development So Hard?

© 2007 Progress Software Corporation 26 INNOV-9: Adventures in Development Methodologies Common Causes 1. Unrealistic or unarticulated project goals 7. Use of immature technology 2. Inaccurate estimates of needed resources 8. Inability to handle the project’s complexity 3. Badly defined system requirements 9. Sloppy development practices 4. Poor reporting of project’s status 10. Poor project management 5. Unmanaged risks11. Stakeholder politics 6. Poor communication among customers, developers, and users 12. Commercial pressures “Why Software Fails”, Robert N. Charette. IEEE Spectrum

© 2007 Progress Software Corporation 27 INNOV-9: Adventures in Development Methodologies How Can We Make It Less Hard?

© 2007 Progress Software Corporation 28 INNOV-9: Adventures in Development Methodologies QUIZ: What is the 1 most important thing we need in developing software?

© 2007 Progress Software Corporation 29 INNOV-9: Adventures in Development Methodologies How do we avoid failures like the ones that we all know about?

© 2007 Progress Software Corporation 30 INNOV-9: Adventures in Development Methodologies Agile development methods to the rescue!

© 2007 Progress Software Corporation 31 INNOV-9: Adventures in Development Methodologies The Manifesto for Agile Software Development, 17 anarchists agree by Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas Feb 13, 2001

© 2007 Progress Software Corporation 32 INNOV-9: Adventures in Development Methodologies Agile manifesto: We follow the following principles:  Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.  Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.  Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.  Business people and developers work together daily throughout the project.  Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.  The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.  Working software is the primary measure of progress.  Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.  Continuous attention to technical excellence and good design enhances agility.  Simplicity—the art of maximizing the amount of work not done—is essential.  The best architectures, requirements and designs emerge from self-organizing teams.  At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

© 2007 Progress Software Corporation 33 INNOV-9: Adventures in Development Methodologies Agile Manifesto - values  Individuals and interactions over processes and tools.  Working software over comprehensive documentation.  Customer collaboration over contract negotiation.  Responding to change over following a plan. That is, while we value the items on the right, we value the items on the left more. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

© 2007 Progress Software Corporation 34 INNOV-9: Adventures in Development Methodologies How are agile methods different?  Emphasizes direct communication  Focuses on working software  Raises the level of skill

© 2007 Progress Software Corporation 35 INNOV-9: Adventures in Development Methodologies Agile approaches  Chief programmer teams  XP  Scrum  Gus’s method

© 2007 Progress Software Corporation 36 INNOV-9: Adventures in Development Methodologies Chief programmer teams  Team consists of Chief programmer Backup chief programmer Librarian Programmers Project administrator Toolsmith Testers An old idea that shouldn’t be forgotten F.T. Baker, IBM, late 60’s

© 2007 Progress Software Corporation 37 INNOV-9: Adventures in Development Methodologies Agile approaches  Chief programmer teams  XP  Scrum  Gus’s method

© 2007 Progress Software Corporation 38 INNOV-9: Adventures in Development Methodologies Extreme Programming (XP)  Communication  Simplicity  Feedback  Courage  Respect XP Values:

© 2007 Progress Software Corporation 39 INNOV-9: Adventures in Development Methodologies XP Summarised  Team of 5 to 10 work at one location with customer representation  Development occurs in frequent iterations each is releasable each delivers more functionality  Requirements specified as “user stories” each is a chunk of functionality user needs  Programmers work in pairs  Requirements, architecture, and design emerge over the course of the project

© 2007 Progress Software Corporation 40 INNOV-9: Adventures in Development Methodologies XP Summarised  In an iteration Write automated tests first Coding, by pairs of programmers –follow strict coding standards –complete when  all tests pass  programmers can’t think of any more needed tests Refactor Demo

© 2007 Progress Software Corporation 41 INNOV-9: Adventures in Development Methodologies XP works best for  New or prototype technology requirements change frequently  Research projects goal is knowledge gained, not product  Smaller projects

© 2007 Progress Software Corporation 42 INNOV-9: Adventures in Development Methodologies Agile approaches  Chief programmer teams  XP  Scrum  Gus’s method

© 2007 Progress Software Corporation 43 INNOV-9: Adventures in Development Methodologies Scrum Summary  Timebox everything  Work from “backlogs” Product backlog Sprint backlog  Short cycles  Small teams

© 2007 Progress Software Corporation 44 INNOV-9: Adventures in Development Methodologies Scrum: Teams  Team of 5 to 9 people  One member serves as “Scrum Master” to Facilitate meetings Manage outside interference Record decisions, track action items Keep everyone informed

© 2007 Progress Software Corporation 45 INNOV-9: Adventures in Development Methodologies Scrum: phases  Review release plans  Distribution, review and adjustment of product standards  “Sprints” lasting about 4 weeks  Repeat as many times as necessary  Closure

© 2007 Progress Software Corporation 46 INNOV-9: Adventures in Development Methodologies Product backlog IDNameImpEstHow to demoNotes 1Deposit305Log in, open deposit page, deposit €10, go to my balance page and check that it has increased by €10. Need UML sequence diagram. No need to worry about encryption for now. 2See your own transaction summary 109Log in, click on “transactions”. Do a deposit. Go back to transactions, verify the new transaction shows up Use paging to avoid large queries. Design similar to view users page Source: “ Scrum and XP from the Trenches ”, Henrik Kniberg

© 2007 Progress Software Corporation 47 INNOV-9: Adventures in Development Methodologies Scrum: sprints  Plan  Develop  Wrap  Review  Adjust

© 2007 Progress Software Corporation 48 INNOV-9: Adventures in Development Methodologies Scrum: Team meetings  Daily, always at same time  15 minutes  3 questions only, by Scrum master What have you done since last meeting? What impedes your work? What will you do by next meeting?  Follow-up meetings held after, if needed

© 2007 Progress Software Corporation 49 INNOV-9: Adventures in Development Methodologies Scrum: advantages  Product becomes series of manageable chunks  Progress made, even if requirements not stable  Everything visible to everyone  Communication improves  Success shared along the way  Customers see how product actually works  Strong customer relationships develop  Culture created where everyone expects project to succeed

© 2007 Progress Software Corporation 50 INNOV-9: Adventures in Development Methodologies Agile approaches  Chief programmer teams  XP  Scrum  Gus’s method

© 2007 Progress Software Corporation 51 INNOV-9: Adventures in Development Methodologies Gus’ Method  Think  Think more  Think still more  Write code  Test, fix  Write code again, test  Doc Not recommended for general use. Works for me.

© 2007 Progress Software Corporation 52 INNOV-9: Adventures in Development Methodologies Adobe’s Recent Experience with a new development process

© 2007 Progress Software Corporation 53 INNOV-9: Adventures in Development Methodologies Adobe: the new process  Incremental development Work on a few features at a time, finish them before moving on  Any dev with 20 bugs can’t work on features  Only complete features can be merged  Ready to ship within short notice some features may not be there everything that is there is complete and works  More beta cycles PhotoShop CS3

© 2007 Progress Software Corporation 54 INNOV-9: Adventures in Development Methodologies Adobe: results with new dev process  Higher quality Fewer bugs overall Fewer bugs in mid cycle Much more useful (and earlier) beta cycles  More predictable dev cycles  Can demo product at almost any stage  No more nights and weekends PhotoShop CS3

© 2007 Progress Software Corporation 55 INNOV-9: Adventures in Development Methodologies The question I know you are all wanting to ask:

© 2007 Progress Software Corporation 56 INNOV-9: Adventures in Development Methodologies Does this stuff actually work?  Yes. Agile methods succeed at Fidelity, VA Software, Medtronic, Adobe, IBM, Sapient, Yahoo, SAP, BMC Software, Symantec, Verizon, Microsoft, others  Results can be excellent  Agile scales  Agile methods will NOT solve every problem.

© 2007 Progress Software Corporation 57 INNOV-9: Adventures in Development Methodologies Agile Works Standish Group: 16 percent of waterfall projects succeed 41 percent of agile projects succeed !!! (2006 Chaos report) Forrester: Agile works. 17% of companies using agile. (2006 Agile Adoption Survey)

© 2007 Progress Software Corporation 58 INNOV-9: Adventures in Development Methodologies Agile Development Myths  No process  No documentation  Only works for small projects  Only works for small teams  Chaotic

© 2007 Progress Software Corporation 59 INNOV-9: Adventures in Development Methodologies Agile Development Myths  No process  No documentation  Only works for small projects  Only works for small teams  Chaotic

© 2007 Progress Software Corporation 60 INNOV-9: Adventures in Development Methodologies "A great lathe operator commands several times the wage of an average lathe operator, but a great writer of software code is worth 10,000 times the price of an average software writer.” Bill Gates There’s something else to consider …

© 2007 Progress Software Corporation 61 INNOV-9: Adventures in Development Methodologies Methodology/process isn’t everything  Huge difference between best and worst 10 x and more variance in productivity 10 x and more variance in quality  Motivation  Experience  Work environment Top performers have quiet work area and few interruptions People are the most important thing

© 2007 Progress Software Corporation 62 INNOV-9: Adventures in Development Methodologies Call to Action  Try something on a small scale first Share your stories  A given methodology may not suit your or your development teams  Adapt to suit your needs  Adapt and adopt, based on your results  Read this: “Scrum and XP from the Trenches”, Henrik Kniberg

© 2007 Progress Software Corporation 63 INNOV-9: Adventures in Development Methodologies Homework  Books “The Mythical Man Month”, Fred Brooks “Scaling Software Agility”, Dean Leffingwell “Agile Software Development”, Alistair Cockburn  Paper “Chief Programmer Team Management of Production Programming”, F.T. Baker, IBM Systems Journal, Vol 11, No 1, 1972  Wikipedia articles ( Agile software development Waterfall Method Extreme Programming Software development process

© 2007 Progress Software Corporation 64 INNOV-9: Adventures in Development Methodologies Answers