2 Overview Introduction Motivations Some vocabulary Agile methods in generalResponse to heavyweight waterfall modelMany small conceptsStories of Agile projects
3 Henning Müller Diploma in Medical Informatics University of Heidelberg ( )Daimler Benz research and technologyPortland, OR, USA ( )PhD in image analysisUniversity of Geneva ( )Medical image analysis and information systemsUniversity and Hospitals of Geneva (2002-now)HES SO (2007-now)
4 You Background Expectations from this course Practical experiences (also concerning agile methods)Expectations from this courseBrainstorming: What is Agile for you?
5 GoalsObtain a solid knowledge on modern information system development methodsAgile vs. iterative methods vs. waterfallWhy Agile can helpUnderstand how to choose the right methodology for each projectOr components of itThere is no one size that fits all
6 TextbooksKen SCHWABER, Mike BEEDLE, Agile Software development with SCRUM, Prentice Hall, 2002.Craig LARMAN, Agile and iterative development, Addison-Wesley, 2006.
7 Definition Agile methods (wikipedia) Agile software development is a conceptual framework for software engineering that promotes development iterations throughout the life-cycle of the project.There are many agile development methods; most minimize risk by developing software in short amounts of time. Software developed during one unit of time is referred to as an iteration, which may last from one to four weeks. Each iteration is an entire software project: including planning, requirements analysis, design, coding, testing, and documentation. An iteration may not add enough functionality to warrant releasing the product to market but the goal is to have an available release (without bugs) at the end of each iteration. At the end of each iteration, the team re-evaluates project priorities.Agile methods emphasize face-to-face communication over written documents. Most agile teams are located in a single open office sometimes referred to as a bullpen. At a minimum, this includes programmers and their "customers" (customers define the product; they may be product managers, business analysts, or the clients). The office may include testers, interaction designers, technical writers, and managers.
8 Definition iterative/incremental methods Incremental development is a scheduling and staging strategy in which the various parts of the system are developed at different times or rates, and integrated as they are completed. It does not imply, require nor preclude iterative development or waterfall development - both of those are rework strategies. The alternative to incremental development is to develop the entire system with a "big bang" integration.Iterative development is a rework scheduling strategy in which time is set aside to revise and improve parts of the system. It does not presuppose incremental development, but works very well with it. A typical difference is that the output from an increment is not necessarily subject to further refinement, and its' testing or user feedback is not used as input for revising the plans or specifications of the successive increments. On the contrary, the output from an iteration is examined for modification, and especially for revising the targets of the successive iterations.
9 Definition waterfall model Strict following of phasesNo way backWas often mandatoryDARPA, CIA, governmentsStaticNo changes foreseenNot realistic for most software projects
10 Critics towards the waterfall model Software projects are not as static as many other product developments (scope creep)Up front specifications may not be realisticThe final product is often not known at the startRisk in the waterfall model is pushed towards the endPlus “big bang” is risky per seA single contact with the customer is not enough
11 General problems with software development: Exercise Read the text “Standish Group Report 1995”, 1-4What are the most surprising results?Read pages 5-7Which factors are linked with software success?Why are so many software projects failing?
12 Standish Report 2000/2003 Read the one page abstract What has improved? What has not?Any propositions or explications?
13 Exercise: Read the text on agile methods and the lack of their use. Why is the waterfall model still used so broadly?What needs to be done to increase the number of agile development projects?
14 Product development Software vs. other products Cell phonesPlastic dinosaursHouses – architecturePredictable vs. managing changeEstimate efforts and costsPossible or not?Plan a schedule, order all activities, details
15 Predictable/new product manufacturing Most software is not a predictable or mass manufacturing problem. Software development is new product developmentOften new, buggy technologies are used, increasing the novelty and unpredictability of processesHow long will this version of Java be supported?
16 Prevention of detailed up front specifications Clients are not sure what exactly they wantClients have difficulty stating everythingDetails will only be revealed during the developmentDetails are very complex for peopleExternal forces (actions of competitors, economic crisis) lead to changeRequirements to act quickly in Internet economy
17 Iterative development Several iterations in sequences constitute the full development cycleEvery iteration is an independent sub (mini) projectEach iteration has analysis, design, development, testingGoal is an iteration releaseIncomplete but working prototypeSoftware across teams is integrated every release (subset of full system, no throw away)
20 Length of iteration Depends on methods Sometimes fixed (SCRUM), sometimes variableMost often recommended is 1-6 weeksDepends on total length of the projectCutting a project into smaller blocks helps to improve the success rateReduce complexitySatisfaction in finishing off something
22 Risk-driven development Reduce overall project risk early in the projectStart each iteration with the riskiest partsIf there is a real problem it is discovered quickly and can be addressedRisk is not always easy to define or estimateProbability and impact can be estimated
25 Client-driven development Choice of features for next iteration comes from the client directlyClient steers the development iteration by iterationClient has ongoing control of the project and makes most important decisions based on latest data, and not speculativelyClient is working directly with the developers
26 Timeboxing Fix end date for each iteration without change If too many features for completion, reduce the scope (improve estimates over time)Remove features with lower priorityAlways a running and tested system at the end of each iterationEntire project can be timeboxed, or only iterationsIterations can have varying length
27 Evolutionary and adaptive development Requirements, plans and solutions evolve and are refined over timeNo fully frozen specification at the startUnpredictable discoveries can occurSystem can be adapted based on feedback from users, clients, or developersInternet economy is based on quick reaction times and quick changes based on the market
28 Evolutionary requirements This does not mean that everything changes all the time!No sloppy specificationsFirst iterations have often a larger percentage of requirements analysisMost requirements are defined early on and iterations have increasingly short analysesRequirements workshops in some early iterations for larger projectsDetails on risky developments
30 Adaptive planning Concentrate on high level influential parts Architecturally importantHigh risk (load, usability, …)Initial phase of high uncertaintyMake a more detailed planning 10%-20% into the projectAfter first developments and experiencesInitial exploratory phase
31 Cowboy coding Absence of a defined method Members of the team do what they think is rightNo details on communicationThis is not agile development!Agile development follows methods and protocols, sometimes very strictly
32 Prizing Short fixed-prize exploratory iterations are possible Then larger-scale parts with possibility to stop at any momentSoftware should be produced not documentsWaterfall method is easier to pay forBut is this realistic (promises)Risk is mainly with the customer
33 Incremental/iterative delivery Running product is delivered in several iterationsDelivery cycle is usually different from the development cycleFor example 6-month delivery cycle having 6-10 short iterations per deliveryFeedback can be integrated directly into each releaseWeb environment has many projects like this
35 Common mistakes Project leader X: Sure, we do not apply the waterfall, everyone knows that it does not work. We use Method X and are in our first project.We are now into it for two months and the use case analysis is nearly finished. Then, we will plan and schedule what will be done in each iteration before the programming will start.
36 Text on success of agile methods Read the text (4 pages)What are the characteristics of the described projects?What are the main characteristics of Agile development mentioned in the text?Why are these methods successful?
37 The Agile alliance http://www.agilealliance.com/ Grouping of people active on Agile development (founded in 2001)4’200 members, many events, articles, publications available from the web sideAgile manifesto, and definition of main principles of Agile software developmentDefinitions described on their web pages
38 A little bit of history Evo – evolutionary project management 1960s: Evo and first elements of iterations etc.1986: SCRUM1996: eXtreme ProgrammingMid 1990s: tendencies to more lightweight development as result of many failures2001: Agile alliance
39 The Agile manifestoIndividuals and interactions over processes and toolsWorking software over comprehensive documentationCustomer collaboration over contract negotiationResponding to change over following a plan
40 The Agile principles (1) 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.2. Welcome changing requirements, even late in the development. Agile processes harness change for the customer’s competitive advantage.3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to a shorter time scale.
41 The Agile principles (2) 4. Business people and developers must work together daily throughout the project.5. Build projects around motivated individuals. Give them the environment and support their need, and trust them to get the job done.6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
42 The Agile principles (3) 7. Working software is the primary measure of progress.8. Agile processes promote sustainable development.9. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.10. Continuous attention to technical excellence and good design enhances agility.
43 The Agile principles (4) 11. Simplicity – the art of maximizing the amount of work not done – is essential.12. The best architectures, requirements, and designs emerge from self-organising teams.13. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.
44 Suitability of Agile methods The culture of an organisation must be supportive of negotiationPeople must be trustedFewer staff with higher level of competencyOrganisation must live with decisions the developers makeLow level of hierarchies is betterOrganisation needs to have a development that facilitates rapid communication between team members
45 A little break in the presentation … Read the text exploring agile developmentWhat are the two most important advantages of the waterfall approach and of Agile methods? What are the disadvantages?
46 So what exactly is Agile? Not a single definitionTimeboxed, iterative, incremental developmentChange is embracedPromote evolutionary deliverySimplicity, lightness, communicationSome methods common to a single practiceCommon project room, standup meetings, pair programming, …
47 Classification of Agile methods Length and number of cycles/iterationsFrom a few days to monthsCeremony or strict guidelines to followFormal steps, …Suitability for particular sorts of projectsProjects with many/few developersProjects with a long/short duration
49 Function points to measure project size FPU – Function Point analysisEstimate size of a software projectBudget application developmentDetermine project productivityComplex analysis in closed documents
50 Agile project management (Augustine et al.) Guiding Vision: Establish a guiding vision for the project and continuously reinforce it through words and actionsTeamwork and collaboration: Facilitate teamwork and collaboration through relationships and communitySimple rules: Establish and support the team’s set of guiding practices such as SCRUM or XP
51 Agile project management (2) (Augustine et al.) Open Information: Provide visible and open access to project management and other informationLight touch: Apply just enough control to foster emergent behaviour in a self-directed teamAgile vigilance: Reinforce the vision, follow or adapt the rules, listen to the people
52 People matter! Programming is human activity Happy developers work betterOverwork is not good, reduces concentration and increases errorsSustainable development is the goalProductivity of programmers varies by factor 10Education and mentoring by othersRight knowledge is important for the right task
53 Simplicity Low tech – high touch Stand up meetings Simple web tools Using paper cards, white boards, …Stand up meetingsSimple web toolsWikis etc. are changing the tool worldDevelopers should quickly see benefit of simple tools
54 Empirical vs. defined processes Defined processes have well-defined steps or ordered activitiesPredictable manufacturingEmpirical processes in high-change and instable domainsFrequent measurements and response mechanisms
55 Test-driven development Part of many Agile methodsExact definition of function tests before programming often with the customerAutomated tests of almost everythingPass or fail, no human interventionTests are re-executed after each build cycleTest suits grow with the project
56 Cockburn scale to compare projects PeopleSmallLargeCriticalityHighLow
58 Several agile methods SCRUM XP Adaptive Software Development (ASD) Dynamic Solutions Delivery Model (DSDM)Crystal ClearFeature Driven Development (FDD)Lean DevelopmentPragmatic Programming
59 StoryRead page 41-47Mention all agile principles you encounter as soon as they appear
60 Motivations for agile projects If the waterfall has good results there is no need to change it in an organisationDo not simply follow fashions but needs!!Change is usually motivated by partial failureFailure has been high in software projectsFailure can have many facesTime, budget, wrong product
61 Facts of change in software projects Uncertainty is inherent and inevitable in software projectsScope creep
62 Risk management Iterative development has lower risk Tackle the hardest problems firstProject can be stopped in case a problem is insurmountableKeep a running system with a subset of the functionality and avoid the big bang in the endPermanent testingIKIWISI – I’ll Know It When I See It
63 Timeboxing benefits Focus increases productivity Psychological focus (day before vacation)3-week milestone different than 3-month milestonePeople remember slipped dates but not necessarily slipped featuresNeed to be realistic with goals… as presented regularly to customers
66 Waterfall Made for: Complexity overload Low-change projectsLow-novelty projectsLow-complexity projectsComplexity overloadDoD in the US changed the waterfall dogma in 1994
67 Estimation of complexity - Wideband Delphi Method to obtain estimations for projectsHave several developers make estimations independently (sometimes anonymously)Kickoff togetherEstimation separatelyMeeting to discuss results with a facilitatorEstimate=(Optimistic+Pessimistic+4*MostLikely)/6LikelyDeviation= (Pessimistic-Optimistic)/6
68 Studies on software practices Many studies exist:Less than 5% of source code is actually usedSoftware pollutionProductivity increases when releasing products/prototypes earlyVery successful projects have fewer roles than average
69 Productivity research Small projects are often more productiveIterative projects are more productive by cutting a big project into piecesTimeboxing increases productivity by focussing the scopeComplexity of a project decreases productivity
71 Software qualityNumber of defects per function point becomes often larger in large projectsTests and evaluations increase software qualityTest early, test oftenShort time between coding and testing can decrease errorsEarly contact with users can help finding errors
72 Abbreviations ASD - Adaptive Software Development CIA - Central Intelligence AgencyDARPA - Defence Advanced Research Project AgencyDSDM - Dynamic Solutions Delivery ModelFDD - Feature Driven DevelopmentFPU - Function Point UnitsIS - Information SystemSCRUM - no abbreviation, term from RugbySOA - Service Oriented ArchitecturesSURF - Standish User Research ForumUML - Unified Modeling LanguageXP - eXtreme Programming