2 From Nothing, to Heavy, to Light (1) Most software development is a chaotic activity, often characterized by the phrase “code and fix”.We’ve lived with this style of development for a long time, but we’ve also had an alternative for a long time: methodology.The most frequent criticism of these methodologies is that they are bureaucratic (hence the name heavy methodologies).We’ve lived with this style of development for a long time, but we’ve also had an alternative for a long time: methodology. Methodologies impose a disciplined process upon software development with the aim of making software development more predictable and more efficient.The most frequent criticism of these methodologies is that they are bureaucratic. There’s so much stuff to do to follow the methodology that the whole pace of development slows down. Hence they are often referred to as heavy methodologies.
3 From Nothing, to Heavy, to Light (2) As a reaction to these methodologies, a new group of methodologies have appeared in the last few years – the light methodologies.They are less document-oriented and more code-oriented.Lack of documentation is just a symptom of two deeper differences:Light methods are adaptive rather than predictive.Light methods are people-oriented rather than process‑oriented.As a reaction to these methodologies, a new group of methodologies have appeared in the last few years. Although there’s no official name for them, they are often referred as light methodologies – signaling a clear reaction to the heavyweight methodologies.Light methods are adaptive rather than predictive. Heavy methods tend to try to plan out a large part of the software process in great detail for a long span of time, this works well until things change. So their nature is to resist change. The light methods, however, welcome change. They try to be processes that adapt and thrive on change, even to the point of changing themselves.Light methods are people-oriented rather than process‑oriented. They explicitly make a point of trying to work with peoples’ nature rather than against them and emphasize that software development should be an enjoyable activity.
4 The Self-Adaptive Process Adaptivity is not only in the context of a project adapting its software frequently to meet the changing requirements of its customers.There’s another angle of adaptivity: that of the process changing over time.The first part of self-adaptivity is regular reviews of the process.A consequence of self-adaptivity is that you should never expect to find a single corporate methodology.There’s another angle of adaptivity: that of the process changing over time. A project that begins using an adaptive process won’t have the same process a year later. Over time, the team will find what works for them, and alter the process to fit.The first part of self-adaptivity is regular reviews of the process. Usually you do these with every iteration. At the end of each iteration, have a short meeting and ask yourself the following questions: What did we do well? What have we learned? What can we do better? What puzzles us? These questions will lead you to ideas to change the process for the next iteration. In this way a process that starts off with problems can improve as the project goes on, adapting better to the team that uses it.
5 Agile Processes Introduction to Agile Methodology Agile Communication Agile ManifestoImpact of Agile on RequirementsPrinciples & PracticesAgile CommunicationSweet SpotsSelf AdaptationSpecific MethodsXPCrystal
6 Welcome to Agile Techniques What Agile processes try to avoid:
8 Principles behind the Agile Manifesto We follow these principles:Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.Business people and developers must work together daily throughout the project.
9 Principles behind the Agile Manifesto (continued)Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.Working software is the primary measure of progress.Agile processes promote sustainable development.The sponsors, developers, and users should be able to maintain a constant pace indefinitely..
10 Principles behind the Agile Manifesto (continued)Continuous attention to technical excellence and good design enhances agility.Simplicity--the art of maximizing the amount of work not done--is essential.The best architectures, requirements, and designs emerge from self-organizing teams.At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
11 Agile Methodologies XP (eXtreme Programming) Crystal DSDM (Dynamic Systems Development Method)SCRUMASD (Agile Software Development)And others…“Agile Methodologies” not only XP
12 Impact of Agile on Requirements Traditional software convention is based on the assumption that:A system should be clearly specified before beginning design and implementation.A specification should be unambiguous, consistent, concise, traceable and implementation independent.How is requirements engineering special in agile?- Different assumptions- Special Environmental conditions
13 … Impact of Agile on Requirements Requirements and models are a valid expression of user needsAll requirements have equal valueCost of changing requirements becomes more expensive as a project progresses.Expression : they are able to verify that what they want is captured on paper.By value we mean that all requirements will bring the same business value to the system, not the same as priority.
14 … Impact of Agile on Requirements But agile processes have little upfront analysis/design, minimal documentation and little structure for ensuring consistency and traceability, how can it work effectively as a software methodology?After all, reports show that poor requirements gathering is responsible for 45% of cancelled projects.Common criticisms.
15 … Impact of Agile on Requirements Of delivered systems, most did not provide the expected business value.Agile approaches allow requirements which provide the most value to the business emerge during development.Most users cannot intelligently respond to only requirements and models.Agile assumptions
16 … Impact of Agile on Requirements Agile processes focus on business value :Business value = f(cost,time,functionality,quality)Example : DSDM supposes that 80% of a systems value is delivered by 20% of a systems requirements.
17 … Impact of Agile on Requirements New development tools, testing tools, source management tools and version control tools allow for solid development environments which support the ability to rapidly generate high quality re-factored releases of a system.Agile approaches have emerged in the past ten years.tools can even generate certain diagrams from codeno need to maintain diagrams manually.
18 … Impact of Agile on Requirements To be successful, Agile techniques must possess tools to cope with a constantly changing and poorly defined environment.Communication must be strongly supported, encouraged and practiced by all participants.Self evaluation and modification is vital.Other engineering techniques are in same environmentThey try to control itAgile works with it
19 Agile Principles Assume Simplicity Embrace Change Software is Your Primary GoalEnabling the Next Effort is your Secondary GoalIncremental ChangeMaximize Stakeholder InvestmentModel with a PurposeAssume SimplicitySimplest solution is bestNo extras until neededEmbrace Changerequirements changePeople changeUnderstanding changesSoftware is Your Primary GoaldocumentsManagement processesModelsAll means to an end, software is that endEnabling the Next Effort is your Secondary GoalProject can still be failure even if successfully deliveredPart of the game is to set up the next gameIncremental Changeunlikely you will get it right the first timeDetail is not a goalCan always throw out incomplete model or add to oneMaximize Stakeholder Investmentuse customer resources as best as possibleEmpower the customer to make certain decisionsModel with a Purposegain understandingCommunicate ideas to othersCreate documentation for next team
20 … Agile Principles Multiple Models Quality Work Rapid Feedback Travel LightMultiple Modelseach job will likely need multiple models to describe itWhich ones are needed is project specificQuality Workneeded to be sustainable (refactoring)Needed for durabilityNeeded for debugging/maintenance/upgradingRapid FeedbackCriticalEliminates miscommunication and wasted effortTravel LightEvery time you decide to keep a model you trade-off agility for the convenience of having that information available to your team in an abstract manner (hence potentially enhancing communication within your team as well as with project stakeholders). Never underestimate the seriousness of this trade-off. Someone trekking across the desert will benefit from a map, a hat, good boots, and a canteen of water they likely won't make it if they burden themselves with hundreds of gallons of water, a pack full of every piece of survival gear imaginable, and a collection of books about the desert. Similarly, a development team that decides to develop and maintain a detailed requirements document, a detailed collection of analysis models, a detailed collection of architectural models, and a detailed collection of design models will quickly discover they are spending the majority of their time updating documents instead of writing source code.
21 Agile Practices Active Stakeholder Participation Apply the Right Artifact(s)An artifact is a modeling technique (graphical or textual) used during application developmentCollective OwnershipConsider TestabilityCreate Several Models in ParallelActive Stakeholder ParticipationAcceptanceFeedbackInvestmentRelationshipApply the Right Artifact(s)know what to use whenKnow various techniquesCollective Ownershipget everyone to buy inSharing infoSharing responsibilityImplementing suggestionsConsider Testabilitycan’t test it? Why doing it?Create Several Models in Parallelone model cannot capture allOne model may not convey something clearly, acurately
22 … Agile Practices Create Simple Content Depict Models Simply Display Models PubliclyIterate to Another ArtifactModel in Small IncrementsModel with OthersProve it with CodeUse the Simplest ToolsCreate Simple Contentdo not add future featuresSimplify models, architectures when possibleDepict Models SimplyDo not put too much detail in one diagramMake series, with each going into more detail.Display Models PubliclyWOW, modeling wallOpen honest communicationCan be virtual but must be accessibleIterate to Another Artifactif you get stuck, go to another artifactModel in Small Incrementsincreases agilityDeliver product more oftenAllows for changes in direction without large lossesModel with Othersmodel with purpose – often model for someoneOften need the input of the people you will be modeling forProve it with Codewill it work?Use the Simplest Toolsuse simple tools – paper, whiteboard, cardsIncreases participation and rapid feedback
24 Sweet Spots Two to Eight People in One Room Onsite Usage Experts One-Month IncrementsFully Automated Regression TestsExperienced DevelopersDrastically increase the effectivenessRequirementsPlanningDesignCodingMention that these vary…
25 Self AdaptationTraditional Software techniques, if at all, rarely do post-evaluationsAgile techniques use on-the-fly self adjustment to fine tune the methodology to the project.Communication, trust, relevance and openness is key to success in agile processes.Changing the team, the tools and the goals.
26 A Self Adaptation Technique When ?Right nowAt the start of the projectIn the middle of the first incrementAfter each incrementIn the middle of subsequent increments
27 A Self Adaptation Technique Ask to see one sample of each work product produced.Ask for a short history of the projectAsk what should be changedAsk what should be repeatedIdentify prioritiesFind Holes
28 Adaptation Conclusion Post project Review ?Reflection Workshops.Most important rule of stabilizing and maintaining communication in agile processes :BOTHER TO REFLECT
29 Specific MethodsSpecific Agile methods and how they deal with requirementsXP (eXtreme Programming)Crystal
32 ContextThe roots of XP lie in the Smalltalk community, and in particular the close collaboration of Kent Beck and Ward Cunningham in the late 1980’s.The crucial step from informal practice to a methodology occurred in the spring of 1996:The Chrysler C3 project (Chrysler Comprehensive Compensation).The roots of XP lie in the Smalltalk community, and in particular the close collaboration of Kent Beck and Ward Cunningham in the late 1980’s. Both of them refined their practices on numerous projects during the early 90’s, extending their ideas of a software development approach that was both adaptive and people-oriented.The crucial step from informal practice to a methodology occurred in the spring of Kent was asked to review the progress of a payroll project for Chrysler. The project was being carried out in Smalltalk by a contracting company, and was in trouble. Due to the low quality of the code base, Kent recommended throwing out the entire code base and starting from scratch his leadership. The result was the Chrysler C3 project (Chrysler Comprehensive Compensation) which since became the early flagship and training ground for XP.
33 Values Communication Simplicity Feedback Courage ... Respect Values. A value in the XP sense is a description of “how software development should feel”.The value of communication represents the XP belief that communication between project members is key to a successful project (and hence needs to be supported by practices). Communication is viewed as the main enabler of coordination and collaboration in a project. In the XP sense, communication always means verbal communication.The value of simplicity represents the XP belief that you should not invest into the future but only into your immediate needs. If future needs materialize as immediate needs at some point in the future, the proper application of XP practices will have put developers into a situation that lets them successfully cope with the new need. Underlying this value is the assumption that the future can not be reliably predicted and that taking care of it today is economically unwise.The value of feedback represents the XP belief that it is important to have a running system at any time that gives developers reliable information about its functioning (here feedback is not feedback between humans but rather feedback about the development state). Effectively, the system and its code base serve as the incorruptible oracle to report about the progress and state of development. Feedback serves as a means for orientation and deciding where to go.The value of courage represents the XP belief that humans are the single most significant aspect of software development. It is human courage that solves problematic situations and lets a team leave local optima behind to reach greater goals. The value of courage represents XP’s fundamental trust in humans to make a project succeed.Respect. If members of a team don’t care about each other and what they are doing, XP is doomed.
34 Fundamental Principles Rapid feedbackAssume simplicityIncremental changeEmbracing changeQuality workThe values are too vague to give us much help in deciding which practices to use. We need to distill the values into concrete principles we can use.Principles. A principle in XP is something that we use to determine whether a practice is something that can be used successfully in an XP context.Rapid feedback. Time between an action and its feedback is critical to learning. So, one of the principles is to get feedback, interpret it, and put what is learned back into the system as quickly as possible. The business learns how the system can best contribute, and feeds back that learning in days or weeks instead of months or years. The programmers learn how best to design, implement and test the system, and feed back that learning in seconds or minutes instead of days, weeks, or months.Assume simplicity. Treat every problem as if it can be solved with ridiculous simplicity. In many ways, this is the hardest principle for programmers to swallow. We are traditionally told to plan for the future, to design for reuse. Instead, XP says to do a good job (tests, refactoring, communication) of solving today’s job today and trust your ability to add complexity in the future where you need it.Incremental change. Big changes made all at once just don’t work. Any problem is solved with a series of the smallest changes that make a difference.Embracing change. The best strategy is the one that preserves the most options while actually solving your most pressing problem.Quality work. Everybody likes doing a good job. If you don’t enjoy your work, you don’t work well, and the project goes down the drain.
35 Activities Coding Testing Listening Designing Testing. While all processes mention testing, most do so with a pretty low emphasis. However XP puts testing at the foundation of development, with every programmer writing tests as they write their production code. The tests are integrated into a continuous integration and build process which yields a highly stable platform for future development.Design. Designing is creating a structure that organizes the logic in the system.
36 Practices (1/2) The Planning Game Small Releases Metaphor Simple DesignTestingRefactoringPractices. A practice in XP is a technique that project members use to successfully carry out any of the aforementioned activities.The Planning Game. Quickly determine the scope of the next release by combining business priorities and technical estimates. As reality overtakes the plan, update the plan.Small Releases. Put a simple system into production quickly, then release new versions on a very short cycle.Metaphor. Guide all development with a simple shared story of how the whole system works.Simple Design. The system should be designed as simply as possible at any given moment. Extra complexity is removed as soon as it is discovered.Testing. Programmers continually write unit tests, which must run flawlessly for development to continue. Customers write tests demonstrating that features are finished.Refactoring. Programmers restructure the system without changing its behavior to remove duplication, improve communication, simplify, or add flexibility.
37 Practices (2/2) Pair Programming Collective Ownership Continuous Integration40-Hour WeekOn-Site CustomerCoding StandardsPair Programming. All production code is written with two programmers at one machine.Collective Ownership. Anyone can change any code anywhere in the system at any time.Continuous Integration. Integrate and build the system many times a day, every time a task is completed.40-Hour Week. Work no more than 40 hours a week as a rule. Never work overtime a second week in a row.On-Site Customer. Include a real, live user on the team, available full-time to answer questions.Coding Standards. Programmers write all code in accordance with rules emphasizing communication through the code.
38 Strategies Management Strategy Facilities Strategy Splitting Business and Technical ResponsibilityPlanning StrategyDevelopment StrategyDesign StrategyTesting StrategyStrategies. Strategies describe experiences and heuristics of how to successfully execute practices in the real world.
39 XP: User Stories One thing the customer wants the system to do 1-5 programming weeksShould be testable
41 XP: Types of Risk addressed Scheduling slipsProject canceledSystem goes sourDefect rateBusiness misunderstoodBusiness changesFalse feature richExplanation of each how XP addresses them:1) Scheduling slipsRisk: Unable to complete by delivery date; have to delay delivery2) Project cancelledRisk: after numberous slips, project gets cancelled. Never implemented3) System goes sourRisk: put into production, but after a time cost of changes or defect rate make it necessary to replace the system4) Defect rateRisk: software put into production, but defect rate is so high it never gets used5) Business misunderstoodRisk: software produced, but it does not solve the business problem it was designed fori.e. req’ts error6) Business changesRisk: software produced, but business problem it was designed to solve six months ago has been replaced by a more pressing problem7) False feature richRisk: many interesting features, few of which make any money for the customerThe ones that are related to requirments are (and illustrated in the story)…XP attempts to reduce the risks of these by:Scheduling slipsShort release cycles => scope of a slip is limited.Highest priority features 1st => normally features that slip past are lower priority; can wait2) CancelationIf client has direct input and can prioritize the req’ts, still get business value up frontCustomer is part of the teamBoth customer and team learn more about software/needs as we progressShorter release cycle => less change during development of 1 releaseCustomer can substitute new functionality for functionality not yet completedHighest priority tasks 1stXP contract:Can make it based on passing tests based on stories rather than implementation of req’tsi.e. get paid in full if 80% of the tests pass, 20% less if only 60%, etc…OR can make it based on some type of monthly burn rate…
42 XP: Strengths Negotiation and Cooperation, not Adversarial Customer InvolvementCustomer OwnershipIN TERMS OF REQ”TSCust Inv - availabilty of customers/experts minimizes delay from question to answerCust own - customers take ownership of the process by providing req’ts actively through user storiesmore confident in the outcome
43 XP: Weaknesses Requires a lot of customer’s time Absence of documentationRestriction to small teamsAllegiance to current projectIN TERMS OF REQ”TSrequires a lot of the customer’s timeNo doc – hard for new members to get up to speedPair programming helps though…Small teams onlyNeeds tweaking to work for larger ones, but sometimes canAllegience to current projectTacit knowledgeFuture maintenance – what if old team is gone?Maybe video tape to help?
44 One reason XP avoid work products (like UML diagrams)…
46 Crystal: Goals & Core values “Software development is viewed as cooperative game of intervention and communication, with a primary goal of delivering useful, working software and setting up the next game”Cockburn, 2001Family of shrink-to-fit, human-powered software development methodologiesMeant to be “ultralight”- Reduces paperwork
47 The Crystal Family Criticality Number of people involved . . . Life Prioritized for Legal LiabilityPrioritized for Productivity & ToleranceLife(L)C6C20C40C100C200C500C1000D6D20D40D100D200D500D1000E6E20E40E100E200E500E1000L6L20L40L100L200L500L1000(defects cause loss of...)CriticalityEssentialmoney(E)Discretionarymoney(D)Comfort(C)1 - 6- 20- 40- 100- 200- 500- 1,000Crystal ClearNumber of people involved+20%
48 The Crystal Family of Methodologies Crystal ClearThe Crystal Family of MethodologiesThe Crystal family of methodologies is predicated in two things:Capturing the elements that successful projects repeatedly cite as cause for their successGive the team members the most latitude possible in doing their workDifferent kinds of projects require different kinds of methodologies. Typically, projects differ in:The number of people in the projectThe consequences of errorsAlistair Cockburn has been working on methodology ever since he was tasked by IBM to write about methodology in the early 90’s. His approach is unlike most methodologists, however. Instead of building on solely personal experience to build a theory of how things should be done, he supplements his direct experience with actively seeking to interview projects to see how they work.
49 Crystal ClearWhat’s Crystal Clear?The Crystal Clear methodology addresses 4‑6 person projects.Pretends to be the lightest full methodology that will still produce good results.Recommendations from project members on these types of projects include:Reduce the bureaucratic loadIncrease communicationIncrease the rate of feedback back to the team
50 Values Strong on communications. Light on deliverables. Crystal ClearValuesStrong on communications.Light on deliverables.Keep the communications paths short, rich and informal.Deliver frequently.Reduce overhead.More deliveries and better communication reduce the need for intermediate work products.
51 In compliance with Crystal Clear (1/2) You have short, rich communication paths.You deliver increments regularly, no longer than 3 months apart.You have a real user on the project, even if only part time.You have a project mission statement, you are using usage-based requirements, probably use cases, and you have a description of the system design.You have short, rich communication paths. That means you visit each other, draw on each other’s whiteboards (or better yet, printing whiteboards), look over each other’s shoulders at your programs and tests. You are sitting very close together, preferably one big room. You might be sitting in adjacent rooms, if there are really only 4-6 of you and you visit each other a lot. Recognize that the quality of communication drops the moment you have to pick up the phone or walk out the door.You deliver increments regularly, no longer than 3 months apart. You schedule and track the project by milestones, preferably code execution milestones, not by document completion milestones.You have a real user on the project, even if only part time. Your user helps you construct your screen sketches and both validates and breaks your UI designs. You get the system reviewed by real users al least once per increment prior to delivering it.You have a project mission statement, you are using usage-based requirements, probably use cases, and you have a description of the system design using one of the many description techniques you can invent.
52 In compliance with Crystal Clear (2/2) You have a clear ownership model for your work products.There is a regression testing framework and you have regression tests for your system. You do some form of code peer reviewing.You have a clear ownership model for your work products. For each class or module or use case, you know who it is that can change, update or, most importantly, delete parts of it. The answer may be either a single person, anyone in a given sub-team, or, rarely but possibly, anyone on the team. You should never have to have the whole team sit around the screen and ask, “Who put this in here? What is it doing here? Can we delete it?” If you have to do this, you know you have a breakdown in the ownership model.There is a regression testing framework and you have regression tests for your system. You do some form of code peer reviewing. It might be daily, weekly or per increment.
53 Crystal Clear (2) Workproducts: release sequence schedule of user viewingsuse cases or feature descriptionsdesign sketchescommon object modelrunning codemigration codetest casesuser manualEach of these is officially “required”, but not strictly definedCan take one out if not neededProject documentationRequired, but not definedCan be simply be pics of whiteboard drawingsUse cases written by business owners
55 Crystal: Requirements Techniques Use CasesCommunication toolsAnything from other methods…Use cases written by business ownersUnderstanding that develpers need background knowledge and terminology to undertand them.Will not magically understand req’ts from use casesCase studies of using Crystal included:user storiesPrioritizing cards and affinity analysisJAD
56 Crystal: Strengths Scalable Variable project risk Adaptable Based on human communication
57 Crystal: Weaknesses Lack of defined structure Does not optimize production or documentation
59 Agile and SASD Agile SASD Business value for the customer/user Develop as quickly and cheaply as possibleUse models and work products to ensure qualityInvolve the user in developmentUser is involved early on, but not during developmentIteratively elicit requirements (negotiation/emergence)With req’ts established, sequentially follow process modelPriority on eliminating unneeded req’tsPriority on traceability and work productsamount of documentation can be a benefit for either oneFor SASD, gives new team members something to work fromFor Agile, less work…NO checkmarks…
61 Should you go light?The following factors suggest an adaptive process:Uncertain or volatile requirementsResponsible and motivated developersCustomer who understands and will get involvedThese factors suggest a predictive process:A team over fiftyFixed price, or more correctly a fixed scope, contract
62 Which Adaptive Process? Team size and discipline.XP imposes a stronger discipline than any of the others.ASD focus on developing good-enough software.Crystal Clear appears to be the “lightest of the light”.Take charge of your process, monitor it and adapt it to your circumstances.
64 References (1/3)Martin Fowler. “The New Methodology”. Abridged version published in Software Development Magazine, December 2000.Dirk Riehle. “A Comparison of the Value Systems of Adaptive Software Development and Extreme Programming: How Methodologies May Learn from Each Other”. SKYVA International, 2000.
65 References (2/3)Jim Highsmith. “Messy, Exciting and Anxiety-Ridden: Adaptive Software Development”. Published in American Programmer Magazine, April 1997.Jim Highsmith. “Adaptive Software Development”. Presented at OOPSLA 2000.Kent Beck. Extreme Programming Explained: Embrace Change. Addison‑Wesley, 2000.
66 References (3/3)Mike Beedle, Martine Devos, Yonat Sharon, Ken Schwaber, Jeff Sutherland. “SCRUM: A Pattern Language for Hyperproductive Software Development.” In Pattern Languages of Program Design 4. Edited by Neil Harrison, Brian Foote, and Hans Rohnert. Addison‑Wesley, PageAlistair Cockburn. Crystal “Clear”. AOL, 2000.
67 Sources Adapted from “Lightweight Methodologies” by Dias (Slides 2-4, 32-38, 48-53, 61-62,64-66)“Requirements Engineering in Agile Approaches” by Balas, Fournie, & Luo(Slides 5-6, 12-28, 31, 39-44, 46, 54-59)