2The Software Development Process A structured set of activities required to develop a software systemSpecification;Design;Implementation;Validation;Evolution.A software process model is an abstract representation of a process. It presents a description of a process from some particular perspective.
3Generic Software Process Models The waterfall modelSeparate and distinct phases of specification and development.Evolutionary developmentSpecification, development and validation are interleaved.Component-based software engineeringThe system is assembled from existing components.There are many variants of these models e.g. formal development where a waterfall-like process is used but the specification is a formal specification that is refined through several stages to an implementable design.
5Waterfall Model Problems Inflexible partitioning of the project into distinct stages makes it difficult to respond to changing customer requirements.Therefore, this model is only appropriate when the requirements are well-understood and changes will be fairly limited during the design process.Few business systems have stable requirements.
6Evolutionary development Exploratory developmentObjective is to work with customers and to evolve a final system from an initial outline specification. Should start with well-understood requirements and add new features as proposed by the customer.Throw-away prototypingObjective is to understand the system requirements. Should start with poorly understood requirements to clarify what is really needed.
8Evolutionary development ProblemsLack of process visibility;Systems are often poorly structured;Special skills (e.g. in languages for rapid prototyping) may be required.ApplicabilityFor small or medium-size interactive systems;For parts of large systems (e.g. the user interface);For short-lifetime systems.
9Component-Based Software Engineering Based on systematic reuse where systems are integrated from existing components or COTS (Commercial-off-the-shelf) systems.Process stagesComponent analysis;Requirements modification;System design with reuse;Development and integration.This approach is becoming increasingly used as component standards have emerged.
11Process IterationSystem requirements ALWAYS evolve in the course of a project so process iteration where earlier stages are reworked is always part of the process for large systems.Iteration can be applied to any of the generic process models.Two (related) approachesIncremental delivery;Spiral development.
12Incremental deliveryRather than deliver the system as a single delivery, the development and delivery is broken down into increments with each increment delivering part of the required functionality.User requirements are prioritised and the highest priority requirements are included in early increments.Once the development of an increment is started, the requirements are frozen though requirements for later increments can continue to evolve.
14Incremental Development Advantages Customer value can be delivered with each increment so system functionality is available earlier.Early increments act as a prototype to help elicit requirements for later increments.Lower risk of overall project failure.The highest priority system services tend to receive the most testing.
15Spiral DevelopmentProcess is represented as a spiral rather than as a sequence of activities with backtracking.Each loop in the spiral represents a phase in the process.No fixed phases such as specification or design - loops in the spiral are chosen depending on what is required.Risks are explicitly assessed and resolved throughout the process.
17Spiral Model Sectors Objective setting Specific objectives for the phase are identified.Risk assessment and reductionRisks are assessed and activities put in place to reduce the key risks.Development and validationA development model for the system is chosen which can be any of the generic models.PlanningThe project is reviewed and the next phase of the spiral is planned.
18Process activities Software specification Software design and implementationSoftware validationSoftware evolution
19Software Specification The process of establishing what services are required and the constraints on the system’s operation and development.Requirements engineering processFeasibility study;Requirements elicitation and analysis;Requirements specification;Requirements validation.
21Software Design and Implementation The process of converting the system specification into an executable system.Software designDesign a software structure that realises the specification;ImplementationTranslate this structure into an executable program;The activities of design and implementation are closely related and may be inter-leaved.
24Structured MethodsSystematic approaches to developing a software design.The design is usually documented as a set of graphical models.Possible modelsObject model;Sequence model;State transition model;Structural model;Data-flow model.
25Programming and Debugging Translating a design into a program and removing errors from that program.Programming is a personal activity - there is no generic programming process.Programmers carry out some program testing to discover faults in the program and remove these faults in the debugging process.
26Software ValidationVerification and validation (V & V) is intended to show that a system conforms to its specification and meets the requirements of the system customer.Involves checking and review processes and system testing.System testing involves executing the system with test cases that are derived from the specification of the real data to be processed by the system.
28Testing stages Component or unit testing System testing Individual components are tested independently;Components may be functions or objects or coherent groupings of these entities.System testingTesting of the system as a whole. Testing of emergent properties is particularly important.Acceptance testingTesting with customer data to check that the system meets the customer’s needs.
30Software evolution Software is inherently flexible and can change. As requirements change through changing business circumstances, the software that supports the business must also evolve and change.Although there has been a demarcation between development and evolution (maintenance) this is increasingly irrelevant as fewer and fewer systems are completely new.
32The Rational Unified Process A modern process model derived from the work on the UML and associated process.Normally described from 3 perspectivesA dynamic perspective that shows phases over time;A static perspective that shows process activities;A practive perspective that suggests good practice.
33RUP Phases Inception Elaboration Construction Transition Establish the business case for the system.ElaborationDevelop an understanding of the problem domain and the system architecture.ConstructionSystem design, programming and testing.TransitionDeploy the system in its operating environment.
34RUP Best Practice Develop software iteratively Manage requirements Use component-based architecturesVisually model softwareVerify software qualityControl changes to software
35Best Practice 1: Develop Iteratively Best Practices Instructor NotesBest Practice 1: Develop IterativelyWe need to keep the concept of iterative developme nt at a very high level in this module. It is easy to get bogged down in the many ramificatio ns of iterative developme nt.Try just to get the concept across. The Rational Unified Process Fundament als course covers this in more detail.Developing iteratively is a technique that is used to deliver the functionality of a system in a successive series of releases of increasing completeness. Each release is developed in a specific, fixed time period called an “iteration.”Each iteration is focused on defining, analyzing, designing, building and testing some set of requirements.Module 1 - Best Practices of Software Engineering
36Waterfall Development Characteristics Best Practices Instructor NotesWaterfall Development CharacteristicsTo illustrate a problem with the waterfall model: Suppose I estimate that the project will take 2 years, and it really takes 3 years. At the end of 2 years, what do I have? Nothing useful works. No partial delivery is possible. Diagrams and models are great, but they can’t execute.Waterfall ProcessDelays confirmation of critical risk resolutionMeasures progress by assessing work-products that are poor predictors of time-to-completionDelays and aggregates integration and testingPrecludes early deploymentFrequently results in major unplanned iterationsRequirementsanalysisDesignCode and unit testSubsystem integrationSystem testWaterfall is conceptually straightforward because it produces a single deliverable. The fundamental problem of this approach is that it pushes risk forward in time, where it’s costly to undo mistakes from earlier phases. An initial design will likely be flawed with respect to its key requirements, and furthermore, the late discovery of design defects tends to result in costly overruns and/or project cancellation. The waterfall approach tends to mask the real risks to a project until it is too late to do anything meaningful about them.Module 1 - Best Practices of Software Engineering
37Iterative Development Produces an Executable Best Practices Instructor NotesIterative Development Produces an ExecutableInitialPlanningRequirementsAnalysis & DesignImplementationDeploymentTestEvaluationManagementEnvironmentEach iteration results in an executable releaseThe earliest iterations address greatest risks. Each iteration produces an executable release. Each iteration includes integration and test. Iterations help:resolve major risks before making large investments.enable early user feedback.make testing and integration continuous.focus project short-term objective milestones.make possible deployment of partial implementations.Iterative processes were developed in response to these waterfall characteristics. With an iterative process, the waterfall steps are applied iteratively. Instead of developing the whole system in lock step, an increment (i.e. a subset of system functionality) is selected and developed, then another increment, etc. The selection of the first increment to be developed is based on risk, the highest priority risks first. To address the selected risk(s), choose a subset of use cases. Develop the minimal set of use cases that will allow objective verification (that is, through a set of executable tests) of the risks that you have chosen. Then select the next increment to address the next highest risk, and so on. Thus you apply the waterfall within each iteration and the system evolves incrementally.Module 1 - Best Practices of Software Engineering
38Best Practices Instructor Notes Risk ProfilesRisk ReductionWaterfall RiskRiskIterative development produces the architecture first, allowing integration to occur “as the verification activity” of the design phase, and allowing design flaws to be detected and resolved earlier in the lifecycle. Continuous integration throughout the project replaces the “big bang” integration at the end of a project. Iterative development also provides much better insight into quality, because system characteristics that are largely inherent in the architecture (for example, performance, fault tolerance, maintainability) are tangible earlier in the process. Thus, issues are still correctable without jeopardizing target costs and schedules.Iterative RiskTimeModule 1 - Best Practices of Software Engineering
39Best Practice 2: Manage Requirements Best Practices Instructor NotesBest Practice 2: Manage RequirementsYou can generate discussion with the students concerning how they get requiremen ts and how they manage them now.Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,Module 1 - Best Practices of Software Engineering
40Aspects of Requirements Management Best Practices Instructor NotesAspects of Requirements ManagementAnalyze the ProblemUnderstand User NeedsDefine the SystemManage ScopeRefine the System DefinitionManage Changing RequirementsModule 1 - Best Practices of Software Engineering
41Best Practice 3: Use Component-Based Architectures Best Practices Instructor NotesBest Practice 3: Use Component-Based ArchitecturesYou can generate discussion with the students concerning how they get requiremen ts and how they manage them now.Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,Module 1 - Best Practices of Software Engineering
42Purpose of a Component-Based Architecture Best Practices Instructor NotesPurpose of a Component-Based ArchitectureBecause students vary in how familiar they are with the concept of architecture applied to software, it is best to get a sense of this from the students before beginning this section. If they are fairly unfamiliar, it helps to use the analogy of buildings or civil engineering. The more complex the building, the more critical a good architecture is. The longer you want the building to be useful, the more effort and expense you will put into the architecture. And in both of these cases, the choice of architect is critical.Basis for reuseComponent reuseArchitecture reuseBasis for project managementPlanningStaffingDeliveryIntellectual controlManage complexityMaintain integrityComponent-based architecture with layersSystem-softwareMiddlewareBusiness-specificApplication-Definition of a (Software) Component:RUP Definition: A nontrivial, nearly independent, and replaceable part of a system that fulfills a clear function in the context of a well-defined architecture. A component conforms to and provides the physical realization of a set of interfaces.UML Definition: A physical, replaceable part of a system that packages implementation, and conforms to and provides the realization of a set of interfaces. A component represents a physical piece of implementation of a system, including software code (source, binary, or executable) or equivalents such as scripts or command files.Module 1 - Best Practices of Software Engineering
43Best Practice 4: Model Visually Best Practices Instructor NotesBest Practice 4: Model VisuallyYou can generate discussion with the students concerning how they get requiremen ts and how they manage them now.Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,Module 1 - Best Practices of Software Engineering
44Visual Modeling with Unified Modeling Language Best Practices Instructor NotesVisual Modeling with Unified Modeling LanguageMultiple viewsPrecise syntax and semanticsThe UML emphasizes a graphical language for representing models, but provides little or no guidance on when and how to use these diagrams. This is an area where the RUP helps. It describes the kinds of project artifacts needed, including diagrams, and puts them in the context of an overall project plan.ClassDiagramsUse-CaseDiagramsObjectDiagramsSequenceDiagramsCollaborationDiagramsComponentDiagramsModelsIn building a visual model of a system, many different diagrams are needed to represent different views of the system. The UML provides a rich notation for visualizing our models. This includes the following key diagrams:Use-Case diagrams to illustrate user interactions with the systemClass diagrams to illustrate logical structureObject diagrams to illustrate objects and linksStatechart diagrams to illustrate behaviorComponent diagrams to illustrate physical structure of the softwareDeployment diagrams to show the mapping of software to hardware configurationsInteraction diagrams (that is, collaboration and sequence diagrams) to illustrate behaviorActivity diagrams to illustrate flows of eventsStatechartDiagramsDeploymentDiagramsStatic DiagramsActivityDiagramsDynamic DiagramsModule 1 - Best Practices of Software Engineering
45Best Practice 5: Continuously Verify Quality Best Practices Instructor NotesBest Practice 5: Continuously Verify QualityYou can generate discussion with the students concerning how they get requiremen ts and how they manage them now.Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,Module 1 - Best Practices of Software Engineering
46Continuously Verify Your Software’s Quality Best Practices Instructor NotesContinuously Verify Your Software’s QualityMany people remember Barry Boehm’s groundbre aking work in Software Economics where he quantified the relative expense to fix a bug at different times in the developme nt lifecycle. Be cautious, however, since his work was based on the waterfall model, not an iterative developme nt model.The iterative model fundament ally changes how and when we test.Software problems are 100 to 1000 times more costly to find and repair after deploymentCost to Repair SoftwareCost of Lost OpportunitiesCost of Lost CustomersCostThis principle is driven by a fundamental and well-known property of software development: it’s a lot less expensive to correct defects during development than to correct them after deployment.Tests for key scenarios ensure that all requirements are properly implementedPoor application performance hurts as much as poor reliabilityVerify software reliability—memory leaks, bottlenecksTest every iteration—automate test!InceptionElaborationConstructionTransitionModule 1 - Best Practices of Software Engineering
47Best Practices Instructor Notes Test Each IterationUML Model and ImplementationTestsIteration 1Test Suite 1Iteration 2Test Suite 2Iteration 3Test Suite 3Iteration 4The point to be made on this slide is that it is important to start automating test in the earliest iterations. Some people think that you don’t need to automate testing until the later iterations when requirements are pretty stable. Even if you expect that the requirements, and hence the tests, are still unstable, it is still more efficient to automate.Test Suite 4In each iteration, automated tests are created that test the requirements addressed in that iteration. As new requirements are added in subsequent iterations, new tests are generated and run. At times, a requirement may be changed in a later iteration. In that case, the tests associated with the changed requirement may be modified or simply regenerated by an automated tool.Module 1 - Best Practices of Software Engineering
48Best Practice 6: Manage Change Best Practices Instructor NotesBest Practice 6: Manage ChangeYou can generate discussion with the students concerning how they get requiremen ts and how they manage them now.Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,Module 1 - Best Practices of Software Engineering
49Best Practices Instructor Notes Aspects of a CM SystemChange Request Management (CRM)Configuration Status ReportingConfiguration Management (CM)Change TrackingVersion SelectionSoftware ManufactureChange Request Management (CRM) addresses the organizational infrastructure required to assess the cost and schedule impacts of a requested change to the existing product. CRM addresses the workings of a Change Review Team or Change Control Board.Configuration Status Accounting (Measurement) is used to describe the “state” of the product based on the type, number, rate and severity of defects found and fixed during the course of product development. Metrics derived under this aspect, either through audits or raw data, are useful in determining the overall completeness of the project.Configuration Management (CM) describes the product structure and identifies its constituent configuration items that are treated as single versionable entities in the configuration management process. CM deals with defining configurations, building and labeling, and collecting versioned artifacts into constituent sets and maintaining traceability between these versions.Change Tracking describes what is done to components for what reason and at what time. It serves as the history and rationale of changes. It is quite separate from assessing the impact of proposed changes as described under Change Request Management.Version Selection is to ensure that the right versions of configuration items are selected for change or implementation. Version selection relies on a solid foundation of “configuration identification.”Software Manufacture covers the need to automate the steps to compile, test, and package software for distribution.Module 1 - Best Practices of Software Engineering
50RUP is Organization Along Time Best Practices Instructor NotesRUP is Organization Along TimeTimeThe time aspect of the process as it is enacted is shown by phases, iterations, and milestones. Use of these helps minimize risk of your resource allocation since resources are allocated only on a firm basis.Organization by phases helps minimize the risks of resource allocation.Module 1 - Best Practices of Software Engineering
51Major Milestones: Business Decision Points Best Practices Instructor NotesMajor Milestones: Business Decision PointsCustomeracceptanceor end of lifeProductReleaseProduct sufficiently mature for customersInitial OperationalCapabilityMilestonePhases represent the management perspective of the project, the “35,000 foot level.” The details are left to the engineering perspective, which is at the iteration level.The names of the milestones are identical to the ones proposed by Barry Boehm in the article “Anchoring the Software Process,” IEEE Software, July 1996, ppCommit resourcesfor constructionLifecycleArchitectureMilestoneCommit resources for the elaboration phaseLifecycleObjectiveMilestoneInceptionElaborationConstructionTransitionThe phases of RUP were chosen such that phase boundaries correspond to significant decision points in the life of a project. For example, at the end of Inception, enough work has been done to capture the problem to be solved, and a vision of the system has been developed. An initial set of risks has also been identified and evaluated. Based on this information, a decision must be made whether to fund the project. Similar decision points correspond to the end of Elaboration, Construction, and Transition.Milestones help us assess the progress of a project at key points. Management can use these to establish clear criteria from which to decide the course of a project. They provide opportunities to change course. But unlike the waterfall approach, phases contain iterations which yield executable results.timeModule 1 - Best Practices of Software Engineering
52Best Practices Instructor Notes What Is an Iteration?An iteration is one pass through all disciplines.An iteration can be seen as a mini project with a plan, deliverables and assessment, in which periodic corrections are applied to the remainder of the project.If you are using the four phases of RUP, you are already using iterations, but more iterations can be added to each phase if needed.Module 1 - Best Practices of Software Engineering
53Iteration: Number and Duration Best Practices Instructor NotesIteration: Number and Duration“+” indicates factors which might cause longer iterations.“-” indicates factors which might shorten an iteration.Duration driven by+ size of organization+ size of project- familiarity with the process, maturity- technical simplicity6, plus or minus 3Inception:Elaboration:Construction:Transition:Once you know the length of an iteration appropriate to your organization, you can decide how many iterations to plan in each phase.If Inception consists only of planning and marketing, then there may be no real iteration. If you need a prototype or need to immediately address a technical risk, you will need an iteration.The number of iterations in Elaboration may depend on the complexity of the architecture, where you’re starting from, and the experience of the team.During Construction, you can use iterations to do a good job of testing and integration. Sufficient automation can make more iterations easier to deal with. Also, a mature process could accommodate more iterations.Transition would require at least one iteration, and more if conditions at the time of transition make more iterations necessary.Module 1 - Best Practices of Software Engineering
54Best Practices Instructor Notes One IterationIn an iteration, you walk through all disciplines.A given iteration includes multiple disciplines. The form the discipline will take varies, depending on its position within the overall lifecycle, and the nature of the project.Module 1 - Best Practices of Software Engineering
55Organization Based on Content Best Practices Instructor NotesOrganization Based on ContentContentOrganization into disciplines shows the content of the process—the activities, artifacts, and roles.Module 1 - Best Practices of Software Engineering
56Best Practices Instructor Notes Nine DisciplinesRUP uses the term Discipline to describe all activities you may go through to produce a particular set of artifacts—a summary of all roles, activities, artifacts, concepts and guidelines that are involved.The disciplines partition the roles and activities into logical groupings. The groupings should not be taken as a waterfall process. You visit these disciplines repeatedly throughout the lifecycle. The emphasis and levels of intensity change with each iteration.Module 1 - Best Practices of Software Engineering
57Artifact Set Evolution Over the Development Phases Best Practices Instructor NotesArtifact Set Evolution Over the Development PhasesWith the iterative approach, artifact sets mature over time.Artifacts are organized into sets, one set per discipline.In an iterative development process, the various artifacts are not produced in one phase, completed or even frozen before moving on to the next phase. Instead, the artifact sets evolve throughout the development cycle.Module 1 - Best Practices of Software Engineering