1. Software Processes Overview
Don Evans – CSE

2. The Software Development Process
A structured set of activities required to develop a software system
Specification;
Design;
Implementation;
Validation;
Evolution.
A software process model is an abstract representation of a process. It presents a description of a process from some particular perspective.

3. Generic Software Process Models
The waterfall model
Separate and distinct phases of specification and development.
Evolutionary development
Specification, development and validation are interleaved.
Component-based software engineering
The system is assembled from existing components.
There are many variants of these models e.g. formal development where a waterfall-like process is used but the specification is a formal specification that is refined through several stages to an implementable design.

4. Waterfall Model

5. Waterfall Model Problems
Inflexible partitioning of the project into distinct stages makes it difficult to respond to changing customer requirements.
Therefore, this model is only appropriate when the requirements are well-understood and changes will be fairly limited during the design process.
Few business systems have stable requirements.

6. Evolutionary development
Exploratory development
Objective is to work with customers and to evolve a final system from an initial outline specification. Should start with well-understood requirements and add new features as proposed by the customer.
Throw-away prototyping
Objective is to understand the system requirements. Should start with poorly understood requirements to clarify what is really needed.

7. Evolutionary development

8. Evolutionary development
Problems
Lack of process visibility;
Systems are often poorly structured;
Special skills (e.g. in languages for rapid prototyping) may be required.
Applicability
For small or medium-size interactive systems;
For parts of large systems (e.g. the user interface);
For short-lifetime systems.

9. Component-Based Software Engineering
Based on systematic reuse where systems are integrated from existing components or COTS (Commercial-off-the-shelf) systems.
Process stages
Component analysis;
Requirements modification;
System design with reuse;
Development and integration.
This approach is becoming increasingly used as component standards have emerged.

10. A Reuse-Oriented Development Process

11. Process Iteration
System requirements ALWAYS evolve in the course of a project so process iteration where earlier stages are reworked is always part of the process for large systems.
Iteration can be applied to any of the generic process models.
Two (related) approaches
Incremental delivery;
Spiral development.

12. Incremental delivery
Rather than deliver the system as a single delivery, the development and delivery is broken down into increments with each increment delivering part of the required functionality.
User requirements are prioritised and the highest priority requirements are included in early increments.
Once the development of an increment is started, the requirements are frozen though requirements for later increments can continue to evolve.

13. Incremental development

14. Incremental Development Advantages
Customer value can be delivered with each increment so system functionality is available earlier.
Early increments act as a prototype to help elicit requirements for later increments.
Lower risk of overall project failure.
The highest priority system services tend to receive the most testing.

15. Spiral Development
Process is represented as a spiral rather than as a sequence of activities with backtracking.
Each loop in the spiral represents a phase in the process.
No fixed phases such as specification or design - loops in the spiral are chosen depending on what is required.
Risks are explicitly assessed and resolved throughout the process.

16. Spiral model of the software process

17. Spiral Model Sectors Objective setting
Specific objectives for the phase are identified.
Risk assessment and reduction
Risks are assessed and activities put in place to reduce the key risks.
Development and validation
A development model for the system is chosen which can be any of the generic models.
Planning
The project is reviewed and the next phase of the spiral is planned.

18. Process activities Software specification
Software design and implementation
Software validation
Software evolution

19. Software Specification
The process of establishing what services are required and the constraints on the system’s operation and development.
Requirements engineering process
Feasibility study;
Requirements elicitation and analysis;
Requirements specification;
Requirements validation.

20. The Requirements Engineering Process

21. Software Design and Implementation
The process of converting the system specification into an executable system.
Software design
Design a software structure that realises the specification;
Implementation
Translate this structure into an executable program;
The activities of design and implementation are closely related and may be inter-leaved.

22. Design Process Activities
Architectural design
Abstract specification
Interface design
Component design
Data structure design
Algorithm design

23. The Software Design Process

24. Structured Methods
Systematic approaches to developing a software design.
The design is usually documented as a set of graphical models.
Possible models
Object model;
Sequence model;
State transition model;
Structural model;
Data-flow model.

25. Programming and Debugging
Translating a design into a program and removing errors from that program.
Programming is a personal activity - there is no generic programming process.
Programmers carry out some program testing to discover faults in the program and remove these faults in the debugging process.

26. Software Validation
Verification and validation (V & V) is intended to show that a system conforms to its specification and meets the requirements of the system customer.
Involves checking and review processes and system testing.
System testing involves executing the system with test cases that are derived from the specification of the real data to be processed by the system.

27. The testing process

28. Testing stages Component or unit testing System testing
Individual components are tested independently;
Components may be functions or objects or coherent groupings of these entities.
System testing
Testing of the system as a whole. Testing of emergent properties is particularly important.
Acceptance testing
Testing with customer data to check that the system meets the customer’s needs.

29. Testing Phases

30. Software evolution Software is inherently flexible and can change.
As requirements change through changing business circumstances, the software that supports the business must also evolve and change.
Although there has been a demarcation between development and evolution (maintenance) this is increasingly irrelevant as fewer and fewer systems are completely new.

31. System evolution

32. The Rational Unified Process
A modern process model derived from the work on the UML and associated process.
Normally described from 3 perspectives
A dynamic perspective that shows phases over time;
A static perspective that shows process activities;
A practive perspective that suggests good practice.

33. RUP Phases Inception Elaboration Construction Transition
Establish the business case for the system.
Elaboration
Develop an understanding of the problem domain and the system architecture.
Construction
System design, programming and testing.
Transition
Deploy the system in its operating environment.

34. RUP Best Practice Develop software iteratively Manage requirements
Use component-based architectures
Visually model software
Verify software quality
Control changes to software

35. Best Practice 1: Develop Iteratively
Best Practices Instructor Notes
Best Practice 1: Develop Iteratively
We need to keep the concept of iterative developme nt at a very high level in this module. It is easy to get bogged down in the many ramificatio ns of iterative developme nt.
Try just to get the concept across. The Rational Unified Process Fundament als course covers this in more detail.
Developing iteratively is a technique that is used to deliver the functionality of a system in a successive series of releases of increasing completeness. Each release is developed in a specific, fixed time period called an “iteration.”
Each iteration is focused on defining, analyzing, designing, building and testing some set of requirements.
Module 1 - Best Practices of Software Engineering

36. Waterfall Development Characteristics
Best Practices Instructor Notes
Waterfall Development Characteristics
To illustrate a problem with the waterfall model: Suppose I estimate that the project will take 2 years, and it really takes 3 years. At the end of 2 years, what do I have? Nothing useful works. No partial delivery is possible. Diagrams and models are great, but they can’t execute.
Waterfall Process
Delays confirmation of critical risk resolution
Measures progress by assessing work-products that are poor predictors of time-to-completion
Delays and aggregates integration and testing
Precludes early deployment
Frequently results in major unplanned iterations
Requirements
analysis
Design
Code and unit test
Subsystem integration
System test
Waterfall is conceptually straightforward because it produces a single deliverable. The fundamental problem of this approach is that it pushes risk forward in time, where it’s costly to undo mistakes from earlier phases. An initial design will likely be flawed with respect to its key requirements, and furthermore, the late discovery of design defects tends to result in costly overruns and/or project cancellation. The waterfall approach tends to mask the real risks to a project until it is too late to do anything meaningful about them.
Module 1 - Best Practices of Software Engineering

37. Iterative Development Produces an Executable
Best Practices Instructor Notes
Iterative Development Produces an Executable
Initial
Planning
Requirements
Analysis & Design
Implementation
Deployment
Test
Evaluation
Management
Environment
Each iteration results in an executable release
The earliest iterations address greatest risks. Each iteration produces an executable release. Each iteration includes integration and test. Iterations help:
resolve major risks before making large investments.
enable early user feedback.
make testing and integration continuous.
focus project short-term objective milestones.
make possible deployment of partial implementations.
Iterative processes were developed in response to these waterfall characteristics. With an iterative process, the waterfall steps are applied iteratively. Instead of developing the whole system in lock step, an increment (i.e. a subset of system functionality) is selected and developed, then another increment, etc. The selection of the first increment to be developed is based on risk, the highest priority risks first. To address the selected risk(s), choose a subset of use cases. Develop the minimal set of use cases that will allow objective verification (that is, through a set of executable tests) of the risks that you have chosen. Then select the next increment to address the next highest risk, and so on. Thus you apply the waterfall within each iteration and the system evolves incrementally.
Module 1 - Best Practices of Software Engineering

38. Best Practices Instructor Notes
Risk Profiles
Risk Reduction
Waterfall Risk
Risk
Iterative development produces the architecture first, allowing integration to occur “as the verification activity” of the design phase, and allowing design flaws to be detected and resolved earlier in the lifecycle. Continuous integration throughout the project replaces the “big bang” integration at the end of a project. Iterative development also provides much better insight into quality, because system characteristics that are largely inherent in the architecture (for example, performance, fault tolerance, maintainability) are tangible earlier in the process. Thus, issues are still correctable without jeopardizing target costs and schedules.
Iterative Risk
Time
Module 1 - Best Practices of Software Engineering

39. Best Practice 2: Manage Requirements
Best Practices Instructor Notes
Best Practice 2: Manage Requirements
You can generate discussion with the students concerning how they get requiremen ts and how they manage them now.
Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.
A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,
Module 1 - Best Practices of Software Engineering

40. Aspects of Requirements Management
Best Practices Instructor Notes
Aspects of Requirements Management
Analyze the Problem
Understand User Needs
Define the System
Manage Scope
Refine the System Definition
Manage Changing Requirements
Module 1 - Best Practices of Software Engineering

41. Best Practice 3: Use Component-Based Architectures
Best Practices Instructor Notes
Best Practice 3: Use Component-Based Architectures
You can generate discussion with the students concerning how they get requiremen ts and how they manage them now.
Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.
A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,
Module 1 - Best Practices of Software Engineering

42. Purpose of a Component-Based Architecture
Best Practices Instructor Notes
Purpose of a Component-Based Architecture
Because students vary in how familiar they are with the concept of architecture applied to software, it is best to get a sense of this from the students before beginning this section. If they are fairly unfamiliar, it helps to use the analogy of buildings or civil engineering. The more complex the building, the more critical a good architecture is. The longer you want the building to be useful, the more effort and expense you will put into the architecture. And in both of these cases, the choice of architect is critical.
Basis for reuse
Component reuse
Architecture reuse
Basis for project management
Planning
Staffing
Delivery
Intellectual control
Manage complexity
Maintain integrity
Component-based architecture with layers
System-
software
Middleware
Business-
specific
Application-
Definition of a (Software) Component:
RUP Definition: A nontrivial, nearly independent, and replaceable part of a system that fulfills a clear function in the context of a well-defined architecture. A component conforms to and provides the physical realization of a set of interfaces.
UML Definition: A physical, replaceable part of a system that packages implementation, and conforms to and provides the realization of a set of interfaces. A component represents a physical piece of implementation of a system, including software code (source, binary, or executable) or equivalents such as scripts or command files.
Module 1 - Best Practices of Software Engineering

43. Best Practice 4: Model Visually
Best Practices Instructor Notes
Best Practice 4: Model Visually
You can generate discussion with the students concerning how they get requiremen ts and how they manage them now.
Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.
A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,
Module 1 - Best Practices of Software Engineering

44. Visual Modeling with Unified Modeling Language
Best Practices Instructor Notes
Visual Modeling with Unified Modeling Language
Multiple views
Precise syntax and semantics
The UML emphasizes a graphical language for representing models, but provides little or no guidance on when and how to use these diagrams. This is an area where the RUP helps. It describes the kinds of project artifacts needed, including diagrams, and puts them in the context of an overall project plan.
Class
Diagrams
Use-Case
Diagrams
Object
Diagrams
Sequence
Diagrams
Collaboration
Diagrams
Component
Diagrams
Models
In building a visual model of a system, many different diagrams are needed to represent different views of the system. The UML provides a rich notation for visualizing our models. This includes the following key diagrams:
Use-Case diagrams to illustrate user interactions with the system
Class diagrams to illustrate logical structure
Object diagrams to illustrate objects and links
Statechart diagrams to illustrate behavior
Component diagrams to illustrate physical structure of the software
Deployment diagrams to show the mapping of software to hardware configurations
Interaction diagrams (that is, collaboration and sequence diagrams) to illustrate behavior
Activity diagrams to illustrate flows of events
Statechart
Diagrams
Deployment
Diagrams
Static Diagrams
Activity
Diagrams
Dynamic Diagrams
Module 1 - Best Practices of Software Engineering

45. Best Practice 5: Continuously Verify Quality
Best Practices Instructor Notes
Best Practice 5: Continuously Verify Quality
You can generate discussion with the students concerning how they get requiremen ts and how they manage them now.
Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.
A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,
Module 1 - Best Practices of Software Engineering

46. Continuously Verify Your Software’s Quality
Best Practices Instructor Notes
Continuously Verify Your Software’s Quality
Many people remember Barry Boehm’s groundbre aking work in Software Economics where he quantified the relative expense to fix a bug at different times in the developme nt lifecycle. Be cautious, however, since his work was based on the waterfall model, not an iterative developme nt model.
The iterative model fundament ally changes how and when we test.
Software problems are 100 to 1000 times more costly to find and repair after deployment
Cost to Repair Software
Cost of Lost Opportunities
Cost of Lost Customers
Cost
This principle is driven by a fundamental and well-known property of software development: it’s a lot less expensive to correct defects during development than to correct them after deployment.
Tests for key scenarios ensure that all requirements are properly implemented
Poor application performance hurts as much as poor reliability
Verify software reliability—memory leaks, bottlenecks
Test every iteration—automate test!
Inception
Elaboration
Construction
Transition
Module 1 - Best Practices of Software Engineering

47. Best Practices Instructor Notes
Test Each Iteration
UML Model and Implementation
Tests
Iteration 1
Test Suite 1
Iteration 2
Test Suite 2
Iteration 3
Test Suite 3
Iteration 4
The point to be made on this slide is that it is important to start automating test in the earliest iterations. Some people think that you don’t need to automate testing until the later iterations when requirements are pretty stable. Even if you expect that the requirements, and hence the tests, are still unstable, it is still more efficient to automate.
Test Suite 4
In each iteration, automated tests are created that test the requirements addressed in that iteration. As new requirements are added in subsequent iterations, new tests are generated and run. At times, a requirement may be changed in a later iteration. In that case, the tests associated with the changed requirement may be modified or simply regenerated by an automated tool.
Module 1 - Best Practices of Software Engineering

48. Best Practice 6: Manage Change
Best Practices Instructor Notes
Best Practice 6: Manage Change
You can generate discussion with the students concerning how they get requiremen ts and how they manage them now.
Students who want to know in detail how to manage requiremen ts should take the Requireme nts Manageme nt with Use Cases course.
A report from the Standish Group confirms that a distinct minority of software development projects is completed on-time and on-budget. In their report, the success rate was only 16.2 percent, while challenged projects (operational, but late and over-budget) accounted for 52.7 percent. Impaired projects (canceled) accounted for 31.1 percent. These failures are attributed to poor requirements management, incorrect definition of requirements from the start of the project, and poor requirements management throughout the development lifecycle. (Source: Chaos Report,
Module 1 - Best Practices of Software Engineering

49. Best Practices Instructor Notes
Aspects of a CM System
Change Request Management (CRM)
Configuration Status Reporting
Configuration Management (CM)
Change Tracking
Version Selection
Software Manufacture
Change Request Management (CRM) addresses the organizational infrastructure required to assess the cost and schedule impacts of a requested change to the existing product. CRM addresses the workings of a Change Review Team or Change Control Board.
Configuration Status Accounting (Measurement) is used to describe the “state” of the product based on the type, number, rate and severity of defects found and fixed during the course of product development. Metrics derived under this aspect, either through audits or raw data, are useful in determining the overall completeness of the project.
Configuration Management (CM) describes the product structure and identifies its constituent configuration items that are treated as single versionable entities in the configuration management process. CM deals with defining configurations, building and labeling, and collecting versioned artifacts into constituent sets and maintaining traceability between these versions.
Change Tracking describes what is done to components for what reason and at what time. It serves as the history and rationale of changes. It is quite separate from assessing the impact of proposed changes as described under Change Request Management.
Version Selection is to ensure that the right versions of configuration items are selected for change or implementation. Version selection relies on a solid foundation of “configuration identification.”
Software Manufacture covers the need to automate the steps to compile, test, and package software for distribution.
Module 1 - Best Practices of Software Engineering

50. RUP is Organization Along Time
Best Practices Instructor Notes
RUP is Organization Along Time
Time
The time aspect of the process as it is enacted is shown by phases, iterations, and milestones. Use of these helps minimize risk of your resource allocation since resources are allocated only on a firm basis.
Organization by phases helps minimize the risks of resource allocation.
Module 1 - Best Practices of Software Engineering

51. Major Milestones: Business Decision Points
Best Practices Instructor Notes
Major Milestones: Business Decision Points
Customer
acceptance
or end of life
Product
Release
Product sufficiently mature for customers
Initial Operational
Capability
Milestone
Phases represent the management perspective of the project, the “35,000 foot level.” The details are left to the engineering perspective, which is at the iteration level.
The names of the milestones are identical to the ones proposed by Barry Boehm in the article “Anchoring the Software Process,” IEEE Software, July 1996, pp
Commit resources
for construction
Lifecycle
Architecture
Milestone
Commit resources for the elaboration phase
Lifecycle
Objective
Milestone
Inception
Elaboration
Construction
Transition
The phases of RUP were chosen such that phase boundaries correspond to significant decision points in the life of a project. For example, at the end of Inception, enough work has been done to capture the problem to be solved, and a vision of the system has been developed. An initial set of risks has also been identified and evaluated. Based on this information, a decision must be made whether to fund the project. Similar decision points correspond to the end of Elaboration, Construction, and Transition.
Milestones help us assess the progress of a project at key points. Management can use these to establish clear criteria from which to decide the course of a project. They provide opportunities to change course. But unlike the waterfall approach, phases contain iterations which yield executable results.
time
Module 1 - Best Practices of Software Engineering

52. Best Practices Instructor Notes
What Is an Iteration?
An iteration is one pass through all disciplines.
An iteration can be seen as a mini project with a plan, deliverables and assessment, in which periodic corrections are applied to the remainder of the project.
If you are using the four phases of RUP, you are already using iterations, but more iterations can be added to each phase if needed.
Module 1 - Best Practices of Software Engineering

53. Iteration: Number and Duration
Best Practices Instructor Notes
Iteration: Number and Duration
“+” indicates factors which might cause longer iterations.
“-” indicates factors which might shorten an iteration.
Duration driven by
+ size of organization
+ size of project
- familiarity with the process, maturity
- technical simplicity
6, plus or minus 3
Inception:
Elaboration:
Construction:
Transition:
Once you know the length of an iteration appropriate to your organization, you can decide how many iterations to plan in each phase.
If Inception consists only of planning and marketing, then there may be no real iteration. If you need a prototype or need to immediately address a technical risk, you will need an iteration.
The number of iterations in Elaboration may depend on the complexity of the architecture, where you’re starting from, and the experience of the team.
During Construction, you can use iterations to do a good job of testing and integration. Sufficient automation can make more iterations easier to deal with. Also, a mature process could accommodate more iterations.
Transition would require at least one iteration, and more if conditions at the time of transition make more iterations necessary.
Module 1 - Best Practices of Software Engineering

54. Best Practices Instructor Notes
One Iteration
In an iteration, you walk through all disciplines.
A given iteration includes multiple disciplines. The form the discipline will take varies, depending on its position within the overall lifecycle, and the nature of the project.
Module 1 - Best Practices of Software Engineering

55. Organization Based on Content
Best Practices Instructor Notes
Organization Based on Content
Content
Organization into disciplines shows the content of the process—the activities, artifacts, and roles.
Module 1 - Best Practices of Software Engineering

56. Best Practices Instructor Notes
Nine Disciplines
RUP uses the term Discipline to describe all activities you may go through to produce a particular set of artifacts—a summary of all roles, activities, artifacts, concepts and guidelines that are involved.
The disciplines partition the roles and activities into logical groupings. The groupings should not be taken as a waterfall process. You visit these disciplines repeatedly throughout the lifecycle. The emphasis and levels of intensity change with each iteration.
Module 1 - Best Practices of Software Engineering

57. Artifact Set Evolution Over the Development Phases
Best Practices Instructor Notes
Artifact Set Evolution Over the Development Phases
With the iterative approach, artifact sets mature over time.
Artifacts are organized into sets, one set per discipline.
In an iterative development process, the various artifacts are not produced in one phase, completed or even frozen before moving on to the next phase. Instead, the artifact sets evolve throughout the development cycle.
Module 1 - Best Practices of Software Engineering