2Outline Lecture: software process Review: usefulness of software processesFitting processes to development problemsChoosing a processLab Exercise: Which process is best for DSD?Lecture: from process to project planImportance of well defined processImplementing a process in a project planLab Exercise: familiarize with assembla tools
3Problems in software development Knowing what the customers want; knowing the requirementsPredicting time to developPredicting resources needed to developManaging changeKnowing when you’re doneDesigning to enable distribution of workCoordinating workTracking time, resources, quality, productivity, effectiveness, etc.
4Addressed by Software Processes Developed as a tool for controlling complex software developmentsAnswers the “who”, “what”, “when”, etc. questionsWhat product should we work on next?What kind of person should do the work?What information is needed to do the work?When is the work finished?Intended useProvide guidance to developers in what to produce and when to produce itProvide a basis for planning and assessing development progressDifferent kinds of projects required different processes
5Characteristic Processes: The Waterfall Model Process viewed as a sequential set of activitiesImposes separation of concerns on software development activitiesWin Royce was first to propose the waterfall model. However, no one actually follows the strict sequence suggested by the diagram. There’s always backtracking and concurrency going on.Applies separation of concerns
6Characteristic Processes: The Spiral Model Process viewed as repeating cycles of increasing scaleIdentify risks and determine (next set of) requirements, build next version by extension, increasing scale each timeEarly iterations may be prototypesBarry Boehm created the spiral model at least partly to illustrate the role of risks and mitigating risks in the software development process. As risks are resolved, spirals produce larger versions of the system
7Characteristic Processes: The Iterative Model Process viewed as a sequence of iterations, each building on the lastBuild minimal useful subset, test, release, build next version by extensionEarly iterations may be prototypesIterative models were described in the 1970s by Basili & Turner and by Parnas, among others. Unwrap a spiral model and what you find is an iterative model. Not very clear what the difference is. Agile methods “took over” the idea of iterative models and use them heavily, generally with rather short iterations, say 2-4 weeks.IBM
8Characteristic Processes: Agile (scrum) Process viewed as nested sequence of builds (sprints)Each build adds small feature setCustomer in loop, code centered (little or no documentation)Problem detection and correction though daily team meetings (scrum)
9Formal DefinitionProcess: we define a process as set of artifacts, activities, roles and the relationships between them where:Artifact: any work product of the software development process (requirements specifications, design documents, code, etc.)Activities: the tasks that produce the work products (requirements analysis, design, coding)Roles: responsibility for performing specific activities (requirements analyst, software architect, coder)Relationships: the relations between artifacts, activities, and roles that structure the process (precedes, responsible-for)Intuitively: roles produce artifacts by performing activitiesA coder is responsible for implementing module code as part of codingA tester is responsible for writing test cases as part of verification
10Process Definition Graphic RelationshipsRoles: who does the work?Artifacts: what is produced?Activities: what tasks are done?
11How do processes vary?Content: processes vary in the specific activities performed, artifacts produced, roles required, and the relationships between these, for example:Which specific activities are performedWhich role performs which activitiesFormality: processes vary in how detailed, complex, and prescriptive they areHow much detail is defined on the activities, etc.How closely developers are required to follow the written process
12How do processes vary?Emphasis varies on artifacts, types of artifacts, rules governing activities, gating, roles, for example:Differ in form of requirements, design, test plan:Written document, conforming to standard template, reviewed by peers and users using standard review process, benchmarked and configuration controlledNotes on a web siteKnowledge in the heads of the development teamDiffer in review procedures for documents and code:Formalized inspections with criteria for passing, e.g., Fagan inspections or active reviewsInformal peer review meetingOffice mate reads it overNoneCompare spiral to agile
13Why do processes vary?Different processes reflect different assumptions about the developmental context and goalsContext: project size, complexity, availability of stakeholdersGoals: time-to-market, reliability, usability, maintainability, control of riskProcess is something we can design to address project needsMust considerWhat kind of process do we need: which kinds of activities, artifacts, etc. fit our goals and risks?How much formality/complexity do we need?Formality and complexity imply overhead.
14Process Formality and Project Scale As projects grow larger and more complex, they typically require more formal and detailed processesDifficult for individual developers, or management to track the overall state of the developmentDifficult to keep track of who is supposed to be doing what (“Who do I talk to?”)Difficult to know when your job should be finished or what quality criteria it should satisfyA clear, well-defined process helps keep a large project coordinated
15Distributed Development Contribution to Delay What factors contributed to underestimation? Here are two different perspectives. The top shows that the greater the distribution of the project across sites and organizations, the worse the estimate. For example, on average, Multiple Site projects within a single product group took about 50% more time than estimated. Projects that required external partnerships took about 75% more time than estimated, on average.
16When Process Complexity & Project Complexity/Scale Mismatch But, there can be too much process as wellProcess is overheadUnnecessary process overhead leads to problemsDevelopers feel frustrated“I want to write code, not documents”“I can’t understand what I’m supposed to do”“I’m afraid to touch this code”Progress is slowed“I have to wait for that other team to finish”“I have to wait for my code to be inspected”“We have an integration problem”You can have too little process or too much process. Process complexity should be matched to project complexity. Process should help development and not hinder it.
19Process DevelopmentCan view process development like software development:Choose/create a process to address specific project needs and constraintsThink in terms of requirements, design, etc.Must ask the questions:What are the key problems or risks of DSD?What features of a process would help addresses the risks of DSD?How much formality is needed?I.e., how much detail and specificity about the artifacts, activities, roles and relations?
20DSD Issues and Risks Key Problem: coordination at a distance i.e., the key difficulty is getting all the people involved to do the right task the right way at the right timeKey risk factors:Restricted communication, flow of informationDifferent organization, language, cultureLack of visibility into what remote teams are doingPotential difficulties:Different views of the problem (requirements)Different views of what the process is supposed to beMisunderstanding of what remote teams are doingDifficult to detect and correct problemsDifficult to manage synchronize the workDifficult to detect and correct slips in schedule
22Exercise: Chose a process Assume that you are part of a small company doing distributed developmentThree teams of developers in USA, China, and GermanyTeams are 8-10 people in mixed roles (some coders, testers, etc. at each site)Many team members have not worked together remotely beforeDiscuss as a teamEach person must take turns contributingDecide on team answerDetermine which kind of process would be the best fit for a DSD project and how formalHint: think about what happens over time
24Summary Co-located vs. DSD Co-located DevelopmentDSD Risks*Free flow of information through informal meansShared process viewClear ides of expertise, responsibilityCommon culture eases understandingUnderstand relationshipsPeople to tasksTask interdependenciesRestricted flow of information, mostly formalPossibly different process viewsUnclear idea of expertise, responsibility on remote teamsPossible misunderstandings due to cultural/language differencesVague or incorrect understanding of relationships*Standardizing the process helps mitigate these risks a people fill roles with well-defined responsibilities
27Incremental Development Over Time Acts as a feedback loop with a calibration point at each deliveryAllows cross checking of assumptions, understandingEarly check if remote sites are doing what is expectedEarly check for communication effectivenessAllows plan adjustments at each incrementTimeDeployInteg. &TestCodeDesignRequirementsIncrementInteg. & TestImplication is that the development process over time looks like this, i.e., we pretty much start over each time. This doesn’t mean that people do not carry forward experience or occasionally reuse things but that it is not an organized activity (part of a formalized process). Pretty inefficient.
28Well-defined Process Benefits Process should also be relatively formalWritten down in detailRequired for all of the distributed sitesWell-defined process clearly specifiesThe artifacts to be producedThe set of activates that must be performed (e.g., specify requirements, review design, write code)The set of roles (e.g., coder, tester, designer)The relationshipsWhich roles perform which activities to produce which artifactsThe order of activitiesWhich artifacts are needed as input to produce which other artifacts
29Well-defined Process Benefits Helps address risksEveryone has common definition of the processAssigning roles clearly defines responsibilitiesHelps make clear what people should be working onHelps make clear when a task is finishedShould answer for individuals the questionsIs this my job?What do I do next?Am I done yet?Did I do a good job?However: not enough just to define the process, must check that people understand and follow it.
30Importance of Clearly Defined Roles DSD coordination problems arise from communication problemsLack of contextual information makes unclearExactly who knows what (who has expertise)Exactly who is doing what (work allocation)What questions or problems people haveWhat assumptions people are makingEtc.
31Roles Help! Well defined roles provide a badly need structure Define who is responsible for whatGives guidance for expected expertiseRelations between roles tell youWho needs to talk to each other (e.g., shared responsibility, handoff, etc.)What you need to be talking aboutProvides bases for forming professional relationshipsUpshot: in DSD it is critical thatRoles and their responsibilities are clearly definedWell defined lines of communication are established between roles at different sitesPeople consistently perform the role’s responsibilities
32Examples: Process Specs Release_06/Process Template Design Document.docRelease_06/GEN-001RevisionControl.html
34From Process to PlanProcess definition manifests itself in the project planProcess definition is an abstractionMany possible ways of implementing the same processProject plan makes process concrete, it assignsPeople to rolesArtifacts to deliverablesActivities to tasks over timeProject plan should be one of the first products but expect it to evolveFor DSD, essential that distributed teams agree on the project plan
35Project Plan Minimal plan contents Usually owned by project manager Tasks to be performedPerson(s) assigned to roles and tasksDeadline for each taskSequencing among tasksRisks and mitigation strategiesUsually owned by project managerUpdated as project proceeds
36Project Roles & Responsibilities Number of team members taking on the roleArtifacts for which the role is responsibleSystems Engineer1use cases, requirements, preliminary screenshotsArchitect1 or 2Module, uses, process structures, interface specificationsDeveloper>1Module implementationTester & IntegratorModule tests, System generation and verification plan, test results reportProject ManagerProject plan, project measures, retrospective report
37Work Breakdown Structure This is a technique to analyze the content of work and cost by decomposing it into its component parts. It is produced by:Identifying the key elementsDecomposing each element into component partsContinuing to decompose until manageable work packages have been identified. These can then be allocated to the appropriate personThe WBS is used to allocate responsibilitiesFor the software, the WBS depends on the software architecture (discuss next)
38Work Breakdown Structure Example: work packages by project phase
39Milestone PlanningMilestone planning is used to show the major steps that are needed to reach the goal on timeMilestones typically mark completion of key deliverables or establishment of baselinesBaseline: when a work product is put under configuration management and all changes are controlledOften associated with management review pointsE.g., Requirements baseline, project plan complete, code ready to testCan use Gantt or PERT charts, put milestones in Assembla
40Gantt Charts Method for visualizing a project schedule showing The set of tasksStart and completion timesTask dependenciesResponsibilities
42DSD Project Plan Common project plan is key to coordination Clear definition of roles and responsibilitiesClear dependencies between tasks hence, what needs to be done nextProvides basis for tracking progressJust one part of necessary communication!Teams must agree on project plan but…Still easy to have misunderstanding about meaning of planStill may go off trackMust detect and correct as soon as possibleThis is not easyPlan must be continuously updated
43Plans vs. Reality (Rational vs. Irrational) …...ABCDGFEZLKJIHM…In an ideal world, the project proceeds as planned, moving from one milestone/artifact to the next. Project manager is happy.
44Plans vs. Reality (Rational vs. Irrational) …...ABCD…EFGMZHIJA’C’…KLIn the real world, errors, requirements and other changes cause backtracking and replanning. (Green lines represent backtracking.) Project manager very unhappy. Can’t help this. That’s the way that the world is.A’’B’C’’D’H’PlannedActual w/ backtracking
45Plans vs. Reality Making The Reality Seem Rational Document as if it had been rationalReaders can follow a sequential storyExplain significant changeable decisionsWhat alternatives were (are) there?Why did we choose A’’ rather than A or A’Later readers (maintainers) understand the trade-offs and can be guided by them
46Plans vs. Reality Documenting The Decisions …...A’’B’C’’D’GFEZLKJIH’M…(B)(C,C,’)(D)(A,A’)Make the reality appear to be rational by documenting reasons for backtracking. Why did we do A” instead of A or A’? The documentation should explain reasons for key decisions and the alternatives that were considered. Hard to do, but important. Future developers need it.(H)
47MeasurementMeasuring progress is an important part of any real project planAddressed in Prof. Zhou Minghui’s lectures
48SummaryProcesses provide methods for managing software developments over timeMust choose the right process to address a project’s specific problems and constraintsIncremental process provide the feedback between distributed teams required for DSDProcesses must be defined and realized in a project planThe project plan must evolve as the project progresses
50Lab Exercise Use some of the assembla project management tools Assume we will do a small software development project with your distributed teamCommunicate with your team members to choose roles for each personRequirements engineer, architect, project manager, coder, reviewer, testerOne person may have multiple roles and vice versaEach person should try out two or three rolesAdd a milestone to define rolesAdd roles to your wiki page