1. Week #7 Analysis of Requirements Documents

2. What Is Requirements Analysis? ● Validation – Stakeholders agree that the requirements document matches their needs/wants ● Consistency – Different parts of the document 'line up' with one another; a program can be written that implements every part completely ● Traceability – Backward: any part of the document can be traced back to its source – Forward: any part can be matched to design elements

3. Plan for the Lecture ● 'Classical' view of analysis – We have a complete document and no code – Survey of both informal and formal approaches to analysis ● 'Agile' view – Document and design are constructed together ● Simple prototypes test for validity ● More complete prototypes check consistency ● Traceability is a map into the code-base

4. Validation ● Inspection of document ● Facilitated by standard, hierarchical document structure – No 'surprises' – No large chunks of text to digest and try to place in context ● For example: IEEE requirements documentation standard

5. Desiderata for Specifications ● Valid (or “correct”) – Expresses only the real needs of the stakeholders (customers, users,…) – Doesn’t contain anything that isn’t “required” ● Unambiguous – Every statement can be read in exactly one way ● Complete – Specifies all the things the system must do –...and all the things it must not do! – Conceptual Completeness  E.g. responses to all classes of input – Structural Completeness  E.g. no TBDs!!! ● Understandable (Clear) – E.g. by non-computer specialists ● Consistent – Doesn’t contradict itself  I.e. is satisfiable – Uses all terms consistently ● Ranked – Must indicate the importance and/or stability of each requirement ● Verifiable – A process exists to test satisfaction of each requirement – “every requirement is specified behaviorally” ● Modifiable – Can be changed without difficulty  Good structure and cross-referencing ● Traceable – Origin of each requirement must be clear – Facilitates referencing of requirements in future documentation Source: Adapted from IEEE-STD-830-1998

6. Source: Adapted from Kovitz, 1999 Typical mistakes  Noise  the presence of text that carries no relevant information to any feature of the problem.  Silence  a feature that is not covered by any text.  Over-specification  text that describes a feature of the solution, rather than the problem.  Contradiction  text that defines a single feature in a number of incompatible ways.  Ambiguity  text that can be interpreted in at least two different ways.  Forward reference  text that refers to a feature yet to be defined.  Wishful thinking  text that defines a feature that cannot possibly be validated.  Jigsaw puzzles  e.g. distributing requirements across a document and then cross-referencing  Duckspeak requirements  Requirements that are only there to conform to standards  Unnecessary invention of terminology  E.g., ‘the user input presentation function’, ‘airplane reservation data validation function’  Inconsistent terminology  Inventing and then changing terminology  Putting the onus on the development staff  i.e. making the reader work hard to decipher the intent  Writing for the hostile reader  There are fewer of these than friendly readers

7. Ambiguity Test ● Natural Language – 'The system shall report to the operator all faults that originate in critical functions or that occur during execution of a critical sequence and for which there is no fault recovery response.' (adapted from the specifications for the international space station) ● Or a decision table? Source: Adapted from Easterbrook & Callahan, 1997.

8. Avoiding ambiguity ● Review natural language specs for ambiguity – use people with different backgrounds – include software people, domain specialists and user communities – Must be an independent review (I.e. not by the authors!) ● Use a specification language – E.g. a restricted subset or stylized English – E.g. a semi-formal notation (graphical, tabular, etc) – E.g. a formal specification language (e.g. Z, VDM, SCR, …)

9. Avoiding Ambiguity, II ● Exploit redundancy – Restate a requirement to help the reader confirm her understanding –...but clearly indicate the redundancy – May want to use a more formal notation for the re-statement

10. Organizing the Requirements ● Need a logical organization for the document – IEEE standard offers different templates ● Example Structures - organize by… – …External stimulus or external situation ● e.g., for an aircraft landing system, each different type of landing situation: wind gusts, no fuel, short runway, etc – …System feature ● e.g., for a telephone system: call forwarding, call blocking, conference call, etc – …Subsystem ● e.g. for spacecraft: command&control, data handling, comms, instruments, etc.

11. Organizing Requirements, II – …System response ● e.g., for a payroll system: generate pay-cheques, report costs, print tax info; – …User type ● e.g. for a project support system: manager, technical staff, administrator, etc. – …External object ● e.g. for a library information system, organize by book type – …Mode ● e.g. for word processor: page layout mode, outline mode, text editing mode, etc

12. IEEE Standard for SRS 1 Introduction Purpose Scope Definitions, acronyms, abbreviations Reference documents Overview 2 Overall Description Product perspective Product functions User characteristics Constraints Assumptions and Dependencies 3 Specific Requirements Appendices Index 1 Introduction Purpose Scope Definitions, acronyms, abbreviations Reference documents Overview 2 Overall Description Product perspective Product functions User characteristics Constraints Assumptions and Dependencies 3 Specific Requirements Appendices Index Identifies the product, & application domain Describes contents and structure of the remainder of the SRS Describes all external interfaces: system, user, hardware, software; also operations and site adaptation, and hardware constraints Summary of major functions Anything that will limit the developer’s options (e.g. regulations, reliability, criticality, hardware limitations, parallelism, etc) All the requirements go in here (i.e. this is the body of the document). IEEE STD provides 8 different templates for this section Source: Adapted from IEEE-STD-830-1993 See also, Blum 1992, p160

13. IEEE STD Section 3 (example) 3.1 External Interface Requirements 3.1.1 User Interfaces 3.1.2 Hardware Interfaces 3.1.3 Software Interfaces 3.1.4 Communication Interfaces 3.2 Functional Requirements this section organized by mode, user class, feature, etc. For example: 3.2.1 Mode 1 3.2.1.1 Functional Requirement 1.1 … 3.2.2 Mode 2 3.2.1.1 Functional Requirement 1.1 …... 3.2.2 Mode n... 3.3 Performance Requirements Remember to state this in measurable terms! 3.4 Design Constraints 3.4.1 Standards compliance 3.4.2 Hardware limitations etc. 3.5 Software System Attributes 3.5.1 Reliability 3.5.2 Availability 3.5.3 Security 3.5.4 Maintainability 3.5.5 Portability 3.6 Other Requirements Source: Adapted from IEEE-STD-830-1993. See also, Blum 1992, p160

14. Source: Adapted from Blum, 1992, pp369-373 Reviews, Inspections, Walkthroughs… ● Note: these terms are not widely agreed – formality ● informal: from meetings over coffee, to team get-togethers ● formal: scheduled meetings, prepared participants, defined agenda, specific format, documented output – Management reviews ● E.g. preliminary design review (PDR), critical design review (CDR) ● Used to provide confidence that the design is sound ● Attended by management and sponsors (customers) ● Usually a 'dog-and-pony show'

15. Reviews, Inspections, Walkthroughs II ● Walkthroughs ● developer technique (usually informal) ● used by development teams to improve quality of product ● focus is on finding defects – (Fagan) Inspections ● a process management tool (always formal) ● used to improve quality of the development process ● collect defect data to analyze the quality of the process ● written output is important ● major role in training junior staff and transferring expertise

16. Source: Adapted from Blum, 1992, pp369-373 & Freedman and Weinberg, 1990. Benefits of formal inspection ● Formal inspection works well for programming: – For applications programming: ● more effective than testing ● most reviewed programs run correctly first time ● compare: 10-50 attempts for test/debug approach – Data from large projects ● error reduction by a factor of 5; (10 in some reported cases) ● improvement in productivity: 14% to 25% ● percentage of errors found by inspection: 58% to 82% ● cost reduction of 50%-80% for V&V (even including cost of inspection)

17. Inspection Benefits, II – Effects on staff competence: ● increased morale, reduced turnover ● better estimation and scheduling (more knowledge about defect profiles) ● better management recognition of staff ability ● These benefits also apply to requirements inspections – E.g. See studies by Porter et. al.; Regnell et. al.

18. Source: Adapted from Blum, 1992, pp369-373 & Freedman and Weinberg, 1990. Inspection Constraints ● Size – “enough people so that all the relevant expertise is available” – min: 3 (4 if author is present) – max: 7 (less if leader is inexperienced) ● Duration – never more than 2 hours  concentration will flag if longer ● Outputs – all reviewers must agree on the result  accept; re-work; re-inspect; – all findings should be documented  summary report (for management)  detailed list of issues ● Scope – focus on small part of a design, not the whole thing ● Timing – Examines a product once its author has finished it – not too soon  product not ready - find problems the author is already aware of – not too late  product in use - errors are now very costly to fix ● Purpose – Remember the biggest gains come from fixing the process  collect data to help you not to make the same errors next time

19. Source: Adapted from Freedman and Weinberg, 1990. Inspection Guidelines ● Prior to the review – schedule Formal Reviews into the project planning – train all reviewers – ensure all attendees prepare in advance ● During the review – review the product, not its author ● keep comments constructive, professional and task- focussed – stick to the agenda ● leader must prevent drift – limit debate and rebuttal ● record issues for later discussion/resolution

20. Inspection Guidelines, II – identify problems but don't try to solve them – take written notes ● After the review – review the review process

21. Choosing Reviewers ● Possibilities – specialists in reviewing (e.g. QA people) – people from the same team as the author – people invited for specialist expertise – people with an interest in the product – visitors who have something to contribute – people from other parts of the organization Source: Adapted from Freedman and Weinberg, 1990.

22. Choosing Reviewers, II ● Exclude – anyone responsible for reviewing the author ● i.e. line manager, appraiser, etc. – anyone with known personality clashes with other reviewers – anyone who is not qualified to contribute – all management – anyone whose presence creates a conflict of interest

23. Source: Adapted from Porter, Votta and Basili, 1995 Structuring the inspection ● Can structure the review in different ways – Ad-hoc ● Rely on expertise of the reviewers – Checklist ● uses a checklist of questions/issues ● checklists tailored to the kind of document (Porter et. al. have examples) – active reviews (perspective based reading) ● each reviewer reads for a specific purpose, using specialized questionnaires ● effectively different reviewers take different perspectives

24. Structuring the Inspection, II ● The differences may matter – E.g. Porter et. al. study indicates that: ● active reviews find more faults than ad hoc or checklist methods ● no effective different between ad hoc and checklist methods ● the inspection meeting might be superfluous!

25. Agile Validation ● Use-case-wise – Elicit a use-case (including exceptional behaviour) – Prototype it – Validate the prototype – Increment the prototype with the next use-case ● Validate the new use-case ● Check for interactions with already-validated ones – Check NFRs at each cycle ● Aspect-wise

26. Kinds of Consistency ● Between parts of the same kind of representation (in-viewpoint) – Say constraints in the object model.. – All pairs of constraints must be mutually satisfiable ● Between different kinds of representation (inter-viewpoint) – Statechart and object model – Use-case and statechart –...

27. In-Viewpoint Consistency ● How prone is a representation to introduction of accidental inconsistency? – Natural language: extremely so – Single simple diagram: mostly much less – Set of simple diagrams, hierarchically structured (like DFDs, Statecharts): more than a single diagram – Set of non-hierarchically structured diagrams (such as MSCs): more than hierarchical diagrams – Logical theory (such as OCL): fairly prone; but automated detection is facilitated

28. Inter-Viewpoint Consistency (informally) ● Consider 3D models as an example ● How to represent a cube in 2D: Top View Perspective View

29. Inter-Viewpoint Consistency (formally) ● Rely on semantics, not intuition ● Consider views of the cube again – Each has a semantic domain: P( ) – A complete description picks out a single element of the semantic domain – A viewpoint is not complete (in general): many elements of the domain may be consistent with it ● The perspective could mean a complete cube ●.. or just the three faces visible with nothing behind!

30. UML Examples ● Statechart/Object model – Domain: states of each object, events (source, target, parameters) ● Each state in an object's statechart must – Be describable by a predicate over the object's state-space – If it produces events, each must be an operation of either that object, or one it is associated with ● If s is an OR-substate of t, then the predicate describing s must imply the predicate describing t ●...

31. UML Examples, II ● Use-case/message sequence chart – Domain: finite sequences of messages and responses ● Types must match ● Creation events must match ● Step in UC must represent fragment of MSC ● If either the UC or MSC is more detailed, then they must agree on the common events

32. Elements of Formal Consistency ● What do we need? – A clearly-defined semantic domain ● Does not need to be the most detailed one; the richer, the harder to check consistency ● Should also be an abstraction of the concrete (program) domain – A mapping from each notation into the semantic domain ● May be imprecise (one diagram = many elements of semantic domain) and probably will be – Consistency rules ● That guarantee (or at least strongly suggest) that the diagrams are consistent, without building big state- machines

33. Semantic Domains (From Concrete to Abstract) ● Concrete program state-space – All possible assignments to global variables – All possible heap configurations – All possible program locations (one location per thread) ● Behaviours are possible sequences of program states ● If multi-threaded, thread scheduling must be modelled explicitly ● Statechart state-space – All possible activations – All possible triggered events ● Behaviours are possible sequences of statechart states

34. Semantic Domains, II ● Object model state-space (constraints) – All possible instantiations of objects – All possible assignments to attributes of each object (possible = allowed by constraints) – All possible values of associations – Events being sent between compatible objects ● Behaviours are all sequences of states allowed by pre/postcondition constraints ● Object model state-space (no constraints) – Same but less restricted – Behaviours are all type-correct sequences of attributes and messages

35. Handling Inconsistency ● Most commonly inconsistency is just a memory lapse – Event in object model returns type A – Statechart assumes type B (not a superclass/subclass of A) – Analyst forgot the type of event, picked one that seemed right ●..but it can be a sign of requirements needing further capture – Need a common superclass of A and B; they have something in common in the domain model

36. Backward Traceability ● Each requirement must be justified – Belongs to at least one stakeholder – Can be explained in terms of goals – Does not conflict with any others

37. Forward Traceability ● Each requirement must map into the program – What is a 'single requirement'? ● Paragraph of text? ● Diagram – Functional requirements map to modules of the program – Non-functional requirements do not map so well.. ● Aspect-oriented programming attempts to mitigate this ● Good commenting, and use of relationships between quality factors and design decisions helps too

38. Forward Traceability Questions ● Does every requirement correspond to some element of the program? ● (More difficult..) Does every design decision/module/line of code trace back to one or more requirements?

39. Conflicts In General ● Requirements R, S are inconsistent when satisfying R makes S impossible to satisfy ● Assuming satisfaction is binary ● If satisfaction is measurable (such as with performance criteria), other relationships exist – Positive correlation (++, +) – Negative correlation (--, -) – Unspecified correlation (relationship undetermined) – Definitely no correlation

40. Example of Conflict ● Requirement: Quick server response – “File server shall respond to requests within 0.05s” ● Requirement: Serve 200 connections – “File server shall serve up to 200 connections simultaneously” ● Consistent, but under some circumstances may be impossible to satisfy both – These circumstances are called obstacles

41. Correlation Types ● Structural – R is similar to S ● Resource – Both depend on the same resource ● Task – R describes a dependent task of S ● Causality – R describes a consequence of S

42. Semantic Conflicts ● Perception – High level goals of agents conflict – Agents' assumptions conflict – Agents have different views of domain boundary ● Communication – Agents use different levels of abstraction – Agents' positions have different levels of precision – One agent's justification appears circular ● Resource/behaviour

43. Frameworks for Negotiation ● What is required for negotiation? – Documentation on some level of initial approximation to requirements, structured to allow browsing and analysis – Ways for stakeholders to identify issues with a given requirement of another stakeholder – Ways to generate possible resolution of issues (options) – Documentation of agreement – Historical view: agreement on an option does not make the issue disappear, simply resolves it

44. WinWin: an Example Framework ● Developed by Barry Boehm out of decades of research and practice ● Basic ideas: – Pragmatic approach: use what works and is readily comprehensible – Cast negotiation as a non-zero-sum game: the goal is to find conditions under which every stakeholder wins – The process cannot completely enforce this, but should encourage it as much as possible ● Empirical studies support this hypothesis

45. WinWin Activities ● Review/expand negotiation topics – Develop a structure and taxonomy of requirements – IEEE templates can be a guide here ● Brainstorm stakeholder interests – Brainstorming is structured via software as multiple discussion threads, started by facilitator – Users are periodically switched to another discussion thread to keep from spending too much time on one topic – Threads are keyword-indexed and linked in real time

46. WinWin Activities, II ● Converge on win conditions – Each stakeholder prompted to fix one (or a small number of) win conditions as a basis for negotiation – Each win condition must be accepted as such by all stakeholders: this is just agreeing that they understand it and believe it makes sense, not that they accept it as a requirement! – No contesting or endorsement of win conditions at this stage

47. WinWin Activities, III ● Capture a glossary of terms – Familiar activity from elicitation.. ● Prioritize win conditions – Evaluate each for business importance and feasibility – Results in 4 broad categories: 'low hanging fruit', 'important, with hurdles', 'maybe later', 'forget it' ● Determine conflicts – Stakeholders identify and explain disagreements – Explanation may uncover ambiguity: not a 'real' disagreement

48. WinWin Activities IV ● Identify issues and options – Prioritize issues by contentiousness ● How many different requirements are involved? ● How severe is the conflict, and of what type? – Identifying options is the hardest task! – Choosing the best option is second hardest.. ● Reach agreements on options – If option T resolves an issue between R and S, then the agreement between stakeholders that it replaces the old R and S is recorded – Record of this agreement is part of traceability

49. Searching for Resolution ● Relaxation – Stakeholders loosen their requirements to find mutually-satisfactory option ● Refinement – Conflicting requirements are split into sub-goals, only some of which are in conflict ● Restructuring – Instead of changing the requirements, can some change to the problem domain be specified? – Or, can the requirements be limited to a specific context where no obstacles exist?

50. More Resolutions ● Abandonment ● Postponement ● Creative compromise by facilitator

51. Decision Theory ● Alternatives are ordered by a preference relation ● Preference relation translates to a normalized utility function assigning a value between 0 and 1 to each alternative ● If there are multiple criteria for evaluation, each has a preference relation (and corresponding utility function) ● A social choice or preference aggregation function combines the criteria

52. Decision Theory: Pro ● In small domains, enables automated search for options ● Clearly impartial and authoritative ● Requires clear prioritization and foregrounding of decision criteria

53. Decision Theory: Con ● Eliciting preferences from stakeholders is time-consuming ● Constraint satisfaction in the software domain is not, in general, computationally feasible ● Arrow's Theorem: finding a preference aggregation that is simultaneously fair and democratic is impossible