Presentation on theme: "«CSI5112» D. Amyot U. Ottawa CSI 5112 REQUIREMENTS ENGINEERING (Part I) Adapted from: Lethbridge & Laganière 2001, Kontoya & Sommerville 1998, Amyot 2005-2014,"— Presentation transcript:
«CSI5112» D. Amyot U. Ottawa CSI 5112 REQUIREMENTS ENGINEERING (Part I) Adapted from: Lethbridge & Laganière 2001, Kontoya & Sommerville 1998, Amyot , Others where cited...
«CSI5112» D. Amyot U. Ottawa Requirements Engineering2 The beginning is the most important part of the work. Plato, 4 B.C.
«CSI5112» D. Amyot U. Ottawa Requirements Engineering3 Overview This presentation Part I: —Motivation and definitions —Types of requirements —Processes and activities Part II: Writing better requirements Part III: Overview of RE techniques Other presentations Requirements Management User Requirements Notation
«CSI5112» D. Amyot U. Ottawa Requirements Engineering4 Mars Climate Orbiter In 1999, the Mars Climate Orbiter disappears around Mars Cost: about $125M US. Problem caused by a misunderstanding between a team in Colorado and one in California. One team used the metric system while the other used the English system of a key function…
«CSI5112» D. Amyot U. Ottawa Requirements Engineering5 GIRES GIRES (Gestion intégrée des ressources) — integrated management of resources, — to replace >1000 existing systems, — in 140 organisations / departments, — affecting employees! 8-year project of the Quebec government, started $80 million budget. Could not cope with changes to the requirements… — Cost of $400 millions after 5 years, and very late. — Project cancelled in
«CSI5112» D. Amyot U. Ottawa Requirements Engineering6 Canadian Gun Registry Law adopted in Was supposed to cost $119M, with revenues of $117M (net cost of $2M). 30 types of permits, long questionnaires, 90% of errors in requests Rising costs ($327M in 2000, $688M in 2002, plus others…) Many political and legal issues, and a few scandals… Customer asked for over 2000 changes in the computer system! ~$1G in 2004, even more by the time the system is fully functional. Tons of unhappy customers and taxpayers… Not required to register as of May 17, 2006! Bill C-391 An Act to amend the Criminal Code and the Firearms Act (repeal of long-gun registry). Passed in April 2013…
«CSI5112» D. Amyot U. Ottawa You Think One Would Learn… List of badly managed IT projects in governments (Quebec and Canada) iques_dans_les_organismes_publiques iques_dans_les_organismes_publiques In French, but you will get the idea. To be remembered when you pay your income taxes! SAGIR is one in the making… Follow-up to GIRES started in 2005 Was supposed to be done in 2007 for $83M Not finished, >$1G, obsolete, restarted now informatique-prendra-t-il-fin-en informatique-prendra-t-il-fin-en-2015 Requirements Engineering7
«CSI5112» D. Amyot U. Ottawa Requirements Engineering8 You said “Requirements”? A requirement is… an expression of the ideas to be embodied in the system or application under development a statement about the proposed system that all stakeholders agree must be made true in order for the customer’s problem to be adequately solved. —Short and concise piece of information —Says something about the system —All the stakeholders have agreed that it is valid —It helps solve the customer’s problem
«CSI5112» D. Amyot U. Ottawa Requirements Engineering9 According to IEEE A requirement is defined as: (1)a condition or capability needed by a user to solve a problem or achieve an objective; (2)a condition or a capability that must be met or possessed by a system … to satisfy a contract, standard, specification, or other formally imposed document …”
«CSI5112» D. Amyot U. Ottawa Requirements Engineering10 You said “Requirements Engineering”? Requirements Engineering (RE) is… The activity of development, elicitation, specification, analysis, and management of the stakeholder requirements, which are to be met by systems RE is concerned with identifying the purpose of a software system… and the contexts in which it will be used.
«CSI5112» D. Amyot U. Ottawa Requirements Engineering11 Requirements Engineering Activities Requirements Engineering Requirements Development Requirements Management Elicitation Analysis Specification Verification Larry Boldt, Trends in Requirements Engineering People-Process-Technology, Technology Builders, Inc., 2001
«CSI5112» D. Amyot U. Ottawa Requirements Engineering12 About these RE Activities… Requirements elicitation Requirements discovered through consultation with stakeholders Requirements analysis and negotiation Requirements are analyzed and conflicts resolved through negotiation Requirements specification A precise requirements document is produced Requirements validation The requirements document is checked for consistency and completeness Requirements management Needs and contexts evolve, and so do requirements!
«CSI5112» D. Amyot U. Ottawa Perspectives on RE from SWEBOK 3.0 Requirements Engineering13
«CSI5112» D. Amyot U. Ottawa Requirements Engineering14 Statistics from NIST Report NIST (National Institute of Standards and Technology's) has published a comprehensive (309 pages) and very interesting report on project statistics and experiences based on data from a large number of software projects 10.htm (May 2002).http://www.nist.gov/public_affairs/releases/n htm 70 % of the defects are introduced in the specification phase 30 % are introduced later in the technical solution process. Only 5 % of the specification inadequacies are corrected in the specification phase 95 % are detected later in the project or after delivery where the cost for correction in average is 22 times higher compared to a correction directly in the specification effort. The NIST report concludes that extensive testing is essential, however testing detects the dominating specification errors late in the process.
«CSI5112» D. Amyot U. Ottawa Requirements Engineering15 (Martin & Leffinwell) Distribution of Effort to Fix Defects Code 7% Other 10% Design 27% Requirements 56% Code 1% Other 4% Design 13% Requirements 82% Distribution of Defects Why Focus on Requirements ?
«CSI5112» D. Amyot U. Ottawa Requirements Engineering16 CHAOS Report, 2004
«CSI5112» D. Amyot U. Ottawa Requirements Engineering17 Progression since 1994 Standish Group Inc., ~50,000 projects over 14 years Note: these numbers are criticized in: J.L. Eveleens and C. Verhoef, “The Rise and Fall of the Chaos Report Figures”. IEEE Software, 27(1), Jan/Feb 2010, pp
«CSI5112» D. Amyot U. Ottawa CHAOS Report 2012 Requirements Engineering18
«CSI5112» D. Amyot U. Ottawa Success Factors, Requirements Engineering19
«CSI5112» D. Amyot U. Ottawa Success Factors, 2012 Requirements Engineering20
«CSI5112» D. Amyot U. Ottawa A Different View (2014) Requirements Engineering21 S. Fricker, R. Grau, A. Zwingli (2014). Requirements Engineering: Best Practice. Requirements Engineering for Digital Health. Springer.Requirements Engineering: Best Practice
«CSI5112» D. Amyot U. Ottawa Requirements Engineering22 A Different View (2014) S. Fricker, R. Grau, A. Zwingli (2014). Requirements Engineering: Best Practice. Requirements Engineering for Digital Health. Springer.Requirements Engineering: Best Practice Companies still think they do too little RE…
«CSI5112» D. Amyot U. Ottawa Dilbert on User Involvement Requirements Engineering23
«CSI5112» D. Amyot U. Ottawa Requirements Engineering24 Managing Evolving Requirements Source: 1999http://standishgroup.com/sample_research/PDFpages/extreme_chaos.pdf “Changing requirements is as certain as death and taxes”
«CSI5112» D. Amyot U. Ottawa Requirements Engineering25 Time Why? What? How? Who? When? If-Then Does It? Where? Project Management Process Quality Management Process Component & Configuration Management Process Risk Management Process Identify Business Needs and Goals Derive User & Functional Requirements Design Solutions TIME RE Process and Related Activities
«CSI5112» D. Amyot U. Ottawa Requirements Engineering26 Overview This presentation Motivation and definitions Types of requirements Processes and activities
«CSI5112» D. Amyot U. Ottawa Requirements Engineering27 So Many “Requirements”… A goal is an objective or concern used to discover and evaluate functional and non-functional requirements. A goal is not yet a requirement… A functional requirement is a requirement defining functions of the system under development Describes what the system should do A non-functional requirement is a requirement characterizing a system property such as expected performance, robustness, usability, maintainability, etc. Constraints that must be adhered to during development A user requirement is a desired goal or function that a user and other stakeholders expect the system to achieve Does not become necessarily a system requirement A domain requirement is a requirement derived from the application domain Can be functional or non-functional
«CSI5112» D. Amyot U. Ottawa Requirements Engineering28 Functional Requirements What inputs the system should accept What outputs the system should produce What data the system should store that other systems might use What computations the system should perform The timing and synchronization of the above Examples: —The system shall support searching a subset of the initial set of databases. —The system shall provide a PDF viewer for the user to read documents in the document store.
«CSI5112» D. Amyot U. Ottawa Requirements Engineering29 Non-Functional Requirements (NFR) Also called quality requirements, or extra-functional requirements. Three main types (according to L&L): 1. Categories reflecting: usability, efficiency, reliability, maintainability and reusability —Response time —Throughput —Resource usage —Reliability —Availability —Recovery from failure —Allowances for maintainability and enhancement —Allowances for reusability
«CSI5112» D. Amyot U. Ottawa Requirements Engineering30 Non-Functional Requirements (NFR) 2. Categories constraining the environment and technology of the system. —Platform (minimal requirements, OS, devices…) —Technology to be used (language, DB, …) 3. Categories constraining the project plan and development methods —Development process (methodology) to be used —Cost and delivery date -Often put in contract or project plan instead
«CSI5112» D. Amyot U. Ottawa Requirements Engineering31 Various NFR Types (Sommerville & Kontoya, 1998) Other ontologies also exist.
«CSI5112» D. Amyot U. Ottawa Examples of Non-Functional Requirements Product requirement It shall be possible for all necessary communication between the system and the user to be expressed in the standard ISO Latin 1 character set. Process requirement The system development process and deliverable documents shall conform to the process and deliverables defined in MIL-STD-498.MIL-STD-498 Security requirement The system shall not disclose any personal information about customers apart from their name and reference number to the operators of the system. Requirements Engineering32
«CSI5112» D. Amyot U. Ottawa Four Challenges with NFRs 1) Selecting appropriate and relevant categories E.g., Learnability 2) Selecting appropriate metrics E.g., Number of hours of training to execute tasks with a certain success rate. 3) Selecting appropriate values E.g., The system shall require fewer than 3 hours of training for beginners to solve a puzzle correctly 95% of the time 4) Specifying our level of confidence in that NFR At this time: 40% sure about it. Requirements Engineering33
«CSI5112» D. Amyot U. Ottawa False/True Positives/Negatives… Confusion Matrix 34 ? Requirements Engineering Techniques [http://en.wikipedia.org/wiki/Sensitivity_and_specificity]
«CSI5112» D. Amyot U. Ottawa Some Requirements Are Domain Specific Surveillance of seal population in the North Transmitting tags attached to seals, to track their locationTransmitting tags Hard to attach a transmitter (and dangerous!) Project over several years Problem: batteries last several weeks, while the project spans many years Options proposed for the tags: 1. Transmit every 40 minutes for 9 months (4500$) 2. Transmit every 60 minutes for 1 year (4500$) 3. Transmit every 40 minutes for 2 years (5500$) Domain constraints… Tags are attached to the fur of seals Seals lose their fur every 9 months!!! Requirements Engineering35
«CSI5112» D. Amyot U. Ottawa Challenges of Domain-Specific Requirements Understandability Requirements are expressed in the language of the application domain This is often not understood by software engineers developing the system Implicitness / Tacit Knowledge Domain specialists understand the area so well that they do not think of making the domain requirements explicit People are often unaware of the tacit knowledge they possess and therefore cannot express it to others “It’s obvious” “Assume” Requirements Engineering36
«CSI5112» D. Amyot U. Ottawa Requirements Engineering37 Non-functional requirements may be very difficult to state precisely (especially at the beginning) and imprecise requirements may be difficult to verify. Goal Convey the intention or the objective of one or many stakeholders. Can guide the discovery of verifiable non-functional requirements that can be tested objectively. Goals
«CSI5112» D. Amyot U. Ottawa Requirements Engineering38 Overview This presentation Motivation and definitions Types of requirements Processes and activities
«CSI5112» D. Amyot U. Ottawa Requirements Engineering39 Requirements and Modeling Hull, Jackson, Dick: Requirements Engineering, 2004
«CSI5112» D. Amyot U. Ottawa Requirements Engineering40 Typical Layered Approach (V-shaped) Hull, Jackson, Dick: Requirements Engineering, 2004
«CSI5112» D. Amyot U. Ottawa Requirements and Iterative, Step-Wise Development Requirements Engineering41 [Lamsweerde]
«CSI5112» D. Amyot U. Ottawa Requirements Engineering42 Validation vs. Verification
«CSI5112» D. Amyot U. Ottawa Requirements Engineering43 The Requirements Analyst Plays an essential communication role talks to users: application domain talks to developers: technical domain translates user requirements into functional requirements and quality goals Needs many capabilities interviewing and listening skills facilitation and interpersonal skills writing and modeling skills organizational ability RE is more than just modelling… This is a social activity! Karl Wiegers – In Search of Excellent Requirements
«CSI5112» D. Amyot U. Ottawa CSI 5112 W RITING B ETTER R EQUIREMENTS (Part II) Sources: Ian Zimmerman (Telelogic, 2001) and Ivy Hooks (Compliance Automation, 1998) Gunter Mussbacher (2009)
«CSI5112» D. Amyot U. Ottawa Anatomy of a Good Requirement The challenge is to seek out the subject, positive end result, and success measure in every requirement you define! Requirements Writing45 The Online Banking System shall allow the Internet user to access her current account balance in less than 5 seconds. Defines the system under discussion Verb with correct identifier (shall or may) Defines a positive end result Quality criterion
«CSI5112» D. Amyot U. Ottawa Requirements Writing46 “The Internet user quickly sees the balance of her account on the laptop screen.” Cannot write a requirement on the user Vague quality criterionWhat versus how No identifier for the verb Example of a Bad Requirement X
«CSI5112» D. Amyot U. Ottawa Standard for Writing a Requirement Each requirement must first form a complete sentence — Not a bullet list of buzzwords Each requirement contains a subject and predicate —Subject: a user type or the system under discussion —Predicate: a condition, action, or intended result. Consistent use of the verb: —“shall,” “will”, or “must” to show mandatory nature —“should” or “may” to show optionality —Usually, shall and should are used. A requirement contains a success criterion or other measurable indication of the quality. A requirement describes what must be done, rather than how. —Although often “your what is my how”… Requirements Writing47
«CSI5112» D. Amyot U. Ottawa Standard for Writing a Requirement Several standards define fairly precisely how to use key words (verbs and adjectives) in their documents. Example: IETF RFC 2119: Key words for use in RFCs to Indicate Requirement LevelsIETF RFC 2119 MUST, REQUIRED or SHALL: mean that the definition is an absolute requirement of the spec. MUST NOT or SHALL NOT: absolute prohibition SHOULD or RECOMMENDED: think twice about not doing it! SHOULD NOT or NOT RECOMMENDED: think twice about doing it! MAY or OPTIONAL: truly optional 48Requirements Writing
«CSI5112» D. Amyot U. Ottawa Standard for Writing a Requirement Look for the following characteristics in each requirement Feasible (not wishful thinking) Needed (provides the specifics of a desired end goal or result) Testable (contains a success criterion/other measurable indication of quality) Clear, unambiguous, precise, one thought Prioritized With a unique identifier — Do not encode information in the identifier Note: several characteristics are mandatory (feasible, needed, testable) whereas others improve communication Requirements Writing49
«CSI5112» D. Amyot U. Ottawa Writing Pitfalls to Avoid Never describe how the system is going to achieve something (over-specification), always describe what the system is supposed to do Refrain from designing the system prematurely —Danger signs: using names of components, materials, software objects, fields & records in the user or system requirements Designing the system too early may increase system costs Do no mix different kinds of requirements (e.g., requirements for users, system, and how the system should be designed, tested, or installed) Do not mix different requirements levels (e.g., the system and subsystems) —Danger signs: high level requirements mixed in with database design, software terms, or very technical terms Beware: may depend on the level of abstraction Requirements Writing50
«CSI5112» D. Amyot U. Ottawa Writing Pitfalls to Avoid “What versus how” test The system shall use Microsoft Outlook to send an to the customer with the purchase confirmation. The system shall inform the customer that the purchase is confirmed. X Requirements Writing51
«CSI5112» D. Amyot U. Ottawa Writing Pitfalls to Avoid Never build in let-out or escape clauses Requirements with let-outs or escapes are dangerous because of problems that arise in testing Danger signs: if, but, when, except, unless, although —These terms may however be useful when the description of a general case with exceptions is much clearer and complete that an enumeration of specific cases Avoid ambiguity Write as clearly and explicitly as possible Ambiguities can be caused by: —The word or to create a compound requirement —Poor definition (giving only examples or special cases) —The words etc, …and so on (imprecise definition) Requirements Writing52
«CSI5112» D. Amyot U. Ottawa Writing Pitfalls to Avoid Do not use vague indefinable terms Many words used informally to indicate quality are too vague to be verified Danger signs: user-friendly, highly versatile, flexible, to the maximum extent, approximately, as much as possible, minimal impact The Easy Entry System shall be easy to use and require a minimum of training except for the professional mode. X Requirements Writing53
«CSI5112» D. Amyot U. Ottawa Writing Pitfalls to Avoid Do not make multiple requirements Keep each requirement as a single sentence Conjunctions (words that join sentences together) are danger signs: and, or, with, also Do not ramble Long sentences with arcane language References to unreachable documents The Easy Entry Navigator module shall consist of order entry and communications, order processing, result processing, and reporting. The Order Entry module shall be integrated with the Organization Intranet System and results are stored in the group’s electronic customer record. X Requirements Writing54
«CSI5112» D. Amyot U. Ottawa Writing Pitfalls to Avoid Do not speculate There is no room for “wish lists” – general terms about things that somebody probably wants Danger signs: vague subject type and generalization words such as usually, generally, often, normally, typically Do not express suggestions or possibilities Suggestions that are not explicitly stated as requirements are invariably ignored by developers Danger signs: may, might, should, ought, could, perhaps, probably Avoid wishful thinking Wishful thinking means asking for the impossible (e.g., 100% reliable, safe, handle all failures, fully upgradeable, run on all platforms) The Easy Entry System may be fully adaptable to all situations and often require no reconfiguration by the user. X Requirements Writing55
«CSI5112» D. Amyot U. Ottawa Rate these Requirements Invoices, acknowledgments, and shipping notices shall be automatically faxed during the night, so customers can get them first thing in the morning. The Order Entry system provides for quick, user- friendly and efficient entry and processing of all orders. Changing report layouts, invoices, labels, and form letters shall be accomplished. The system shall be upgraded in one whack. The system has a goal that as much of the IS data as possible be pulled directly from the T&M estimate. X X X X X Requirements Writing56
«CSI5112» D. Amyot U. Ottawa Towards Good Requirements Specifications Valid (or “correct”) Expresses actual requirements Complete Specifies all the things the system must do (including contingencies)...and all the things it must not do! Conceptual Completeness (e.g., responses to all classes of input) Structural Completeness (e.g., no TBDs!!!) Consistent Does not contradict itself ( satisfiable ) Uses all terms consistently Note: inconsistency can be hard to detect, especially in concurrency/timing aspects and condition logic Formal modeling can help Beneficial Has benefits that outweigh the costs of development Source: Adapted from Blum 1992, pp164-5 and the IEEE-STD Requirements Writing57
«CSI5112» D. Amyot U. Ottawa Towards Good Requirements Specifications Necessary Doesn’t contain anything that isn’t “required” Unambiguous Every statement can be read in exactly one way Clearly defines confusing terms (e.g., in a glossary) Uniquely identifiable For traceability and version control Verifiable A process exists to test satisfaction of each requirement “every requirement is specified behaviorally” Understandable (clear) E.g., by non-computer specialists Modifiable Must be kept up to date! Source: Adapted from Blum 1992, pp164-5 and the IEEE-STD Requirements Writing58
«CSI5112» D. Amyot U. Ottawa Typical Mistakes Noise the presence of text that carries no relevant information to any feature of the problem. Silence a feature that is not covered by any text. Over-specification text that describes a feature of the solution, rather than the problem. Contradiction text that defines a single feature in a number of incompatible ways. Ambiguity text that can be interpreted in at least two different ways. Forward reference text that refers to a feature yet to be defined. Wishful thinking text that defines a feature that cannot possibly be validated. Jigsaw puzzles e.g. distributing requirements across a document and then cross-referencing Inconsistent terminology Inventing and then changing terminology Putting the onus on the development staff i.e. making the reader work hard to decipher the intent Writing for the hostile reader There are fewer of these than friendly readers Source: Steve Easterbrook, U. of Toronto Requirements Writing59
«CSI5112» D. Amyot U. Ottawa Remember the key questions: “Why?” or “What is the purpose of this?” Feasible Needed Testable (Verifiable) Key Questions and Characteristics Requirements Writing60
«CSI5112» D. Amyot U. Ottawa A Few Syntactic Analysis Tools QuARS Quality Analyzer of Requirements Specification ARM Automated Requirement Measurement Tool alue-pf.htmlhttp://www.stcsig.org/quality/newsletters/NL0603/NL0603_Doc_V alue-pf.html TIGER Pro Tool to Ingest and Elucidate Requirements Requirements Writing61
«CSI5112» D. Amyot U. Ottawa CSI 5112 Requirements Engineering Techniques (Part III) Adapted from: Kontoya & Sommerville 1998, Lethbridge & Laganière 2001, Atlee, Day, and Berry 2004, Amyot & Somé , Others where cited...
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques63 Requirements Engineering Activities Requirements Engineering Requirements Development Requirements Management Elicitation Analysis Specification Verification
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques64 Typical Requirements Elicitation Approach…
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques65 What is Requirements Elicitation? Elicitation means “to bring out, to evoke, to call forth” Requirements elicitation is “the process of discovering the requirements for a system by communication with customers, system users and others who have a stake in the system development” Human activity involving interaction between human beings: —Clients, buyers, users —Domain experts, software engineers and architects —Market specialists, lawyers, people involved in related systems, anyone who can bring some added value
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques66 Elicitation Goals Determine sources of information Determine appropriate techniques to get it Get information on domain, problem, constraints Determine the scope and feasibility Produce a first document Mainly user requirements and elicitation notes Potentially incomplete, disorganized, inconsistent But we must start somewhere!
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques67 Elicitation Risks and Challenges Problems of scope System boundaries inadequately defined or defined too soon Unnecessary technical details Problems of understanding Stakeholder not sure of what is needed Stakeholder has trouble communicating needs Stakeholder does not understand capabilities and limitation of computing environment Stakeholder does not have full understanding of domain Stakeholders state conflicting requirements Problems of volatility Stakeholders will not commit to a set of written requirements
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques68 Elicitation Risks and Challenges Other typical issues Experts seldom available Finding an adequate level of precision/detail Common vocabulary often missing Requirements do not fall from the sky ! Sometimes hidden Sometimes too obvious, implicit, ordinary… Assume == “ass” of “u” and “me” Participants often lack motivation and resist to change We need much efforts and discussion to come up with a common agreement and understanding!
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques69 On Stakeholders Availability… Stakeholders are generally busy! Have priorities other than you Are rarely entirely disconnected from their daily routine and tasks See their participation to the elicitation process as a supplementary task Hence, you must have the support and commitment of managers (especially theirs!) You must also avoid being perceived as a threat: Loss of jobs caused by the improved system Loss of autonomy, powers, or privileges To the recognition and visibility of their work
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques70 RE: More an Art than Science
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques71 Sources of Requirements Various stakeholders Clients, customers, users (past and future), managers, domain experts, developers, marketing and QA people, lawyers, auditors, anyone who can bring added value! Pre-existing systems Not necessarily software systems Pre-existing documentation Competing systems Documentation about interfacing systems Standards, policies, collective agreements, legislation …
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques72 Elicitation Techniques Elicitation techniques Analysis of existing systems or documentation — Including data mining Observation, ethnography Questionnaires Interviewing Brainstorming, Joint Application Design (JAD), workshops and other participatory approaches Prototyping, pilot systems Use cases and scenarios —To be revisited later… See also: elicitation-procedures http://www.slideshare.net/hayriyesakarya/ elicitation-procedures
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques73 Analysis of Existing Systems Useful for new improved version Important to know: What is used, not used, or missing What works (keep?), what does not work (drop?) How the system is really used (with frequency and importance) and was supposed to be used, and how we would like to use it Users may not like the new system if it is too different (nostalgia!)
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques74 Observation and Ethnography Observation Get into the trenches, observe silently (not to get biased information) Shadow important potential users as they do their work —ask the user to explain everything he or she is doing Session videotaping (need permission) Beware of the observer effect! Ethnography “Writing the culture" Also attempts to discover social, human, and political factors, which may also impact requirements.
«CSI5112» D. Amyot U. Ottawa Interviewing (1/3) Three main objectives: Record information to be used as input to requirements analysis and modeling Discover information accurately and efficiently Reassure interviewee that his/her understanding of the topic has been explored, listened to, and valued Four important steps: Planning, interview session, consolidation, follow-up Many strategies for questioning 75Requirements Engineering Techniques
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques76 Interviewing (2/3) Interview as many types stakeholders as possible Not just clients and users Ask problem-oriented questions Ask about specific details, but… Detailed and solution-specific questions may miss the stakeholder’s real requirements. Example: —Would you like Word 2012, Excel 2012 or both? vs. —Would you like to do word processing, computations, or both? Watch for unanswerable questions… How do you tie your shoelaces? Please explain.
«CSI5112» D. Amyot U. Ottawa Interviewing (3/3) Hints: Make the interviewee comfortable and confident, no smartass! Record, but only with permission — Discourse might change on record… Balance interview goals with emerging opportunities Interview several people at once to create synergy — Exercise: Tell me 10 jokes each… Do not assume that stated needs are correct! Thank the participants (e.g., by ), ask 2-3 very short questions, and keep the door open to future interactions See interesting video: Requirements Engineering77
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques78 Brainstorming To invent a new way of doing things, or when there are many unknowns Roles: scribe, moderator, participants Two main steps: Storm: Generating as many ideas as possible (quantity, not quality) Encourage creativity (provoke!). Wild is good! Calm: Filtering out of ideas (combine, clarify, prioritize, improve…) to keep the best one(s). May require some voting or prioritization strategy. Joint Application Development (JAD) is a more structured and intensive brainstorming approach. More roles, steps, and forms…
«CSI5112» D. Amyot U. Ottawa Questionnaires and Surveys Some benefits: Can reach multiple people, maybe in an anonymous way Asynchronous, distributed, and can be quick to answer Cheap Challenges: Preparation time! Choice of questions: open-ended and close (lack of flexibility) Choice of answers and scales (nominal, intervals, Likert…); avoid centralist tendencies! Statistical significance during analysis Validity of questions (bias, ambiguities) Repetition and order of questions Determining suitable participants to invite (risk of bias, of fraud) Getting people to answer everything (exhaustion, unattractive…)! Requirements Engineering Techniques79
«CSI5112» D. Amyot U. Ottawa Types of Questions to Consider Demography questions (for classification) Age, country, occupation… For analysis from many angles Beware of re-identification risks if supposed to be anonymous Attitudinal questions What do you think of…? Do you agree with…? Scale with 4-6 values (no center) or 5-7 values (with neutral value) 1.Strongly agree 2.Agree somewhat 3.Neither agree nor disagree (Undecided) 4.Disagree somewhat 5.Strongly disagree Supplementary open questions (instructive, but qualitative) Optional/alternative questions, by population Redundant questions, for robustness… Requirements Engineering Techniques80
«CSI5112» D. Amyot U. Ottawa Analysis to Consider… In Advance! Will the survey be repeated (before/after, for comparison)? Determine the type of (statistical) analysis, e.g.: Statistical significance? Test your questionnaire and your analysis on a small group! See also this important video on surveys and questionnaires: Requirements Engineering Techniques81
«CSI5112» D. Amyot U. Ottawa Dilbert on Brainstorming Requirements Engineering Techniques82
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques83 Prototyping I’ll know it when I’ll see (or won’t see) it! Can help find new functionalities, discuss usability, and establish priorities. Prototypes can take many forms: Paper prototypes (see ) Screen mock-ups Interactive prototypes (e.g., with Flash/Shockwave) Models (executable) Pilot systems…
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques84 Prototyping: Some Design Decisions… Prototypes can be Evolutive: turned into a product incrementally Throw-away: less precise, throwable, and focusing on the less well-understood aspects of the system to design Fidelity levels High: more costly and difficult to change, but more precise verdicts about requirements Low: cheaper and faster to develop, but may not cover enough and may appear “unprofessional” to some stakeholders.
«CSI5112» D. Amyot U. Ottawa Comparison of a few techniques Requirements Engineering Techniques85 TechniqueGood forKind of dataPlusMinus QuestionnairesAnswering specific questions Quantitative and qualitative data Can reach many people with low resource The design is crucial. Response rate may be low. Responses may not be what you want InterviewsExploring issuesSome quantitative but mostly qualitative data Interviewer can guide interviewee. Encourages contact between developers and users Time consuming. Artificial environment may intimidate interviewee Focus groups and workshops Collecting multiple viewpoints Some quantitative but mostly qualitative data Highlights areas of consensus and conflict. Encourages contact between developers and users Possibility of dominant characters Naturalistic observation Understanding context of user activity QualitativeObserving actual work gives insight that other techniques cannot give Very time consuming. Huge amounts of data Studying documentation Learning about procedures, regulations, and standards QuantitativeNo time commitment from users required Day-to-day work will differ from documented procedures Source: Preece, Rogers, and Sharp “Interaction Design: Beyond human-computer interaction”, p214 See also this intro and comparison from Ying Chen :
«CSI5112» D. Amyot U. Ottawa Dilbert and Elicitation Requirements Engineering Techniques86
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques87 Requirements Engineering Activities Requirements Engineering Requirements Development Requirements Management Elicitation Analysis Specification Verification
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques88 Requirements Analysis The process of studying and analyzing the customer and the user needs to arrive at a definition of (software) requirements. Objectives detect and resolve conflicts between (user) requirements discover the bounds of the software and how it must interact with its environment negotiate priorities elaborate system requirements to derive software requirements Analysis is often combined with modelling
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques89 Typical Approaches Many approaches involve modeling to get a global picture of the requirements. Structured analysis (1970) Object-oriented analysis (1990) Problem Frames (1995) Scenario analysis Analysis can be on individual requirements as well Remember presentation on writing better requirements. More on analysis and feature interactions soon…
«CSI5112» D. Amyot U. Ottawa Modeling Notations Natural language (English) + No special training required - Ambiguous, verbose, vague, obscure... - No automation Ad hoc notation (bubbles and arrows) + No special training required - No syntax formally defined meaning not clear, ambiguous - No automation Semi-formal notation (URN, UML...) + Syntax (graphics) well defined + Partial common understanding, reasonably easy to learn + Partial automation - Meaning only defined informally - Still a risk of ambiguities Formal notation (Logic, SDL, Petri nets, FSM...) + Syntax & semantics defined + Great automation (analysis and transformations) - More difficult to learn & understand Requirements Engineering Techniques90
«CSI5112» D. Amyot U. Ottawa Modeling Notations (2) Informal language is better understood by all stakeholders Good for user requirements, contract But, language lacks precision Possibility for ambiguities Lack of tool support Formal languages are more precise Fewer possibilities for ambiguities Offer tool support (e.g. automated verification and transformation) Intended for developers Source (for decision table): Easterbrook and Callahan, 1997 Requirements Engineering Techniques91
«CSI5112» D. Amyot U. Ottawa Model Analysis By construction We learn by questioning and describing the system By inspection Execute/analyze the model in our minds Reliable? By formal analysis Requires formal semantics (mathematical) and tools Reliable (in theory), but expensive (for certain modeling approaches) By testing Execution, simulation, animation, test... Requires well-defined semantics and execution/simulation tools More reliable than inspection for certain aspects Possible to interact directly with the model (prototype) Requirements Engineering Techniques92
«CSI5112» D. Amyot U. Ottawa Requirements Negotiation Potential resolution strategies: Agreement Compromise Voting Definition of variants Overruling Mediation Arbitration Goal analysis (think GRL!) Conflict resolution hence often involves negotiation Negotiating a coherent set of requirements is not easy But it is one task of the requirements analyst Difficult to satisfy everyone, to achieve all goals, make good decisions! Requirements Engineering93
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques94 Prioritisation
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques95 Which Sector Should We Focus On? 0 A lot! Cost A lot! Value R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 R11 R12 R13 R14R15 R16 To avoid!
«CSI5112» D. Amyot U. Ottawa 96 https://gupea.ub.gu.se/bitstream/2077/1789/1/stenbeck_svensson_ pdf Example: Volvo Car Corporation
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques97 On Triage and Prioritization Must be simple and fast, for real adoption Must yield accurate and trustworthy results Must help: Make acceptable tradeoffs among goals of value, cost and time-to-market Project planning in allocating resources based on requirements importance to the project as a whole Determine when a requirements should become part of the product What to minimize/maximize?
«CSI5112» D. Amyot U. Ottawa Chemical tracking system priority = (value%)/((cost% * cost weight)+(risk% * risk weight)) Wiegers’ Prioritization example Requirements Engineering Techniques98
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques99 Pairwise Comparisons Finding scores and weights is difficult and subjective Potential solution: pairwise comparison Which requirements (A or B) is more important: A >> B Example approach Analytic Hierarchy Process [Karlsson et Ryan, 1997] New problems Large number of pairs —Solved using transitivity and other tricks! Dependencies between requirements —Can actually be used to further reduce the # of pairs —Ex: group many requirements as features Tool example: IBM Rational Focal PointFocal Point Visit Wikipedia! Analytic_Hierarchy_Process Prioritizing_Requirements_using_a_Cost-Value_Approach Video:
«CSI5112» D. Amyot U. Ottawa Challenge: Complexity Management! 100Requirements Engineering Techniques
«CSI5112» D. Amyot U. Ottawa Features Evolve Too! Feature Survival Charts give an overview of changes done (or decisions taken) during one or multiple projects. Also useful for process improvement. Source: Krzysztof Wnuk, Björn Regnell, Lena Karlsson: Visualization of Feature Survival in Platform ‐ Based Embedded Systems Development for Improved Understanding of Scope Dynamics, 3rd International Workshop on Requirements Engineering Visualization REV 08, Barcelona, Spain. Requirements Engineering Techniques101
«CSI5112» D. Amyot U. Ottawa 102
«CSI5112» D. Amyot U. Ottawa Quantitative Analysis of Feature Survival Charts Requirements Engineering Techniques103
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques104 Requirements Engineering Activities Requirements Engineering Requirements Development Requirements Management Elicitation Analysis Specification Verification
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques105 Requirements Specification Specification activity The invention and definition of a behaviour of a new, solution system such that it will produce the required effects in the problem domain Requirements Analysis defined the problem domain and the required effects Specification Document A document that clearly and precisely describes, each of the essential requirements, in a verifiable way.
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques106 IEEE Std IEEE Recommended Practice for Software Requirements Specifications It should help: Software customers to accurately describe what they wish to obtain; Software suppliers to understand exactly what the customer wants; Individuals to accomplish the following goals: —Develop a standard software requirements specification (SRS) outline for their own organizations; —Define the format and content of their specific software requirements specifications; —Develop additional local supporting items such as an SRS quality checklist, or an SRS writer’s handbook. Provides a set of potential, compliant templates
«CSI5112» D. Amyot U. Ottawa ISO/IEC/IEEE 29148:2011 Systems and software engineering — Life cycle processes — Requirements engineering — Provides a unified treatment of the processes and products involved in engineering requirements throughout the life cycle of systems and software. Harmonizes IEEE 830, SWEBOK, and 7 other standards. More emphasis on characteristics of good requirements, RE activities and processes, operations (and operation context), and different information items (including their structures) such as specification of requirements for stakeholders, systems and software. Complies with ISO/IEC and ISO/IEC Requirements Engineering Techniques107
«CSI5112» D. Amyot U. Ottawa SRS Outline Requirements Engineering Techniques108 Verification: This section provides the verification approaches and methods planned to qualify the software. The information items for verification are recommended to be given in a parallel manner with the information items in section 3.
«CSI5112» D. Amyot U. Ottawa Other Relevant Specification Standards CENELEC Internal Regulations Part 3: Rules for the structure and drafting of CEN/CENELEC Publications (ISO/IEC Directives – Part 2, modified), —See also ec-dir2%7Bed6.0%7Den.pdf for the 2011 version in English ec-dir2%7Bed6.0%7Den.pdf 109Requirements Engineering Techniques
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques110 Requirements Engineering Activities Requirements Engineering Requirements Development Requirements Management Elicitation Analysis Specification Verification
«CSI5112» D. Amyot U. Ottawa Dilbert and Validation 111Requirements Engineering Techniques
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques112 Requirements Verification and Validation Check that the product is being built right Check that the right product is being built Help ensure delivery of what the client wants Need to be performed at every stage during the process Elicitation: checking back with the elicitation sources —“So, are you saying that..... ?” Analysis: checking that the domain description and requirements are correct Specification: checking that the defined system requirement will meet the user requirements under the assumptions of the domain/environment. Checking conformity to well-formedness rules, standards…
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques113 Some V&V Techniques Simple checks Review, inspections Often use checklists Prototypes Logical analysis Functional test design Model-oriented V&V May be formal or not, automated or not
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques114 Typical Review / Inspection Steps Plan review The review team is selected and a time and place for the review meeting is chosen. Distribute documents The requirements document is distributed to the review team members Prepare for review Individual reviewers read the requirements to find conflicts, omissions, inconsistencies, deviations from standards and other problems. Hold review meeting Individual comments and problems are discussed and a set of actions to address the problems is agreed. Follow-up actions The chair of the review checks that the agreed actions have been carried out. Revise document The requirements document is revised to reflect the agreed actions. At this stage, it may be accepted or it may be re-reviewed
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques115 Example: Fagan Inspection (many variants exist) Note: the boss is not involved in the process!
«CSI5112» D. Amyot U. Ottawa Requirements Engineering Techniques116 Comments on Reviews and Inspections Advantages Effective (even after considering cost) Allow finding sources of errors (not only symptoms) Authors are more attentive when they know their work will be closely reviewed — Encourage them to conform to standards Familiarize large groups, diffusion of knowledge Risks Reviews can be dull and draining (need to be limited in time) Time consuming and expensive (but usually cheaper than the alternative) Personality problems and office politics…
«CSI5112» D. Amyot U. Ottawa 117 What Is Used? (Recent Survey, 400+ projects) S. Fricker, R. Grau, A. Zwingli (2014). Requirements Engineering: Best Practice. Requirements Engineering for Digital Health. Springer.Requirements Engineering: Best Practice Requirements Engineering Techniques
«CSI5112» D. Amyot U. Ottawa RE Conferences International Requirements Engineering Conference (RE; since 1993): Int. Working Conf. on Requirements Engineering Foundation for Software Quality (REFSQ; since 1995): Workshop on Requirements Engineering (WER; since 1998): Asia Pacific Requirements Engineering Symposium (APRES; since 2014): Dozens of satellite workshops and special tracks elsewhere. Requirements Engineering Techniques118
«CSI5112» D. Amyot U. Ottawa Dilbert and Requirements Engineering Requirements Engineering Techniques119