1. Value Based Requirements Engineering And Prioritization
Percolating ‘Value’ into the SDLC (Software Development Lifecycle)
Barry Boehm and Nupul Kukreja
September 2, 2015

2. Insightful Questions
Given N requirements and budget/schedule constraints which ones to implement?
Which test cases should I run given time constraints?
What programming language should I choose?
Which design should I choose?
What framework should I choose?
What architecture should I choose?
Which project(s) should we choose?
Heck! How do I choose anything?

3. Agenda The ‘science’ of decision making – what, why and how?
The role of decision making in Value Based Requirements Engineering & Prioritization (VBRE/RP)
Importance of ‘Planning’
VBRP – what, why & how?
Understanding VBRP in CS577
VBRP in Action

4. Decision Making – What? Definition:*
Decision making can be regarded as the mental processes (cognitive process) resulting in the selection of a course of action among several alternative scenarios
Every decision making process produces a final choice
The output can be an action or an opinion of choice.
*Courtesy:

5. Decision Making – Making a Choice
Which apartment do you want to live in?

6. Making a Choice – How? So, just how did you choose your apartment?
What all did you ‘see’ / evaluate before signing the lease?
Examples of possible criteria:
Rent per month
Location i.e., quality of area
Proximity to campus
Presence of a good looking neighbor 
…and probably several others

7. Analyzing the ‘Problem’
You have the two key ingredients necessary for a decision problem:
A list of criteria
A set of alternatives
Problem: Given a list of criteria AND a set of alternatives select the alternative that best suits the given criteria
Decision Analysis
Criteria
Alternatives
Rank ordered set of alternatives
Apartment Selection
Rank ordered set of apartments

8. Decision Making – Why Bother?
Converts ‘art’ into science
Adds rigor to the act of decision making
Justification of choice of action i.e., why you chose what you chose?
Helps decrease legal liability
Ex.: Why did Company A select the bid/tender of Company X over Y? (It’s not only cost )
Documents the institutional memory that lead to that particular decision (i.e., helps trace back to provide justification of decision)

9. Decision Making – How do you do it?
If it is a well studied science (it is) does anything exist ‘out there’ to help practice this ‘science’?
There are various techniques that can be employed:
Multi-attribute utility theory (ISE 562 in Fall)
Analytical Hierarchy Process (AHP)
Simple Additive Weighting (SAW)
TOPSIS (Technique of Ordered Preference by Similarity to Ideal Solution)
…and many more (

10. Decision Problem – Look and Feel
When considering multiple criteria in decision making the problem is often referred to as “Multiple Criteria Decision Analysis” (MCDA) or “Multiple Criteria Decision Making” (MCDM)
Almost always has a matrix-like representation:
Criterion 1
Criterion 2
Criterion 3 …
Criterion N
Alternative 1
S11
S12
S13
S1N
Alternative 2
S21
S22
S23
S2N
Alternative 3
S31
S32
S33
S3N
Alternative N
SN1
SN2
SN3
SNN
Sij = Measure of how well Alternative ‘i‘ does on Criterion ‘j’

11. Okay, Great! But what has any of it got to do with VBSE?
Everything!! 
How do you decide:
Which requirement to implement first?
Which component to design first?
Which architectural style/pattern to use?
Which requirements to test first?
Which test-cases form part of the regression suite?
What all to prototype first?
For which component to write the “first line of code”?
Control Theory

12. Wait! But I’ve already done all of this before and I got it right too!
Probably! However…
Were you able to justify your choice i.e., how you came about choosing a specific option?
Was there ever a need to do so? 
Just “how” did you decide?
Perhaps intuition, gut feel, domain knowledge, tossing a coin, do as the Romans do…
Yes, a simple 1-10 works or even MoSCoW (Must, Should, Could, Would – have) but for relatively simple problems

13. Planning

14. Planning and Decision Making
What do we work on now?
What do we work on next?
How much money can/should we spend?
How much is being spent?
Are the risks being monitored/mitigated?
How does risk impact the plan?
Important to plan around ‘value’

15. Purpose of ‘Planning’*
Why do we do it?
Reduce risk & uncertainty
Supporting better decision making
Establishing trust (i.e. frequent delivery)
Conveying expectations
Planning is a ‘quest for value’
Attempt to find an optimal solution of the overall product development question: What should be built?
*Agile Estimation & Planning – Mike Cohn

16. A Good Plan One that stakeholders find sufficiently reliable
One that can be used as basis for decision making
Approximate time to market
An idea of the set of features
Made more precise as project moves on
Is a ‘living’ artifact showing the current status of the project to avoid last minute surprises
Is planned around ‘value’

17. Planning for ‘Value’* Stakeholders needs must be understood
Just delivering features is not important
Necessary to balance scope, schedule, cost and value of features comprising the release
Various factors impacting prioritization
Financial value of having the features
Cost of development
Amount/significance of new knowledge
Product (what)
Project (how)
Reducing Risk (Business/Technical)
Schedule
Cost
Functionality
*Agile Estimation & Planning – Mike Cohn

18. Decision Making and Planning for Value
Decision analysis techniques can be ‘overloaded’ to perform requirements prioritization…
…the rank-ordered output of decision making techniques could also rank-order requirements!
Requirements prioritized against the goals/needs/values of the stakeholders  VBRP
The prioritized requirements form the basis of the ‘plan’ – what is being delivered, for how much and by when?

19. The ‘Dimensions’ of Value
Value is a multi-dimensional quantity, a simple 1-10 may not cut it (too many ties – Must have, Must-must haves, must-must-must haves etc.)
Value lies in the eyes of the beholder
…and so does its dimensions! 
Dimensions are ‘hidden’ within the ‘expected benefits’ of the various stakeholders
The benefits serve as the goals/objectives/criteria on which to prioritize the requirements
Point to consider: Are all dimensions* equally important?
(*dimensions = benefits = criteria = goals = objectives)

20. Prioritizing Criteria/Value Dimensions
All criteria may NOT be equally important
But just HOW do you prioritize the criteria themselves?
A simple 1-10 (Yes, it works )
A highly involved process like multi-attribute utility theory (MAUT) to get the utility functions for each criteria which shows the risk attitude of each criterion
Or another interesting technique - Project Success Sliders*
* Radical Project Management by Rob Thomsett

21. Project Success Sliders
25%
Success Criteria (defined at start of project)
50%
75%
This slide shows the success criteria against which the success of the project/program will be measured and what are the expectations of the stakeholders i.e. what is it that they value the most. The project/management effort can
Image: Radical Project Management by Rob Thomsett

22. Project Success Sliders1 25
It’s ‘subjective’ but an extremely effective tool to understand the importance of expectations and their relative tradeoffs! 2 50 4
100 75 3
They can (and are) interpreted on a relative scale i.e., twice as important, half as much etc., 1 25 3 75 50 2
Image: Radical Project Management by Rob Thomsett

23. Putting It All Together
You have:
Relative ranking of all criteria
List of requirements (or win conditions)
What else do you need?
Scores! How well each requirement/win-condition does on each criteria
You STILL need something else…
The decision analysis ALGORITHM!

24. Decision Analysis Algorithm: Given a set of criteria with possible weights and the measure of how the alternatives rank on the respective criteria find an optimal alternative
Overall Score
Criteria
Alternatives
Criterion 1
Criterion 2
Criterion 3 …
Criterion N W1 W2 W3 WN
S(A1)
Alternative 1
S11
S12
S13
S1N
S(A2)
Alternative 2
S21
S22
S23
S2N
S(A3)
Alternative 3
S31
S32
S33
S3N
S(AN)
Alternative N
SN1
SN2
SN3
SNN
The scale of the scores could be absolute, relative, 1-10 (Likert Scale), Fibonacci... …must be consistent across criterion column

25. Algorithms for Practicing VBRP
There are various algorithms in literature to choose from along with those from MCDA/M:
Kano Analysis
Planning Poker
100-point assignment technique
Simple Additive Weighting
Quality Function Deployment (House of Quality)
Cost of Delay
Weigers’ Prioritization
Theory-W (Business value vs. Technical Feasibility)
TOPSIS
…and quite a few more 

26. What are we using in CS577?
TOPSIS: Technique of Ordered Preference by Similarity to Ideal Solution (MCDM Technique)
Integrated in Winbook for prioritizing MMFs and win conditions
We’ll be using Business Value, Technical Feasibility and Relative Penalty as the criteria against which to rank the requirements
MMFs Prioritized against stakeholders’ value propositions/goals/objectives
Success Sliders for prioritizing value propositions
The WinCondition prioritization template has an implementation of the TOPSIS algorithm. It takes in the criteria with their corresponding relative weights. You can add more dimensions for prioritization, but for CS577 only business value, technical feasibility (i.e., Theory-W based prioritization) are more than sufficient. We however augment it with Weiger’s technique to capture relative penalty if win-condition not delivered.
The business value and technical feasibility have already been estimated in the WinWin sessions.
You need not know the TOPSIS algorithm to start using it as can be seen from the linked excel template.
You can even perform sensitivity analysis by changing the values.
Ideally you will use various dimensions of value, directly derived from the benefits and prioritize against those criteria, but could be an overkill for the size of CS577 projects

27. Ideal Alternative (S’)
TOPSIS Primer
Criterion 1
Alternative 1
Ideal Alternative (S’)
Non-Ideal
Alternative (S*)
Alternative 2
Criterion 2
Aim: Rank order alternatives by their ‘closeness to ideal’ and ‘distance from non-ideal’
Criterion: Has ‘direction of preference’ i.e. more/less of the criterion is preferred
Ideal: Best score for each criterion Non-ideal: Worst score for each criterion

28. Which tablet should I buy? Winbook Project: Tablet Comparison
Decision Theory Demo
Which tablet should I buy? Winbook Project: Tablet Comparison
The rationale for this demo/case study is to show decision theory in action and it’s use for what-if analysis (i.e., sensitivity analysis).

29. Refinements to TOPSIS/VBRP
Prerequisite Handling (not yet integrated in Winbook)
Specifying MMF/WC prerequisites and factoring in prioritization
Simple solution: Priority of item less than all those in its prerequisite graph
Hierarchical Prioritization (partially integrated)
Prioritizing high level MMFs w.r.t. project goals
Prioritizing WCs w.r.t. MMFs
Prioritizing test cases w.r.t. WCs and so on
We were involved in an industrial case study where these techniques were deemed to be extremely important for VBRP to work in practice. You can find more details about it in: “Selecting An Appropriate Framework For Value-Based Requirements Prioritization: A Case Study” N. Kukreja et al. IEEE Requirements Conf. Sept 2012.

30. Points To Note TOPSIS is ONE way of practicing VBRP
You could even perform COTS tradeoff analyses using it. (It’s a decision problem!)
Use the VBRP technique that is best suited to the situation
Some prioritization techniques do not lend themselves to a typical spreadsheet like analysis
The output of some techniques could be bucketed (categorized into the MoSCoW buckets) or ordinal (explicitly rank ordered)

31. Conclusion
Tools and techniques for prioritization are only one side of the coin…
…negotiations and discussions are the other. The latter must be held for the former to be of any value. Using the tool will NOT guarantee VBRP
There “maybe” a Homework on VBRP…keep a watch out for the announcement 