Presentation is loading. Please wait.

Presentation is loading. Please wait.

Value Based Software Engineering What, Why, How & The Mysterious Nupul Kukreja SoCal SPIN May 6 ‘11.

Similar presentations


Presentation on theme: "Value Based Software Engineering What, Why, How & The Mysterious Nupul Kukreja SoCal SPIN May 6 ‘11."— Presentation transcript:

1 Value Based Software Engineering What, Why, How & The Mysterious Nupul Kukreja SoCal SPIN May 6 ‘11

2 About Me Pursuing PhD in Computer Science under Dr. Barry Boehm. Focus Area: Value Based Software Engineering (VBSE) MS in CS from USC and BE in CS from Mumbai University, India Worked as a Software Developer at Capgemini Consulting India Pvt. Ltd. Lecturer for CS Undergraduates at Watumull Institute (Mumbai University) teaching Computer Graphics and Operating Systems 10/6/2010© USC-CSSE2

3 Agenda Section 1* : Overview – Understanding “Theory W” – VBSE’s 4+1 Theory – VBSE Agenda – VBSE: 7 Key Elements (Overview) Section 2: Applying 7 Key Elements in practice – Identifying company’s strategic goals and their overall value – Benefits Realization Analysis – Measurements/Estimation and Value of Information – Creating a Value Based Business Case – Elicitation/Reconciliation of Stakeholder Win Conditions – Prioritization of Win Conditions/Requirements using Kano Analysis – Tracking Value Earned 10/6/2010© USC-CSSE3 *Content courtesy: Dr. Barry Boehm

4 Tools & Techniques Involved Application of VBSE in practice involves understanding practices from various domains: – Decision Theory – Statistics – Management/People Engineering – Software Engineering Principles We shall see how to effectively combine them to help practice VBSE with minimal overhead 10/6/2010© USC-CSSE4

5 Theory W*: Enterprise Success Theorem – An informal proof Theorem: “Your enterprise will succeed if and only if it makes winners of your success-critical stakeholders” Proof of “if”: Everyone that counts is a winner. Nobody significant is left to complain. Proof of “only if”: Nobody wants to lose. Prospective losers will refuse to participate, or will counterattack. The usual result is lose-lose. 10/6/2010 *An Initial Theory of VBSE – Barry Boehm & Apurva Jain 5© USC-CSSE

6 Theory W: WinWin Achievement Theorem Making winners of your success-critical stakeholders (SCSs) requires: Identifying all of the SCSs Understanding how the SCSs want to win Having the SCSs negotiate a win-win set of product and process plans Controlling progress toward SCS win-win realization, including adaptation to change. 10/6/20106© USC-CSSE

7 Initial VBSE Theory: 4+1 Engine: Theory W (stakeholder win-win): What values are important? – Enterprise Success Theorem – Theory of Justice – Win-Win Equilibrium and Negotiation Four Supporting Theories – Dependency Theory: How do dependencies affect value realization? Results chains; value chains; cost/schedule/performance tradeoffs – Utility Theory: How important are the values? Multi-attribute utility; Maslow’s need hierarchy – Decision Theory: How do values determine decisions? Investment theory; game theory; statistical decision theory – Control Theory: How to monitor and control value realization? Feedback control; adaptive control; spiral risk control 10/6/20107

8 VBSE Theory 4+1 Structure 10/6/20108

9 Initial VBSE Theory: 4+1 Process – With a great deal of concurrency and backtracking 10/6/20109

10 VBSE Agenda Objective: Integrating value considerations into the full range of existing & emerging Software Engineering principles in a manner so that they ‘compatibly’ reinforce one another Major Elements: – VB* Requirements Engineering – VB Architecting – VB Design and Development – VB Verification And Validation – VB Planning and Control – VB Risk Management – VB Quality Management – VB People Management *VB = Value-Based 10/6/201010© USC-CSSE

11 VBSE: Seven Key Elements Benefits Realization Analysis Stakeholder Value Proposition Elicitation & Reconciliation Business Case Analysis Continuous Risk and Opportunity Management Concurrent System & Software Engineering Value-Based Monitoring & Control Change as Opportunity 10/6/201011© USC-CSSE

12 1. Benefits Realization Analysis 12© USC-CSSE DMR/BRA* Results Chain INITIATIVE Implement a new order entry system Assumption(s): -Order to delivery time is an important buying criterion Contribution Reduce time to process order OUTCOME Reduced order processing cycle (intermediate outcome) OUTCOME Increased sales Contribution Reduce time to deliver product *DMR Consulting Group’s Benefits Realization Approach Accountable Stakeholder(s)

13 Key ‘Benefits’ of Doing BRA/Results Chain Explicitly maps out the intended benefits of the system to be acquired (or any initiative to be undertaken) Identifies what all initiatives need to be taken to actually realize the benefits (A.K.A. Work Breakdown Structure (WBS) at times i.e., “within” the initiatives) Identifies the ‘key stakeholders’ accountable for the above initiatives NOTE: Must be refined as more is learnt about the initiatives or an unforeseen benefit is uncovered Done at multiple levels of granularity and stabilizes earlier into the SDLC than most artifacts Subtle but valuable: Lays foundation for the relevant metrics required to ‘track’ the benefits 10/6/201013© USC-CSSE

14 2. Stakeholder Value Proposition Elicitation & Reconciliation Myth: ALL SCSs have readily expressible and compatible value propositions that can be easily turned into a set of objects for each initiative and the overall ‘portfolio’ of initiatives The following figure is also known as a “Model Clash” 1410/6/2010© USC-CSSE

15 Techniques There are a myriad techniques that can be applied for stakeholder value proposition reconciliation: – Expectations Management: Just awareness of potential conflicts could help SCSs relax their less critical levels of desire – Visualization & Trade-off Analysis Techniques: Prototyping, scenarios, estimation models help SCSs to mutually understand most important & achievable aspects of the system – Prioritization: Helps identify which combination of capabilities best satisfies the stakeholders’ most critical needs within available resource constraints – Groupware: Some groupware tools have support for collaborative brainstorming, discussions, prioritization etc., 10/6/201015© USC-CSSE

16 3. Business Case Analysis (BCA) Another extremely valuable technique for stakeholder value proposition reconciliation Helps determine where is the best bang for the buck – i.e., capabilities providing the best ROI (We’ll see how to create a VB-Business Case later) Can directly influence capability prioritization TWO Major factors influencing BCA – Unquantifiable benefits (a.k.a. intangibles) – Uncertainties & risk 10/6/201016© USC-CSSE

17 Techniques ROI – most commonly used to justify a business case. Difficult to apply for intangibles Deriving variables/attributes from an objectives hierarchy – closer to value based but we can derive them straight from the benefits chain! Each of the variables need to be ‘estimated’ either using subjective judgment, past data, expert advice… …OR we could ‘calibrate’ the estimator to provide better estimates! We’ll see how to do this in Section 2 10/6/2010© USC-CSSE17

18 4. Continuous Risk & Opportunity Management Understanding & Addressing People’s Utility Functions – Risk Averse – Risk Neutral – Risk Seeking  Implication(s): Very people oriented process Continuous and Iterative Must have plans/processes in place to identify, manage and mitigate risks 10/6/201018© USC-CSSE Utility Benefit 0,0 1,1 It is possible to have negative utilities too  losses

19 10/6/201019© USC-CSSE Central Tenet of Risk Management Metric  Risk Exposure: RE = Probability (Loss) * Size (Loss) –“Loss” of stakeholders’ value – Ex.: profits, reputation, quality of life etc., Prioritizing Risks using Risk Exposure (may be misleading!) Risk Probability High Low Check Utility - Loss Estimate Little Risk Low Loss of Utility High Check Probability Estimate Major Risk

20 5. Concurrent System & Software Engineering Increasing pace of change in IT marketplace  traditional sequential approach to software development (i.e., Waterfall model) is increasingly risky to use! Preferable: Concurrently engineer the product’s operational concept, requirements, architecture, life cycle plans and key sections of code Concurrent engineering further preferable if: – Software system requirements emergent from usage and/or prototyping rather than prespecifiable – Costs, benefits & risks of COTS software and/or outsourcing decisions affects requirements, architectures, cost, schedules etc., 10/6/201020© USC-CSSE

21 Choose Relevant Process Models 21 Lifecycle Objective (LCO) Lifecycle Architecture (LCA) Initial Operational Capability (IOC) For at least one architecture, a system built to that architecture will: For a specific detailed architecture a system built to that architecture will: An implemented architecture, an operational system that has: Support core Operational concept Satisfy core requirements Be faithful to prototype(s) Buildable in budget/schedule in the plan Show a viable Business case Have key SCSs committed to support the ‘next phase’ Support elaborated operational concept Satisfy elaborated requirements Be faithful to prototype(s) Buildable in budget/schedules in the plan Show a viable Business case Have key SCSs committed to support the full lifecycle All Major risks resolved/covered by a risk management plan Realized the operational concept Implemented the initial operational requirements Prepared a system operation & support plan Identified the initial site(s) of deployment for initial transition Identified users, operators & maintainers to assume operational roles Anchor point milestones – of the chosen process model

22 Techniques There are a myriad software process models – choose the one that fits best to your organization Feasibility evidence MUST be provided at each milestone to support the claims as shown on the previous slide 10/6/2010© USC-CSSE22

23 6. Value-Based Monitoring & Control Use project’s business case for monitoring the actual business value of the capabilities to be delivered 10/6/201023© USC-CSSE Value being realized? Determine Corrective Actions Develop/update business case; time phased cost; benefit flows; plans Perform to plans Assumptions still valid? YES NO YES

24 Techniques Forms the bulk of the ‘detailed’ section on VBSE in Practice What metrics to use? Are those the right metrics? Are we gaining benefits? Are we doing the right things and doing them correctly? We shall answer these questions in Section 2. 10/6/2010© USC-CSSE24

25 7. Change as Opportunity Today’s world – a not so good Present Expending resources to adapt to change is often stated as “rework costs” Changes are treated as defects in change tracking systems The real world: Continuous ongoing changes in – Technology – Marketplace – Organizational – Stakeholders’ value propositions and priorities 10/6/201025© USC-CSSE Opportunities can become risks if nothing is done about it!

26 A Brave New World Organizations that can adapt to change more rapidly will succeed better in the their mission Developing change anticipatory modular design can be considered a good investment to enhance system value in the future Two techniques for enhancing adaptability to change: – Architecture based – Refactoring based Example of “Change as Opportunity” Internet and the Web and their effect on ecommerce 26

27 Section 2: Applying VBSE in Practice 10/6/2010© USC-CSSE27

28 Agenda 28 Identify the need Identify the motive/goal/objective of the need to be satisfied Check alignment with company/organization’s goals and objectives: Find/Derive the ‘value’ of the initiative Identify initiatives to realize value Identify validate/assumptions when articulating value Identify success critical stakeholders (SCSs) Elicit value propositions (win conditions) of SCSs Create a value-based business case with information from above and check for business alignment ‘Percolate’ the value to all phases of the SDLC (software development lifecycle) Track/adjust the business case over time to ascertain value gained/created and validate assumptions

29 Company Objectives Before we start – we MUST know on what basis does the organization decide to pursue any endeavor Profit/Revenue is not the only variable… …Competitive advantage, Brand projection, Enhanced customer loyalty etc., to name a few The current undertaking must align with the strategic goals of the company 10/6/2010© USC-CSSE29

30 Ranking of Strategic Goals Say you have two programs each satisfying (or satisficing) the organization’s goals in varying degrees, which program should you choose? 1.Layout the full ‘results chain’ and see what benefits can be gained – disadvantage? There is no inherent ranking 2.Use Multi-Attribute Utility Theory MAUT (could also use Analytical Hierarchy Process (AHP) but we’ll focus on the former) 10/6/2010© USC-CSSE30

31 WHY MAUT?? Mathematically sound Easier to capture the risk attitude in the form of utility functions Strategic goals don’t tend to have too many attributes and thus attribute space is well managed and easily understood Easier to lay coarse grained benchmarks for various attributes 10/6/2010© USC-CSSE31

32 MAUT Example Let’s capture a single utility function for Revenue The more revenue the better – increasing utility 10/6/2010© USC-CSSE32 Gamble Sure thing: Best: $10M_ _____ Worst: $0.5M 50% For which value of the sure thing are you indifferent between the sure thing and the gamble? _______ Best: $10M Worst: $0.5M

33 Graphing Utility 10/6/2010© USC-CSSE33 (0, $0.5M) (1, $10M) Utility Revenue Risk Averse Risk Seeking $7.5M $8.1M $8.6M$1.5M $3M $4.7M

34 Finding Utility Q: Given a value of the ‘attribute’ what is its utility? How do we find that from the graph? This second step involves understanding the ranking of the attributes: – “If you can change one attribute from its worst state to the best state which one would you choose? Ties are allowed.” – Repeat this question till all attributes are ranked For each of the ranked attributes repeat the Gamble vs Sure thing lottery to find the ‘P’ i.e., probability at which you’ll be indifferent! 10/6/2010© USC-CSSE34

35 Calculating the Attribute Tradeoff Scaling Constant (k i ) 10/6/2010© USC-CSSE35 GambleSure thing: Reference System: -All attributes at worst state -Chosen attribute at best state P 1-P Best System All attributes at best state Worst System All attributes at worst state 100% _____ 0% P = For what value of ‘P’ are you indifferent between the sure thing and the gamble? ________

36 Finding Utility (Finally!) 10/6/2010© USC-CSSE36

37 MAUT (Cont’d) Advantages: Simple question answering and choosing the outcomes of a lottery (sure thing vs. gamble) Can easily be ‘computerized’ Understanding a lottery comes naturally to most people and the risk attitude is hidden within the choices Difficult to ‘game’ the system with false attitudes Disadvantages: May be non-intuitive to start with Time consuming to get 5 point utility functions and attribute tradeoff scaling constants Sometimes it’s just difficult to get utility functions (Arrow’s impossibility theorem) 10/6/2010© USC-CSSE37

38 Next: Benefits Analysis Understanding the Value of ‘what you are doing’ 10/6/2010© USC-CSSE38

39 Before We Begin… The previous application of MAUT is optional if your organization already has prioritized it’s strategic goals and objectives (continually updated) Still may be valuable to ‘mathematically’ affirm the choices! There are a myriad of methods available to help you with this: Balanced Score Cards, AHP etc., They are more ‘management’ i.e., no mathematical basis 10/6/2010© USC-CSSE39

40 Approach Use a live example from anyone in the audience (could be any example even something you are currently working on) or use a case study example of Sierra Mountain Bikes (SMB) (Former preferable) Walk through the steps of applying VBSE – ONE such way of applying VBSE – Has research viewpoints – Open to discussion – Combines techniques from Statistical Decision Theory, Management and Software Engineering 10/6/2010© USC-CSSE40 *Taken from: An Initial Theory of VBSE – Barry Boehm & Apurva Jain

41 SMB – Problem Description Susan Swanson, CEO of SMB when developing a shared constructive vision of SMB’s primary opportunities and problems, determined a significant opportunity of growth, since they produced top quality bikes at a competitive price. Major problem area: old manual order processing system. Distributors, retailers and customers were dissatisfied with high rates of late/wrong deliveries, poor synchronization between order entry, confirmation & fulfillment; and disorganized responses to problem solutions. It was decided to outsource the development of the new system, since it wasn’t their area of expertise. 10/6/2010© USC-CSSE41

42 Faster, better order fulfillment system Increased sales, profitability, customer satisfaction Faster order entry steps, fewer errorsOn-time assembly Safety, fair ness of inputs Inter - operability inputs Lesser time, fewer errors/order fulfillment system 10/6/2010© USC-CSSE42 New Order- Entry System Less time, fewer errors in order processing Increased customer satisfaction, decreased operation costs Increased profit, growth New order entry processes, training and outreach Improved supplier coordination New Order- Fulfillment System New Order Fulfillment processes, outreach, training Distributors, retailers, customers DevelopersSuppliers Sales personnel, distributors Assumptions: - Increasing market size -Continuing consumer satisfaction with product -Relatively stable e-commerce infrastructure -Continued high staff performance SO WHAT? What all to do to realize the benefits/ outcomes?? What are the ‘contributions’ of the initiatives? Who are the accountable stakeholders to help realize the benefits? Under what assumptions are these benefits realizable? The initiative identified by SMB is to ‘implement a new order entry system’ to get rid of the manual order processing

43 Benefits Analysis The previous slide lays out the expected benefits of having a new order entry/fulfillment system The ‘initiative’ of having a new system is linked to the final outcome ‘increased profitability/growth’ (strategic goal?) through intermediate outcomes! Each initiative must be followed by an outcome or to realize the benefits of an outcome some initiative(s) need to be taken (may not be so for the final outcome(s)) Key question to ask at each stage: “So what?” to get the subsequent outcome/initative. Stakeholders who are/would be accountable for realizing/reporting the benefits should be identified 10/6/2010© USC-CSSE43

44 Key Questions for performing Benefits Analysis Q1: Why do you want to know the value of “ ”? A: You must know the why to know the purpose of performing the benefits analysis i.e., the value of knowing the value of the initiative It’ll help understand the alignment with the organization’s strategic goals (will also help with identifying the need) Ex: “Why do you want to know the value of having a new order entry processing system?” A: So that the company has a viable case to know/believe if and whether a new system would/could increase productivity, growth, profit and customer satisfaction and by how much 10/6/2010© USC-CSSE44

45 Key Questions (Cont’d) Q2: What IS the value of “ ”? A: Perform the prior benefits analysis to see the value!! Q2.1: How do you define “value”? A: You just did! Questions 1 and 2 help answer this question itself. The “value” in this case is alignment with company goals and ‘how much’ does the said initiative align with it!! Q3: All this is good but ‘what exactly is the value’? What do I track? There are tons of intangibles to complicate the problem further? 10/6/2010© USC-CSSE45

46 Epiphany: Measurement! WHAT to measure? We already have standard metrics that we track for every project Are those ‘standard metrics’ really tracking overall project value or just some numbers? What about intangibles? They are almost impossible to track at best. We have a 1-N rating system to help us with that They may be hard, but not impossible! Douglas Hubbard has even published a book “How to Measure Anything – Finding the Value of Intangibles in Business” …Let’s see how we can use some of that wisdom to alleviate our measurement problem. But first…how we can use some 10/6/2010© USC-CSSE46

47 WHAT to track/measure? This is where we’ll build a value-based business case Directly derivable from our prior benefits analysis! Yes, we’ll also track/measure intangibles – probably won’t be a 1-N scale but actual numbers Risks! We didn’t forget that. We’ll also see how to better identify ‘strategic risks’ and measure them too! 10/6/2010© USC-CSSE47

48 10/6/2010 Safety, fair ness of inputs Faster, better order fulfillment system Increased sales, profitability, customer satisfaction Faster order entry steps, fewer errorsOn-time assembly Inter - operability inputs Lesser time, fewer errors/order fulfillment system New Order- Entry System Less time, fewer errors in order processing Increased customer satisfaction, decreased operation costs Increased profit, growth New order entry processes, training and outreach Improved supplier coordination New Order- Fulfillment System New Order Fulfillment processes, outreach, training Distributors, retailers, customers Developers Suppliers Sales personnel, distributors Assumptions: - Increasing market size -Continuing consumer satisfaction with product -Relatively stable e-commerce infrastructure -Continued high staff performance © USC-CSSE4810/6/2010© USC-CSSE48 Something is missing… Something is still missing… Associated costs of having a new system (make/buy/customize…) What all measurable items can we find from this diagram?

49 But still…What exactly should be measured? The previous exercise should promote the discussion of what is concretely meant by the terms – i.e., how can it be quantified? But this is exactly the problem with measuring coarse grained criteria! And that’s where we start to think If you can observe a difference then it can probably be measured! In fact the same benefits chain diagram can be used to simulate the discussion!! Let’s see how… 10/6/2010© USC-CSSE49 REALLY HARD!

50 Example: Growth 10/6/2010© USC-CSSE50 Increase growth Increased sales Cost savings Increase profits Increase market share SO WHAT? This approach probably looks familiar – something from classical decision theory?

51 Objectives Hierarchy You could replace the previous analysis with that of ‘objectives’ hierarchy In fact you could replace the entire benefits chain analysis with an objectives hierarchy (You already know the top most goal “Why do you want to know the value of “_____”?) You will however lose the connections between outcomes, stakeholders accountable, initiatives to be taken etc., of the benefits chain approach Suggestion: You can use it at this stage when ‘drilling down’ the ambiguous metrics to more specific attributes! OR Transform the benefits chain into an objectives hierarchy if you are comfortable with that! 10/6/2010© USC-CSSE51

52 How to Measure/Estimate Once you’ve drilled down both, tangibles and intangibles to concretely observable (measurable) facts we can proceed with how best to estimate/measure them The prior analysis from benefits chain all the way to measurable attributes should provide enough dimensions on which to capture the value of the project But aren’t the estimates just guesstimates for the ‘value based business case’? Yes, to some extent. Let’s see how to solidify these estimates… 10/6/2010© USC-CSSE52

53 Are you 90% Confident? When was the last time you were 100% confident with your estimates? Were you confident 100% of the time ? 100% confident estimates 100% of the time would make you a priceless asset!! Can you at least be 90% confident 90% of the time? Wait a minute! How can you measure this confidence level?? 10/6/2010© USC-CSSE53 That’s where we take Douglas Hubbard’s help. Remember him?Douglas Hubbard

54 Getting a 90% Confidence Interval (CI) Remember CIs from your statistics class? A 90% CI is a range that has a 90% chance of containing the ‘correct answer’ Let’s try a simple test: (No need to Google ) Provide a range wide enough so that you believe there is a 90% chance that the value will be within this range (Units don’t matter. We are more interested in the range) 10/6/2010© USC-CSSE54 QuestionLower BoundUpper Bound What is the circumference of Mars?_____________

55 Would you Gamble? Let’s say we gave you the following proposition: You win $1000 if the actual answer is within your range You win $1000 by spinning the wheel below: What would you NOW CHOOSE? 10/6/2010© USC-CSSE55 Win $1000 $0 10% 90%

56 Choices & Inferences ChoiceInference Gamble You think the gamble has a higher chance of payoff! Implies your 90% CI is NOT really your 90% CI but lower – 50%, 60%, 80%. Basically an overconfident estimate Your initial estimate This means that you think there is more than a 90% chance your range contains the answer! An underconfident estimate Indifference You are indifferent between your initial estimate and the gamble  an equivalent 90% chance  90% CI 10/6/2010© USC-CSSE56

57 Sure Thing vs Gamble If you are still with me you’ll realize that the previous exercise is similar to the one we did for capturing stakeholder utility!one We just replace the ‘sure thing’ by your estimate and the gamble is a 90% Win and 10% loss! You ARE using utility theory concepts to make a 90% confident estimate that factors in uncertainty and probable prior knowledge of experts! 10/6/2010© USC-CSSE57

58 The Value Based Business Case The prior analyses of benefits, initiatives, stakeholder identification, metrics can provide a great first cut value based business case For quantities that need to be estimated you could apply the 90% CI calibration exercise to get ‘better’/more reliable estimates However, one question is not enough to help you calibrate. You may have to spend some time practicing calibrating yourself* You may need some sample information to help you get started… 10/6/2010© USC-CSSE58 *Please refer “How to Measure Anything…” by Douglas Hubbard for more details/exercises

59 Value of Information How much time/money should you spend (if at all) gathering information ‘about’ the metric before estimating? That depends on the expected value (EV) of information – Commonly applied in statistical decision theory but uncommonly so in software engineering! Defined as Expected Value of Perfect Information (EVPI): EVPI = EV (perfect info) – EV (no info) EVPI is the ‘theoretical maximum’ you should be willing to pay for procuring further information Since perfect information is not obtainable, EVSI (expected value of sample information) is usually employed. Thus the net expected value of information is gained by ‘subtracting’ cost of sampling/procuring information! This was elucidated way back in 1981 by Dr. Boehm when he published the Blue Book*: “Software Engineering Economics”! 10/6/2010© USC-CSSE59 *Chapter 20: Statistical Decision Theory: Value of Information

60 Next: Eliciting Stakeholders’ Win Conditions (a.k.a., Value Propositions) 10/6/2010© USC-CSSE60

61 Stakeholders’ Value Propositions We started our discussion with Theory-W… …we now build on how to ‘implement’ it in action Value Propositions are also known as Win Conditions Theory-W involves eliciting, understanding and reconciling SCS (Success Critical Stakeholders) win conditions with achievable solutionsachievable solutions 10/6/2010© USC-CSSE61

62 Win-Win Negotiation Process * Collect SCS win conditions (WC) – basically brainstorming what the ‘expect’ from the system Converge on WCs – since it’s captured in natural language effort is taken to converge on a concisely worded, non-redundant, unambiguous list of Win Conditions Define glossary of key terms – basically the domain glossary used within a project Prioritize WCs – Business Value vs. Ease of Realization Reveal Issues/Constraints – the prioritization helps facilitate discussion about issues that may arise with the stakeholder’s expectations Identify Issues and Options – the team posts issues (if any) for each of the identified win conditions and identifies the options to deal with the issues Negotiate Agreements – negotiate/agree to the above, using oral discussion and entering it into the ‘online tool’. We say WinWin Equilibrium is reached when all WCs and Options have been agreed to. 10/6/2010© USC-CSSE62 *Stakeholder Value Proposition Elicitation and Reconciliation – Grunbacher et. al.

63 How/Where to Capture Win Conditions I’m currently working on an online collaboration tool, similar to Facebook to help capture stakeholder’s win conditions, raise issues, supply options, prioritize win conditions and indicate the current statue of equilibrium Attached are the screenshots of the tool. The tool will be deployed on USC’s servers for the upcoming class of Software Engineering CS577a in Fall ’11 (The tool will also be available for download for use by anyone) 10/6/2010© USC-CSSE63

64 Winbook 64

65 Prioritization These are two primary approaches that I’ve narrowed down from a value based perspective 1.Incremental Funded Methodology (IFM) Advocated by the book “Software by Numbers” – Mark Denne, Jane Cleland-Huang. Deals with grouping features into minimum marketable features and performing financial analysis using net present values and using that to predict ROI 2.Kano Analysis Commonly applied in Quality Function Deployment for prioritizing voice of customer data. Also advocated by Agile Mentor Mike Cohn for use in agile software development. We’ll go with the latter since the former is mathematically involved. Both could be applied in tandem, but we’ll go with the latter 10/6/2010© USC-CSSE65

66 Kano Analysis 10/6/2010© USC-CSSE66 Image From: Linear: The more of the feature the better Must be: Absence implies severe dissatisfaction. Expected to be present Exciter: If absent customer wouldn’t care but would be delighted to have them So, how do we know which features fall in which category?

67 Kano Questionnaire * You ask the following question(s) to the prospective users: 10/6/2010© USC-CSSE67 Functional Form of Question If feature X were present how would you feel? I like it that way I expect it to be that wayX I am neutral I can live with it that way I dislike it that way Dysfunctional Form of Question If feature X were absent how would you feel? I like it that way I expect it to be that way I am neutral I can live with it that way I dislike it that wayX *Further details can be found in Agile Estimating and Planning – Mike Cohn.

68 Cross-referencing the Responses Customer Requirements Dysfunctional Question LikeExpectNeutralLive withDislike Functional Question LikeQEEEL ExpectRIIIM NeutralRIIIM Live withRIIIM DislikeRRRRQ 10/6/2010© USC-CSSE68 E ExciterL Linear M Must HaveQ Questionable I IndifferentR Reverse

69 Percolation of Value into the SDLC The prioritized win conditions give an idea of how the customers values the features on a scale of desirability The win conditions are the set of value propositions of the stakeholders – their expectation(s) from the system Win conditions can be ‘mapped’ to benefits – either in the form of a matrix or by visual inspection You can even ‘rank’ the benefits in order of importance using MAUT too! Decisions about various win conditions are then taken with respect to the benefits to be achieved Decisions at the project level must factor in the impact on strategic goals This ‘framework’ of expected benefits should always be visible to the development team! 10/6/2010© USC-CSSE69

70 Value Based Tracking I haven’t explored this area in it’s totality but what seems to hold a lot of promise is “Multivariate Statistical Analysis” especially Multivariate Regression Analysis We can pick out key ‘independent variables’ from the intermediate outcomes of the benefits chain and let the ‘dependent variable’ be that from the final intended goal/outcome We can run a regression analysis to know if we really gained the benefits. Statisticians have been doing it all along! This implies that if we DON’T have relevant data we may need to perform sampling to have enough data points to warrant this analysis. Is the sampling effort worth it? It depends on the value of information! 10/6/2010© USC-CSSE70

71 Wrapping Up! …finally 10/6/2010© USC-CSSE71

72 Conclusion Presented ONE way of applying VBSE in practice using ONE set of tools and techniques. There are myriads of possible combinations of tools/techniques that could be used. Use the one best suited to your organization… …The ideas in this presentation should help you get started We’ve covered a lot of ground but it’s only possible to present a limited amount. I’ve attached a set of references for further reading! Hope you gained some ‘value’ out of this presentation! 10/6/2010© USC-CSSE72

73 References Value Based Software Engineering – Stefan Biffl et. al. How to Measure Anything - Douglas Hubbard The Information Paradox – John Thorp Software Engineering Economics – Barry Boehm Making the Software Business Case – Donald Reifer Decision with multiple objectives – Keeney, Raiffa Agile Estimation and Planning – Mike Cohn Software by Numbers – Mark Denne, Jane Cleland- Huang Lean Software Development - Poppendieck Agile Software Requirements – Dean Leffingwell 10/6/2010© USC-CSSE73


Download ppt "Value Based Software Engineering What, Why, How & The Mysterious Nupul Kukreja SoCal SPIN May 6 ‘11."

Similar presentations


Ads by Google