1. 23 March 2005
Boeing “Value Front” Tool (VFT) Multi-Attribute Trade Study Tool A Phantom Works System Assessment team effort
Cost & Affordability
Decision Analysis
Effectiveness Analysis
Mark Schankman Phantom Works Affordability Engineering

2. Agenda Goals of this Presentation Definitions Purpose of Tool
Background to Value Front Tool
Demonstration Trade Study
Methods and Tool Process
Summary and Questions

3. Goals of this Workshop Familiarize you: Trade Study Methods and Tool
Inform you:
Use of Multi-Attribute Decision Methods for Design Trade Studies
Solicit Your Ideas: Potential Applications of Trade Study Tool
Have FUN !!

4. Illustration: Design a Hot Beverage Container
Definition of Terms
VFT: Value Front Tool
LCC: Life Cycle Cost
Uncertainty: Degree to which the cost, technical performance, or schedule is unknown
(measured by probability distributions)
CAIV: Cost as an Independent Variable
(also known as: “Target Costing”)
Illustration: Design a Hot Beverage Container

5. Utility (Customer Satisfaction) or Performance
Value Front Tool Identifies “Best Value” Alternatives Within the Trade Space
Cost Objective
(Target)
Cost Threshold
(not to exceed)
Best Value Concepts Lie on the “Value Front Line”
Performance
Objective (desired)
Utility (Customer Satisfaction) or Performance
Concepts that provide less value per $
Performance
Threshold (minimum)
NOTES:
A key aspect of CAIV is the exploration of the trade space with respect to cost prior to determining requirements. What these cost/performance trades provide is the knowledge base with which the customer can base a decision on what “bang for their buck” they want, and how much risk they’re willing to take for various solutions.
What the the customer is really looking for is not just cost estimates, but trend charts that show the cost of requirements as they change.
Trade Space
“Value Front” = Concepts that Maximize Customer Satisfaction with the Least Increase in Cost
Cost

6. CAIV Analysis Includes Measure of Performance & Cost Uncertainty
Utility =Customer Desirability
Life Cycle Cost
Value Front = Best Value Concepts
Technical Uncertainty
Cost & Schedule
Uncertainty
Trade Space

7. Methods Needed to Evaluate “Best Value”
Customer Need Priorities
(Decision Analysis)
Value Front
Tool
Customer
Desirability
Attribute Value
Attribute Value Distribution
Uncertainty
Simulation
O/S
Dev
Prod
Cost-Risk

8. Value Front Method and Tool Capabilities that Meet Systems Engineering Needs
Link performance, cost, effectiveness and “risk” and measures their ability to meet customer needs
Identify “Best Value” solutions in the design space = Systems Engineering Process for Trade Studies
Include uncertainty in trade study evaluations
(Boeing PRO 4819: ..“balance cost, performance, & risk)
Features of Value Front Tool:
Uses QFD analyses to transform customer needs into technical attributes
Combines four Decision Analysis and Affordability methods:
QFD (results only), Utility, CAIV, and risk
Quantifies probability that concept has greater utility and lower cost
Eliminates purchase of commercial tools (i.e., Dynamic Insight).
Graphically identifies concepts with “best-value”
Quantifies performance and cost uncertainty
Uses Monte Carlo and probabilistic methods
Graphical User Interfaces for inputs
Radar chart outputs compare contribution of each attributes
Uses the results of QFD analyses (external to the model) to transform customer needs into technical performance attributes (for up to 30 attributes) that are evaluated in terms of their “utility” and cost. The tool allows users to define Utility curves for each performance or effectiveness attribute.
Quantifies the probability that a given concept provides statistically greater utility (customer desirability) and lower cost than other concepts evaluated.
Eliminates the need to purchase a $1K desktop license for performing utility analyses with a commercially available tool (i.e., Dynamic Insight).
Graphically identifies any number of alternative concepts that provide the most performance “utility” (customer desirability) per dollar life cycle cost within the trade space.
Quantifies the uncertainty band for both technical risk and cost for each concept.
Excel based tool uses both Monte Carlo and probabilistic methods (i.e., “expected” value). Easy to use Graphical User Interfaces are provided for inputs.
Outputs include radar charts comparing the relative contribution of attributes to customer utility and cost.
Boeing success as Systems Integrator depends on tools that Deploy the Systems Engineering Process

9. Systems Engineering Trade Study Process Key Steps Addressed by Value Front Tool
Define the
purpose
Evaluate “down selected” alternatives
– Performance
– Weight
– Reliability
– Effectiveness
– Safety
– Etc
Establish
evaluation
criteria & weights
that meet customer needs
Steps in Value Front Tool
Identify
alternative
solutions
Analyze results Select “Best Value” alternative
Document results
Screen
alternatives
Accept/reject results
Value Front Tool Provides Discipline to the
Systems Engineering Trade Study Process
Trades -7
7/28/00

10. Value Front Tool Analysis : Which Drip Coffee Maker to Buy ?
Example
Value Front Tool Analysis :
Which Drip Coffee Maker to Buy ?

11. Typical Consumer Reports® Value Comparison
Chart from “Consumer Reports” Dec 2004, “Coffee Makers”
Performance Criteria
Average Cost
Value Comparison
Performance Criteria scored
Total utility score
We all evaluate performance criteria (if just in our heads)

12. Value Front Tool Inputs Required
Performance Criteria relative importance
(from QFD or other Decision Analysis methods)
Performance Criteria values
(e.g., Warranty period in years)
Utility Curves (use threshold and objective values)
(must represent customer’s expectations)
Life Cycle Cost
(Purchase Price + Operating cost ) **
** Assume Development Cost included in Purchase Price

13. Sample Evaluation Criteria Scores Braun KF400 Source: Consumer Reports© December 2004
Attributes evaluated for five coffee makers

14. Choose Your Criteria Scores & Weights “Mr. Coffee©” Coffee MakerX X X X X X

15. Boeing Value Front Tool Main Screen

16. “You Get What You Pay For”
Evaluation Criteria Scores from VFT Five Coffee Maker Comparison Attribute Scores based on Consumer Reports®
Good
Poor
Price: $ $ $60 $ $15
“You Get What You Pay For”

17. Value Front Tool Results Utility Value vs. Cost (CAIV)
Compresso
Braun KF180
Braun KF400
Black & Decker
Proctor Silex

18. Method to Visualize Utility and Best Value Uncertainty
Performance
Attribute (eg, range, speed)
Utility - Extent to which customer is satisfied
Reliability
Producibility
Risk - measured as the probability that attribute is less than expected
(in Value Front Method)
Safety
Mission Effectiveness
(eg, survivability)

19. Value Front Tool Utility Plot for Compresso Coffee Maker
Utility - Extent to which customer is satisfied
Ease of Use
Brewing
Warranty
Programmable
Cool to Touch
Cleaning
Utility Score = 1.0

20. Utility Radar Plots All Coffee Makers Evaluated
Braun KF180
Braun KF400
Compresso
Proctor Silex
Black & Decker

21. Methodology Used in
Value Front Tool

22. Trade Study Methods are Integrated with
Value Front Tool
Requirements Analysis
Facilitated by Decision Analysis
Technical Attributes
Customer Needs
Attribute Importance Score 1
Translate Customer Needs
to System Technical Attributes
Utility Analysis
QFD (house of quality)
Uncertainty simulation
CAIV analysis
Cost-risk estimating
Methods
Used
Utility Analysis uses Inputs
from IPTs
Attribute Value
Score Attribute Utility
for each Concept
100
0.2
0.4
0.6
0.8 1 80 60 40 20
Threshold
Goal
Desirability
Risk 2
Utility =Desirability
Plot Utility vs Cost
Identify Value Front
Utility =Desirability
Cost
Value Front 3
CAIV Analysis identifies “best value”

23. Front End “House of Quality” Inputs
Translate Customer Needs
to System Technical Attributes
Plot Utility vs Cost
Identify Value Front
Utility =Desirability
Cost
Value Front 3
CAIV Analysis identifies “best value”
Utility Analysis
QFD (house of quality)
Uncertainty simulation
CAIV analysis
Cost-risk estimating
Utility Analysis uses Inputs
from IPTs
Attribute Value
Score Attribute Utility
for each Concept
100
0.2
0.4
0.6
0.8 1 80 60 40 20
Threshold
Goal
Desirability
Risk 2
Requirements Analysis
Facilitated by Decision Analysis
Technical Attributes
Customer Needs
Attribute Importance Score
Translate Customer Needs
to System Technical Attributes
Operations Analysis Methods Used 1
Technical Attributes
Customer
Needs
Attribute Importance Score

24. Score Attribute Utility
Value Front Tool
Utility Analysis
Score Attribute Utility
for each Concept
Requirements Analysis
Facilitated by Decision Analysis
Technical Attributes
Customer Needs
Attribute Importance Score 1
Translate Customer Needs
to System Technical Attributes
Operations Analysis Methods Used 2
Threshold
Goal
Plot Utility vs Cost
Identify Value Front
Utility =Desirability
Cost
Value Front 3
CAIV Analysis identifies “best value” 1
0.8
0.6
Utility =Desirability
0.4
0.2 20 20 40 40 60 60 80 80
100
100
Attribute Value
Utility Analysis uses Inputs
from IPTs

25. Transform Attribute Value to Customer Desirability (“Utility”)
0.2
0.4
0.6
0.8
1.0 20 40 60 80
100
Desirability (Utility Score)
Threshold
Objective
Risk
Expected
Value
Distribution Determined
By Worst / Best value expected
Attribute Value

26. Value Front Tool Boeing Potential Program Users
Conceptual Trade Studies:
JDAM Container trade (completed)
LASER Test bed aircraft trade – Phase I used commercial Utility Tool
Advanced Rotorcraft – Considered by Boeing-Philadelphia
Engineering Disciplines Affected:
Systems Engineering
Operations Analysis:
Cost & Affordability
Decision Analysis (Utility Analyses)
Effectiveness

27. Status of Value Front Tool
Successfully applied tool for JDAM Container trade study
User Instructions planned later in year
Beta Test version with “self-contained” Tool Planned
Sensitivity analysis capability
Plotting enhancements
Help screens
Current Excel “Prototype” tool will be converted to
“Visual Basic” Tool for “interactive” use with customers

28. Boeing IDS Example Analysis : Weapon Container Trade Study

29. Value Front Uncertainty Region (Bounded by 10% and 90% Uncertainty)
Weapon Container Affordability Analysis Utility (Value to Customer) vs. Cost Uncertainty Region Define Boundaries of Utility Score & Cost
Value Front Uncertainty Region
(Bounded by 10% and 90% Uncertainty) 1
0.9 3
0.8 4 1 2
0.7 5
0.6
Greater cost uncertainty: bad choice ?
Concept
Weighted Average
Utility Score
0.5
ECS #1 1 1 2
0.4
ECS #3 3
Hardigg
0.3 4
PRC #3
0.2 5
PRC #4
0.1
0.5 1
1.5 2
2.5 3
3.5 4
4.5 5
Cost - Millions of Dollars

30. Value Front Tool Summary
Tool provides standardized, disciplined, methods to evaluate cost, performance, and risk of design & technology solutions
Identifies “best value” solutions in the design space as an essential part of the trade study process
Tool provides contractor a competitive advantage by quantifying a concept’s “value” to the customer.
The metrics necessary to make design and technology investment decisions are all evaluated (i.e., cost, performance, effectiveness, uncertainty).
“Value Front” Methods & Tool Help Deploy
Affordability Best Practices supporting the Systems Engineering Process

31. References Cost & Affordability: Matt Anderson (314-232-0931)
Decision Analysis: David Hamilton
( )
Decision Analysis: Mike Wheeler
( )
Target Costing Society:
Affordability Resource Database:
Affordability Best Practice (Boeing): PRO-4819

32. Backup Charts

33. QFD Matrices Description
Operational Attributes
The Value Front Tool Utilizes the Normalized Priorities Derived In a QFD Effort
Top
Level
Objectives
Objective
Importance
Ratings
Operational
Attribute Priorities
System Attributes
Range
Loiter Time
Sortie Generation
Relative
Priorities
Operational Attributes
Be Able to Fly Long Distances
QFD uses matrices to assist in the prioritization of attributes. As depicted above, a QFD effort may involve multiple matrices which are linked together. The multiple matrices represent the various levels of detail that must first be understood in order to prioritize system attributes. In the above example, before the system attributes could be prioritized, it was necessary to understand the operational attributes that the system attributes were supporting. Additionally, in order to understand the operational attributes, it was necessary to first understand the top level objectives.
At the end of the QFD process, the final system attribute priorities are exported to the Value Front tool. In the QFD effort described above, the first matrix consisted of top level objectives and operational attributes that satisfy those top level objectives. The top level objectives were first prioritized. Then, subject matter experts rated the contribution of the operational attributes to meeting the top level objectives. The relative priorities of the operational attributes were the summed products of the contribution ratings and the relative priority of the top level objectives.
The operational attributes and their priorities are transferred to the second matrix. Then, in the matrix, system attributes are rated based on their relative contribution to the operational attributes. The relative priorities of the system attributes are the summed products of the contribution ratings and the relative priority of the operational attributes.
QFD is more than just the math in the matrices. In fact, the mathematics is the easiest part of the process. The generation of the lists of objectives, operational attributes, system attributes, and underlying assumptions as well as the correct assignment of contribution ratings is much more difficult. For more information on QFD, please contact the Decision Analysis group. 9 9
Be Re-deployable 3 7
Be Able to Easily Detect Moving Targets 1 9 1 5
(157)
Priority 86 45 26
Normalized Priority
.55
.29
.16
(3X7) + (1X5)

34. Value Front Tool – Uncertainty Input Templates Developed at Boeing-Huntington Beach to Select Technical and Cost Uncertainty Ranges for Cost-Risk Analysis
Technical Risk
Cost Risk

35. Utility =Desirability
“Best Value” (CAIV) Analysis is Facilitated with Systems Assessment Integrated Tools
Value Front
Trade Study
Option
Cost
Threshold
Cost & Schedule Uncertainty
Technical Uncertainty
Utility =Desirability
“Value Front”
Trade Study Tool
Customer Satisfaction Metric = “utility” (0 – 1.0)
Combines “utility” metric for each technical attribute
Measures “utility” uncertainty based on technical risk
Analogy Cost- Risk Tool
Estimates concept cost compared to baseline (known cost)
Measures cost & schedule uncertainty
Life Cycle Cost

36. Attributes should be measurable (but can be qualitative)
Example Attribute Criteria (source: Boeing Trade Study Workshop - Al Bruns)
Product capability
Operational utility
Performance
Effectiveness
Survivability
Producibility
Task time
Cycle time
Mistake proofing
Supportability
Maintainability
Design for upgrade
Logistics
Training
Reliability
Environmental
Safety
Political
Ease of use
Attributes should be measurable
(but can be qualitative)

37. Categories of Trade Studies Performed with Value Front Tool
Preferred Design concept:
System, sub-system, or component trade studies
Preferred Technology concept
Evaluate technology investment vs. risk (uncertainty)
Evaluate new/emerging technologies for “best value” that meets Boeing strategic needs
Evaluate manufacturing options
Screen tool to “weed out” concepts with less value to customer (performance vs. cost)
Decisions can be Design or Technology Selection