1. Barry Boehm, USC-CSSE CS 510 and 577a Fall 2007
Value-Based Systems and Software Engineering (VBSSE) and the Incremental Commitment Model (ICM)
Barry Boehm, USC-CSSE
CS 510 and 577a
Fall 2007

2. VBSSE and ICM: Outline Executing the VBSSE
Issues of Scale, Complexity, and Uncertainty
The Cone of Uncertainty - I
Risk-Driven Incremental Definition: ICM Stage I
Buying Information to Reduce Risk
Risk-driven incremental development: ICM Stage II
Achieving both rapid change and high assurance
Multiple views of the ICM
Viewpoints and examples
Evidence of successful use
Implications for system acquisition, career paths
09/05/2007
©USC-CSSE

3. Initial VBSSE Theory: 4+1 Process – With a great deal of concurrency and backtracking
09/05/2007
©USC-CSSE

4. The Cone of Uncertainty: I
09/05/2007
(c) USC-CSSE
©USC-CSSE 4

5. The Cone of Uncertainty: I
1-Pass VBSSE Needed
Familiar Domain
Proven infrastructure
Well-Jelled team
Multi-Pass VBSSE Needed
New Domain
Immature infrastructure
Non-Jelled team
09/05/2007
©USC-CSSE
(c) USC-CSSE 5

6. Agent-based discussion support win condition
Example VBSSE Step 5 Negotiation Uncertainties: Sierra Mountainbikes Case Study
Agent-based discussion support win condition
Uncertain multi-agent coordination feasibility
Best-of-breed COTS product interoperability
Architectural mismatch uncertainty
1-second response time feasibility
Uncertain COTS-product scalability
Uncertainties make premature commitment highly risky
Better to reduce option space before negotiating details
09/05/2007
©USC-CSSE

7. Example of Premature Commitment: Response Time 1980s Query-Response System Example
Arch. A:
Custom
many cache processors
$50M
Arch. B:
Modified
Client-Server
Original Spec
After Prototyping 1 2 3 4 5
Response Time (sec)
09/05/2007
©USC-CSSE

8. Incremental Commitment in Gambling
Total Commitment: Roulette
Put your chips on a number
Commit to best-of-breed COTS components
Wait and see if you win or lose
Incremental Commitment: Poker, Blackjack
Put some chips in
See your cards, some of others’ cards
Decide whether, how much to commit to proceed
09/05/2007
©USC-CSSE

9. Using Risk Analysis to Buy COTS Information - Defer Contingent decisions to next phase
Risk Exposure RE = Prob(Loss) *Size(Loss)
Risk Reduction Leverage for each option
RRL= (RE:before – RE:after)/Cost
Basically, expected return on investment
Composite provides good profit for reasonable investment
Option
RE:before
RE:after
Cost
RRL
Reference Check
$500K
$450K 5 10
Analysis Tool
$400K
Full Prototype
$200K 60
Composite
$250K 25
09/05/2007
©USC-CSSE

10. Conclusion: Shrinking the Cone of Uncertainty
Total commitment only safe in low-risk situations
If risk is higher, incremental commitment better
Buy information to reduce risk
Need to prioritize information-buying investment
RE, RRL assessments helpful
Composite strategy often most cost-effective
09/05/2007
©USC-CSSE

11. VBSSE and ICM: Outline Executing the VBSSE
Issues of Scale, Complexity, and Uncertainty
The Cone of Uncertainty - I
Risk-Driven Incremental Definition: ICM Stage I
Buying Information to Reduce Risk
Risk-driven incremental development: ICM Stage II
Achieving both rapid change and high assurance
Multiple views of the ICM
Viewpoints and examples
Evidence of successful use
Implications for system acquisition, career paths
09/05/2007
©USC-CSSE

12. There is Another Cone of Uncertainty
09/05/2007
(c) USC-CSSE
©USC-CSSE 12

13. Future Process Challenges-I
Multi-owner, multi-mission systems of systems
Integrated supply chain: strategic planning, marketing, merchandising, outsourcing, just-in-time manufacturing, logistics, finance, customer relations management
Over 50 separately evolving external systems
Need to satisfice among multiple stakeholders
Rapid pace of change
In competition, mission priorities, technology, Commercial Off-the-Shelf (COTS), environment
Need incremental development to avoid obsolescence
Need concurrent vs. sequential processes
Need both prescience and rapid adaptability
09/05/2007
©USC-CSSE

14. Rapid Change: Ripple Effects of Changes Breadth, Depth, and Length
Platform N •
Platform 1
Infra
EControl
Enterprise Control
Situation Assessment
Info Fusion
Sensor Data Management
Sensor Data Integration
Sensors
Sensor Components : …
Breadth
Length
Depth
DOTMLPF
Legend:
DOTMLPF Doctrine, Organization,
Training, Materiel, Leadership,
Personnel, Facilities
09/05/2007
©USC-CSSE

15. Average Change Processing Time: 2 Systems of Systems
Average workdays to process changes
09/05/2007
©USC-CSSE

16. Future Process Challenges-II
Emergence and human-intensiveness
Requirements not prespecifiable
Need for evolutionary growth
Need to manage uncertainty and risk
Reused components
NDI: COTS, GOTS, open source, legacy
Affordability: capabilities determine requirements
Always-on, never-fail systems
Need well-controlled, high-assurance processes
Need to synchronize and stabilize concurrency
09/05/2007
©USC-CSSE

17. Process Model Principles
Success-critical stakeholder satisficing
Incremental growth of system definition and stakeholder commitment
3,4. Concurrent, iterative system definition and development cycles
Cycles can be viewed as sequential concurrently-performed phases or spiral growth of system definition
5. Risk-based activity levels and anchor point commitment milestones
The critical success factor principles have been derived from studies of successful and failed projects and study group discussion of particularly important human-system design and development principles. The primary rationale for each principle is provided below.
A system’s success-critical stakeholders include users, acquirers, developers, maintainers, and potentially others. If stakeholders’ key value propositions are not satisfied by a proposed or delivered system, they will refuse to use it or find ways to undermine it.
Single-increment, big-bang system developments take too long to develop and generally produce obsolete, non-responsive systems. Or, they become swamped by change traffic in trying to fix premature commitments.
3,4. Similarly, sequential, non-iterative processes take too long. They also must make premature commitments to poorly-understood requirements, leading to expensive rework or unusable systems.
5. Risk management provides effective ways to prioritize activities and to determine how much of an activity is enough. It also avoids risks turning into expensive or serious problems. Anchor point milestones provide effective ways to synchronize and stabilize concurrent activities.
09/05/2007
©USC-CSSE

18. How Principles Address Challenges
Future Process Challenges
Process Principles
Stakeholder satisficing
Incremental/evolutionary definition and commitment
Iterative development
Concurrent system definition and development
Risk management
Complex, multi-owner systems of systems X
Emergent requirements
Rapid change
Reused components
High assurance of system qualities
09/05/2007
©USC-CSSE

19. Process Model Comparison with Respect to Process Principles
Process Models
Principles
Stakeholder Satisficing
Incremental Growth
Concurrency
Iteration
Risk Management
Sequential Waterfall, V
Assumed via initial requirements; no specifics
Sequential No
Once at the beginning
Iterative, Risk-Driven Waterfall, V
Risk-driven; missing specifics
Risky parts
Yes
Risk-Driven Evolutionary Development
Revisited for each iteration
Concurrent Engineering
Implicit; no specifics
Yes; missing specifics
Agile
Fix shortfalls in next phase
Iterations
Some
Spiral Process 2001
Driven by stakeholder commitment milestones
Risk-driven
Incremental Commitment
Stakeholder-driven; stronger human factors support
Risk-driven; more specifics
09/05/2007
©USC-CSSE

20. ICM Stage II: Increment View
09/05/2007
©USC-CSSE

21. ICM Stage II: Increment View
09/05/2007
©USC-CSSE

22. VBSSE and ICM: Outline Executing the VBSSE
Issues of scale, complexity, and uncertainty
The Cone of Uncertainty – I
Risk-driven incremental definition: ICM Stage I
Buying information to reduce risk
Risk-driven incremental development: ICM Stage II
Achieving both rapid change and high assurance
Multiple views of the ICM
Viewpoints and examples
Evidence of successful use
Implications for system acquisition, career paths
09/05/2007
©USC-CSSE

23. The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point Milestones
09/05/2007
©USC-CSSE

24. The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point Milestones
Concurrently engr. Incr.N (ops), N+1 (devel), N+2 (arch)
Concurrently engr. OpCon, rqts, arch, plans, prototypes
09/05/2007
©USC-CSSE

25. The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point Milestones
Synchronize, stabilize concurrency via FRs
Risk patterns determine life cycle process
09/05/2007
©USC-CSSE

26. Anchor Point Feasibility Rationales
Evidence provided by developer and validated by independent experts that:
If the system is built to the specified architecture, it will
Satisfy the requirements: capability, interfaces, level of service, and evolution
Support the operational concept
Be buildable within the budgets and schedules in the plan
Generate a viable return on investment
Generate satisfactory outcomes for all of the success-critical stakeholders
All major risks resolved or covered by risk management plans
Serves as basis for stakeholders’ commitment to proceed
09/05/2007
©USC-CSSE

27. Incremental Commitment In Systems and Life: Stage I (Definition) Anchor Point Milestones
Common System/Software stakeholder commitment points
Defined in concert with Government, industry organizations
Initially coordinated with Rational’s Unified Software Development Process
Exploration Commitment Review (ECR)
Stakeholders’ commitment to support initial system scoping
Like dating
Validation Commitment Review (VCR)
Stakeholders’ commitment to support system concept definition and investment analysis
Like going steady
Architecting Commitment Review (ACR)
Stakeholders’ commitment to support system architecting
Like getting engaged
Development Commitment Review (DCR)
Stakeholders’ commitment to support system development
Like getting married
09/05/2007
©USC-CSSE

28. Increment N Operational Commitment Review (OCR)
Incremental Commitment In Systems and Life: Stage II (Development and Operations) Anchor Points
Increment N Operational Commitment Review (OCR)
Stakeholders’ commitment to support operations
Like having children
Concurrent Increment N+1 Development Commitment Review (DCR)
Stakeholders’ commitment to support Increment N+1 development
Based on feasibility-validated Increment N+1 architecture, plans
Rebaselined during Increment N development
Accommodating changes in requirements, priorities, NDI
09/05/2007
©USC-CSSE

29. Case Study: Scalable remotely controlled operations
09/05/2007
©USC-CSSE

30. Total vs. Incremental Commitment – 4:1 RPV
Total Commitment
Winning bidder: $800M; PDR in 120 days; 4:1 capability in 40 months using agent technology
PDR: many outstanding risks, undefined interfaces
$800M, 40 months: “halfway” through integration and test
1:1 IOC after $3B, 80 months
Incremental Commitment
$25M, 6 mo. to VCR: may beat 1:2 with agent technology, but not 4:1
$75M, 8 mo. to ACR: agent technology may do 1:1; some risks
$225M, 10 mo. to DCR: validated architecture, high-risk elements
$675M, 18 mo. to IOC: viable 1:1 capability
1:1 IOC after $1B, 42 months
09/05/2007
©USC-CSSE

31. How Much Architecting is Enough?
10000
KSLOC
100 KSLOC
10 KSLOC
Sweet Spot
Sweet Spot Drivers:
Rapid Change: leftward
High Assurance: rightward
Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution
Added Schedule Devoted to Rework
(COCOMO II RESL factor)
Total % Added Schedule
09/05/2007
©USC-CSSE

32. Stakeholder Commitment Ranges vs. Phase
The main point of this chart is to indicate that the relative resource commitment levels that stakeholders must furnish tend to increase with each successive phase, and that the commitments may be significant even in the early phases.
The placement of the boxes is based on rough engineering judgment. They are intended to represent rough 25th and 75th percentile ranges with respect to total resource commitment (in the software area, small (<10 people) projects account for about 60% of the projects, but large (>50 people) projects account for about 60% of the resources committed [Boehm-Turner, 2004]).
Factors causing resource commitment levels to be higher or lower include project size, length of production run, system criticality, system understanding level, stakeholder compatibility, personnel capability, and amount of new modeling and infrastructure capabilities needed.
Drivers
System Size
Length of Production Run
System Criticality
System Understanding Level
Stakeholder Compatibility
Personnel Capability
Amount of New Modeling/Infrastructure
09/05/2007
©USC-CSSE

33. RUP/ICM Anchor Points Enable Concurrent EngineeringV A D C I O C C C C O C R R R D C R
09/05/2007
©USC-CSSE

34. ICM HSI Levels of Activity for Complex Systems
09/05/2007
©USC-CSSE

35. Incremental Commitment Model: Detailed
09/05/2007
©USC-CSSE

36. Different Risk Patterns Yield Different Processes
09/05/2007
©USC-CSSE

37. Common Risk-Driven Special Cases of the Incremental Commitment Model (ICM)
Example
Size, Complexity
Change Rate %
/Month
Criticality
NDI Support
Org, Personnel Capability
Key Stage I Activities : Incremental Definition
Key Stage II Activities: Incremental Development, Operations
Time per Build; per Increment
1. Use NDI
Small Accounting
Complete
Acquire NDI
Use NDI
2. Agile
E-services
Low
1 – 30
Low-Med
Good;
in place
Agile-ready
Med-high
Skip Valuation , Architecting phases
Scrum plus agile methods of choice
<= 1 day;
2-6 weeks
3. Scrum of Scrums
Business data processing
Med
1 – 10
Med-High
most in place
Combine Valuation, Architecting phases. Complete NDI preparation
Architecture-based Scrum of Scrums
2-4 weeks;
2-6 months
4. SW embedded HW component
Multisensor control device
0.3 – 1
Med-Very High
In place
Experienced; med-high
Concurrent HW/SW engineering. CDR-level ICM DCR
IOC Development, LRIP, FRP. Concurrent Version N+1 engineering
SW: 1-5 days;
Market-driven
5. Indivisible IOC
Complete vehicle platform
Med – High
High-Very High
Some in place
Determine minimum-IOC likely, conservative cost. Add deferrable SW features as risk reserve
Drop deferrable features to meet conservative cost. Strong award fee for features not dropped
SW: 2-6 weeks;
Platform: 6-18 months
6. NDI- Intensive
Supply Chain Management
0.3 – 3
Med- Very High
NDI-driven architecture
NDI-experienced;
Thorough NDI-suite life cycle cost-benefit analysis, selection, concurrent requirements/ architecture definition
Pro-active NDI evolution influencing, NDI upgrade synchronization
SW: 1-4 weeks;
System: 6-18 months
7. Hybrid agile / plan-driven system
C4ISR
Med – Very High
Mixed parts:
Mixed parts; Med-Very High
Mixed parts
Full ICM; encapsulated agile in high change, low-medium criticality parts (Often HMI, external interfaces)
Full ICM ,three-team incremental development, concurrent V&V, next-increment rebaselining
1-2 months;
9-18 months
8. Multi-owner system of systems
Net-centric military operations
Very High
Many NDIs; some in place
Related experience, med-high
Full ICM; extensive multi-owner team building, negotiation
Full ICM; large ongoing system/software engineering effort
2-4 months; months
9. Family of systems
Medical Device Product Line
1 – 3
Related experience, med – high
Full ICM; Full stakeholder participation in product line scoping. Strong business case
Full ICM. Extra resources for first system, version control, multi-stakeholder support
1-2 months; 9-18 months
C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review.
DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware.
IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software
09/05/2007
©USC-CSSE

38. Examples of Risk-Driven Special Cases 4
Examples of Risk-Driven Special Cases 4. Software-Embedded Hardware Component Example: Multisensor control device
Biggest risks: Device recall, lawsuits, production line rework, hardware-software integration
DCR carried to CDR level
Concurrent hardware-software design
Criticality makes Agile too risky
Continuous hardware-software integration
Initially with simulated hardware
Low risk of overrun
Low complexity, stable requirements and NDI
Little need for risk reserve
Likely single-supplier software makes daily-weekly builds feasible
09/05/2007
©USC-CSSE

39. Examples of Risk-Driven Special Cases 5
Examples of Risk-Driven Special Cases 5. Indivisible IOC Example: Complete vehicle platform
Biggest risk: Complexity, NDI uncertainties cause cost-schedule overrun
Similar strategies to case 4 for criticality (CDR, concurrent HW-SW design, continuous integration)
Add deferrable software features as risk reserve
Adopt conservative (90%) cost and schedule
Drop software features to meet cost and schedule
Strong award fee for features not dropped
Likely multiple-supplier software makes multi-weekly builds more feasible
09/05/2007
©USC-CSSE

40. Acquisition as Command & Control via Spiral OODA Loops
Observe new/updated objectives, constraints, alternatives
Usage monitoring
Competition, technology, marketplace ISR
Orient with respect to stakeholders priorities, feasibility, risks
Risk/Opportunity analysis
Business case/mission analysis
Prototypes, models, simulations
Operate as current system
Accept new system
Act on plans, specifications
Keep development stabilized
Change impact analysis, preparation for next cycle (mini-OODA loop)
Decide on next-cycle capabilities, architecture upgrades, plans
Stable specifications, COTS upgrades
Development, integration, V&V, risk management plans
Feasibility rationale
Life Cycle Architecture Milestone for Cycle
09/05/2007
©USC-CSSE

41. Agile Change Processing and Rebaselining
- No Single Rebaselining Process
Assess Changes,
Propose Handling
Stabilized
Increment-N
Development Team
Change
Proposers
Future Increment
Managers
Agile Future-
Increment
Rebaselining Team
Negotiate change
disposition
Formulate, analyze options
in context of other changes
Handle
Accepted
changes
Discuss, resolve deferrals to
future increments
Propose
Changes
Discuss, revise,
defer, or drop
Rebaseline
future-increment
LCA packages
Prepare for rebaselined
development
Defer some Increment-N capabilities
Recommend handling
in current increment
Accept changes
Handle in current rebaseline
Proposed changes
Recommend no action,
provide rationale
Recommend deferrals to future increments
09/05/2007
©USC-CSSE

42. Spiral View of Incremental Commitment Model1 2 3 4 5 6
STAKEHOLDER COMMITMENT REVIEW POINTS:
Opportunities to proceed, skip phases backtrack, or terminate
Exploration Commitment Review
Valuation Commitment Review
Architecture Commitment Review
Development Commitment Review
Operations1 and Development2
Commitment Review
Operations2 and Development3
Cumulative Level of Understanding, Cost, Time, Product, and Process Detail (Risk-Driven)
Concurrent Engineering of Products and Processes
EXPLORATION
VALUATION
ARCHITECTING
DEVELOPMENT
OPERATION1
OPERATION2
09/05/2007
©USC-CSSE

43. ICM Frequently Asked Questions
Having all that generality and then using the decision table to come back to a simple model seems like an overkill. If my risk patterns are stable, can’t I just use the special case indicated by the decision table?
Yes, you can – as long as your risk patterns stay stable. But as you encounter change, the ICM helps you adapt to it. And it helps you collaborate with other organizations that may be using different special cases.
All this emphasis on commitment seems too overconstraining. If the world is getting more and more fluid, why not just improvise along?
09/05/2007
©USC-CSSE

44. Shared Commitments are Needed to Build Trust
New partnerships are increasingly frequent
They start with relatively little built-up trust
Group performance is built on a bedrock of trust
Without trust, partners must specify and verify details
Trust is built on a bedrock of honored commitments
Once trust is built up, processes can become more fluid
But need to be monitored as situations change
09/05/2007
©USC-CSSE

45. Elements of Commitments adapted from [Humphrey, 1989]
Made willingly, subject to assumptions
Not made lightly
Agreement on what is to be done, by whom, and when
Openly and publicly stated
Committers make every effort to meet commitments
Unforeseen conflicts and invalidity of assumptions are addressed early, and new commitments negotiated
09/05/2007
©USC-CSSE

46. VBSSE and ICM: Outline Executing the VBSSE
Issues of scale, complexity, and uncertainty
The Cone of Uncertainty – I
Risk-driven incremental definition: ICM Stage I
Buying information to reduce risk
Risk-driven incremental development: ICM Stage II
Achieving both rapid change and high assurance
Multiple views of the ICM
Viewpoints and examples
Evidence of successful use
Implications for system acquisition, career paths
09/05/2007
©USC-CSSE

47. Annual CrossTalk Top-5 Projects
Many candidate projects submitted annually
Providing evidence of success, key practices
Evaluated by leading software experts
Top-5 published in CrossTalk
DoD systems/software journal
20 projects to date: – 2005
09/05/2007
©USC-CSSE

48. Top-5 Use of Key Process Principles
Year
Concurrent Engineering
Risk-Driven
Evolutionary Growth
2002 4 3
2003 5
2004
2005
Total (of 20) 16 14 15
09/05/2007
©USC-CSSE

49. Process Principles in CrossTalk 2002 Top-5 Software Projects
Spiral
Degree
Concurrent
Requirements/
Solution Development
Risk–Driven
Activities
Evolutionary
Increment
Delivery
STARS Air Traffic Control *
Yes
HCI, Safety
For multiple sites
Minuteman III Messaging
(HAC/RMPE)
Safety
Yes; block
upgrades
FA-18 Upgrades
Not described
Census Digital Imaging
(DCS2000) **
No; fixed delivery
date
FBCB2 Army Tactical C3I
09/05/2007
©USC-CSSE

50. Example ICM HCI Application: Symbiq Medical Infusion Pump Winner of 2006 HFES Best New Design Award
09/05/2007
©USC-CSSE

51. Symbiq IV Pump ICM Process - I
Exploration Phase
Stakeholder needs interviews, field observations
Initial user interface prototypes
Competitive analysis, system scoping
Commitment to proceed
Valuation Phase
Feature analysis and prioritization
Display vendor option prototyping and analysis
Top-level life cycle plan, business case analysis
Safety and business risk assessment
Commitment to proceed while addressing risks
09/05/2007
©USC-CSSE

52. Symbiq IV Pump ICM Process - II
Architecting Phase
Modularity of pumping channels
Safety feature and alarms prototyping and iteration
Programmable therapy types, touchscreen analysis
Failure modes and effects analyses (FMEAs)
Prototype usage in teaching hospital
Commitment to proceed into development
Development Phase
Extensive usability criteria and testing
Iterated FMEAs and safety analyses
Patient-simulator testing; adaptation to concerns
Commitment to production and business plans
09/05/2007
©USC-CSSE

53. Conclusions Current processes not well matched to future challenges
Emergent, rapidly changing requirements
High assurance of scalable performance and qualities
Incremental Commitment Model addresses challenges
Assurance via evidence-based milestone commitment reviews, stabilized incremental builds with concurrent V&V
Evidence shortfalls treated as risks
Adaptability via concurrent agile team handling change traffic and providing evidence-based rebaselining of next-increment specifications and plans
Use of critical success factor principles: stakeholder satisficing, incremental growth, concurrent engineering, iterative development, risk-based activities and milestones
Can be applied to other process models as well
Major implications for funding, contracting, career paths
09/05/2007
©USC-CSSE

54. Implications for Funding, Contracting, Career Paths
Incremental vs. total funding
Often with evidence-based competitive down-select
No one-size-fits all contracting
Separate instruments for build-to-spec, agile rebaselining, V&V teams
With funding and award fees for collaboration, risk management
Compatible regulations, specifications, and standards
Compatible acquisition corps education and training
Generally, schedule/cost/quality as independent variable
Prioritized feature set as dependent variable
Multiple career paths
For people good at build-to-spec, agile rebaselining, V&V
For people good at all three
Future program managers and chief engineers
09/05/2007
©USC-CSSE

55. ICM Perspectives ICM principles and process are not revolutionary
They repackage current good principles and practices to make it easier to:
Determine what kind of process fits your project
Keep your process on track and adaptive to change
And harder to:
Misinterpret in dangerous ways
Gloss over key practices
Neglect key stakeholders and disciplines
Avoid accountability for your commitments
They provide enablers for further progress
Guidance for implementing VBSSE process
Reorienting systems engineering principles and processes
Reorienting system acquisition and contracting practices
09/05/2007
©USC-CSSE

56. General References
Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”, Systems Engineering 9(1), pp. 1-19, 2006.
Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004.
Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems of Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006.
Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and Software Engineering,” CrossTalk, October 2007 (to appear).
Humphrey, W., Managing the Software Process, Addison Wesley, 1989.
Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development", USC CSSE Technical Report USC-CSSE , 2006.
Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software Engineering Institute, 2006.
Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New Look, National Academy Press, 2007.
09/05/2007
©USC-CSSE

57. List of Acronyms ACR Architecting Commitment Review B/L Baselined
CCD Core Capability Drive-Through
COTS Commercial Off-the-Shelf
DCR Development Commitment Review
DI Development Increment
DOTMLPF Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities
ECR Exploration Commitment Review
FMEA Failure Modes and Effects Analysis
GUI Graphical User Interface
HSI Human-System Interface
ICM Incremental Commitment Model
IOC Initial Operational Capability
IRR Inception Readiness Review
09/05/2007
©USC-CSSE

58. List of Acronyms (continued)
LCA Life Cycle Architecture
LCO Life Cycle Objectives
OC Operational Capability
OCR Operations Commitment Review
OO&D Observe, Orient and Decide
OODA Observe, Orient, Decide, Act
O&M Operations and Maintenance
PRR Product Release Review
RACRS Regional Area Crisis Response System
SoS System of Systems
SoSE System of Systems Engineering
TSE Traditional Systems Engineering
VCR Validation Commitment Review
V&V Verification and Validation
09/05/2007
©USC-CSSE