1. Where have SE Standards (and Models) come from…and where are they going? INCOSE - WMA Donald J. Gantzer SAIC May 15 , 2007

2. Objectives
To give an overall summary of key SE process standards and models
To illustrate a top level comparison of them
To indicate trends and uses (in DoD)
To briefly address one key process – Technical Planning
Disclaimer: The views and information presented here are the author’s alone and do not necessarily represent SAIC or DoD views.

3. Agenda SE Standards and Models Overview
Evolution of Standards & Models
ISO
EIA - 632
IEEE
CMMI - Dev
Defense Acquisition Guide/SE
Correlations across
Application to Tech Planning
Summary
Some Key References

4. Oh…wither models & standards?! “The Process Standards / Models Quagmire”. Source: Systems and Software Consortium {SSCI}

5. SE Standards and Models Trends
CMMI-Acq (07)
SA-CMM
CMMI-AM
CMM for SW
SW-CMM(97)
CMMII v1.1(02)
CMMI-Dev v2.0(06)
EIA-731 [SECM](99)
SE-CMM
SECAM
MilStd 499 A/B
EIA/IS-632
GEIA-632 (99)
GEIA-632A (07)
ISO 15288(02)
08?
06+
IEEE (04)
IEEE-1220(05)
IEEE-1220
IEEE-1220(99)
CMM- Capability Maturity Mode
ISO : SE-System Life Cycle Processes
EIA - 632:Process for Engineering a System
IEEE-1220: Application & Management of the SE Process
INCOSE SE HB v3 (06)
INCOSE SE HB v2a (04) 95 00 05
Sources: INCOSE, SEI

6. Overview of SE Standards + CMMI
ISO/IEC – SE – System life cycle processes
“Establish a common framework for describing the life cycle of systems” – 25 processes; incl 7 project, 11 technical, and 7 agreement and enterprise; 62 pgs
- it is a hi-level framework [descriptive]
EIA – Processes for Engineering a System
“Provide an integrated set of fundamental processes to aid a developer in the engineering or re-engineering of a system”
- 33 Requirements in 13 processes; 120 pgs
- it is less prescriptive, broader scope [e.g., incl enterprise environment]
IEEE – Application and Management of the SE Process
“Defines the requirements for an enterprise’s total technical effort related to the development of products and processes that will provide life cycle support for the products. It prescribes an integrated technical approach to engineering a system and requires the application of management of the systems process throughout the product life cycle.” - 28 requirements 28; incl 14 general, 6 by LC stages, 8 in SE Process ; 87 pgs –
CMMI - DEV – Capability Maturity Model Integration®
- 27 process areas:11 Management, 6 Engineering, 5 Support, and 5 Process; 573 pgs

7. SE Standards & Models Life Cycle Phases
ISO 15288
EIA 632
IEEE 1220
CMMI (informal)
DoD/DAG
[DoDI ]
Concept
Pre-system Definition
concept, exploration, vision
Development
System Def.,
Subsystem design,
Detailed design
Detailed design;
FAIT
feasibility, design, development
Tech Development;
System Development.& Demonstration
Production
End product, physical,
IT&E
production, manufacturing,
delivery
Production & Deployment: LRIP
Utilization
operations
Operations & Support [O&S]: FRIP
Support
support, maintenance, sustainment
O&S:
Sustainment
Retirement
disposal,
phase out
O&S:Disposal

8. (source: S. Sheard, SPC and J. Lake, SMi; 2004)
Scope of SE Standards
IEEE 1220
EIA 632
Level of Detail
MIL-STD-499B
EIA/IS 632
ISO/IEC 15288
As is to be expected, SE standards have a lower level of detail and a larger scope than individual discipline standards, such as MIL-STD-498.
The first three (and so far only released) standards have the most detail and least breadth of scope of the SE standards, as they try to provide information on how to do systems engineering, and mostly confine themselves to engineering (plus some project management).
The newer SE standards, which do not even recognize the term ”SE”, take a broader approach, adding first the emphasis on Supplier-Customer agreement and then all the other enterprise processes that must be in place to build systems.
Breadth of Scope
(source: S. Sheard, SPC and J. Lake, SMi; 2004) 11

9. SE/SW Standards Harmonization Road Map
[source: SSCI]
EIA 632
ISO/IEC
JTC1/SC7
IEEE
EIA
Software
Engineering …
ISO/IEC 15288
IEEE 1220
ISO/IEC 12207
ISO/IEC 15504
An Harmonization road map
Systems
SC7 Internal Harmonization
Joint Standards Body Harmonization
Assessment
Adapted from: SC7 WG7 N0560 Systems Engineering Study Group Report 11JAN02

10. Simple Generic SE Process
source: Mil Std 499A/B and DAU guidance
Inputs:
System Analysis
& Control
Requirements
Analysis
Functional
Analysis/allocation
Synthesis
Outputs:
Note: Applied to AFIT/CSE SE Case Studies;

11. IEEE 1220: Systems Engineering Process – (2005)
Note: includes detailed flows for each activity; and an example SEMP toc
Clause 4 - General Requirements
Clause 6 – The SE Process
SE process
Polices & procedures for SE
Planning the technical effort: Prepare/update engineering plan; schedule; tech plans.
Development strategies
Modeling & prototyping
Integrated repository: data, tools.
Integrated data package: HW, SW, LC processes, human.
Specification tree
Drawing tree
System breakdown structure
Integration of the SE effort: concurrent engr., Int. teams.
Technical reviews
Quality management
Product and process improvement: re- engr., self-assess., LL.
1. Requirements Analysis
2. Requirements Validation
3. Functional Analysis
4. Functional Verification
5. Synthesis
6. Design Verification
*Requirements/ Functional /Design
trade studies & assessments
7. Systems Analysis*
8. Control
[source:SSCI + mods from IEEE 1220]

12. EIA 632: Processes for Engineering a System (1999; reaffirmed 2003)
Technical Management
Planning Process
Control Process
Assessment Process
Plans, Directives, & Status
Outcomes & Feedback
Supply Process
Acquisition Process
Acquisition and Supply
CONOPS & Requirements
System Design
Requirements Definition Process
Solution Definition Process*
System Products
Acquisition Request
Architectures/Designs
Product Realization
Implementation Process
Transition to Use Process
Note: provides detailed activities and outcomes for each process
Note: under revision to ‘harmonize’ w ISO
Products
Technical Support
Product Verification Process
Product Validation Process
Requirements Validation Process*
Systems Analysis Process*
(source: graph from INCOSE SE Handbook v2)*

13. Draft update to EIA-632 (out for internal review)
EIA - 632a *{source: GEIA report; R. Harwell, 11/05}; 4 domains; Systems Management, System Creation, System Evaluation, Enterprise Environment
Planning
Assessment
Progress Assessment
Control
Requirements Definition
Concept Definition
Solution Definition
System Definition
Product Realization [Implementation/Transition]
System Realization
Systems Analysis
Mission & Systems Analysis
Requirements & End products Val., System ver.
System V&V
Supply & Acquisition
Customer & Supplier Relationship Management
Enterprise Support
Resources & Infrastructure
Governance
Enterprise support
LC Portfolio Mngt.

14. ISO 15288: System Life-Cycle Processes (2002)
Source: INCOSE SE Handbook, v3.0
Each process has purpose, outcomes, and activities

15. CMMI Issues DoD Sponsor Report
Programs execute at lower maturity levels than their organizations have achieved and advertised
High-maturity practices are not consistently applied at the project level after contract award
How to ensure new projects will incorporate CMMI processes
Appraisal sampling procedures – how to ensure adequate coverage of the organizational unit
Appraiser quality – training, consistency
Lack of agreement on what constitutes Levels 4 and 5
Need to converge to a single representation
Content of appraisal disclosure statements is lacking
Inadequate training and education for acquirers
Should CMMI be used for source selection?
Source: NDIA SE Conf., 10/06 K. Baldwin

16. Summary Improvements in CMMI-Dev v1.2 [August, 2006]
New material added to CMMI v1.2 to ensure processes used at project start-up
Appraisal Disclosure Statement (ADS) improved
Sampling rules improved
High maturity appraiser certification instituted
Additionally…
CMMI guide for Acquisition Orgs to leverage Supplier’s efforts – 3/07
CMMI – Acq in final draft [summer 07?]
More improvements are necessary
Source: NDIA SE Conf., 10/06 K. Baldwin

17. Capability Maturity Model Integrated (CMMI- Dev)
CMMI Core SE
Related Examples
Integrated Product and
Process Development
Supplier
Sourcing SW
Organizational Goal
(OPD)
Project Goal (IPM)
IPPD SE
Related Examples
Hardware
Related Examples SW
Related Examples
CMMI Core (now includes SS)
* Sources incl EIA-731(SECM)
Source: SEI

18. CMMI-Dev 1.2 Process Areas— Continuous Representation
Category
Process Area
Project
Management
[added Acquisition Management]
Project Planning
Project Monitoring and Control
Supplier Agreement Management
[becomes Solicitation & Supplier Agreement Management ?]
Integrated Project Management)
Risk Management Quantitative Project Management (4)
(bold is for Levels 2&3 capability; […] addresses correlation with draft CMMI-Acq )
Configuration Management
Process and Product Quality Assurance
Measurement and Analysis Decision Analysis and Resolution
Causal Analysis and Resolution (5)
Support
[Acq] Requirements Management
[Acq] Requirements Development
[Acq] Technical Solution
Product Integration [Acq drops this?]
Verification
Validation
Engineering
Organizational Process Focus; Definition
Organizational Training;
Process Performance (4)
Organizational Innovation and Deployment(5)
Process
Management
(Hi maturity level)

19. Defense Acquisition Guide (DAG) [11/04] – Chapter 4 on SE
[Note: DAG/SE used ISO , EIA- 632, IEEE 1220, and DAU 2001 SE Handbook as key references]

20. Core Technical Processes:
Example Systems Engineering Process Architecture (SE Processes taken from DAG, Chapter 4)
Technical Management Processes:
Tech
Planning
Tech Decision Risk Requirements
Assessment Analysis Management Management
Core Technical Processes:
Requirements Development
Legend:
Management
Engineering
Support
Logical Analysis Design Solution
Feedback from each process
Implementation
Integration
Note: this illustration was adopted from some PI work done for DISA_by author
Transition
Technical Support Processes:
Verification & Validation Interface Management Configuration & Data Management

21. SE Standards/Models Mapping - Management
ISO-15288
EIA-632
IEEE-1220
CMMI-DEV
DAG/SE Chap. 4
Project Planning
Planning
Planning tech effort, SE integration, of SE effort
Project Planning; Integrated PM;
Quantitative PM
Technical Planning;
Interface Mngt.
Project Assessment
Assessment
Control
Project Monitoring & Control; M&A;
Technical Assessment;
Project Control
Project Monitoring & Control
Decision Making
Systems Analysis
Measurement & Analysis (M&A); Decision Analysis & Resolution; Quantitative PM; Causal Analysis & Resolution;
Decision Analysis
Risk Mngt.
Risk Mngt.
Configuration Management (CM) CM
CM;
Int. repository/ data pkg.
Requirements Mngt.
Information Mngt.
Control: info dissemination
Control; Integrated DB/pkg.
Integrated PM;
Technical Data Mngt.
Agreement : Acquisition & Supply
Acquisition & Supply
Supplier Agreement Mngt.; see also draft CMMI-Acq
See other DAG chapters
Enterprise: Environment, Life Cycle, Resource, & Quality Mngt.
Environment & Enterprise Support [e.g., investment, resource, process mngt.]
QM;
Product & Process Improvement
Organizational processes;
Process & Product QA

22. SE Standard/Models Mapping - Technical
ISO-15288
EIA-632
IEEE-1220
CMMI-Dev
DAG/ Chap. 4
Stakeholder Requirements Definition
Requirements Definition
Requirements Analysis
Requirements Development
Systems Analysis (SA)
Requirements & Functional Analysis; SA
Logical Analysis
Architectural Design
Solution Definition
Synthesis; SA;
modeling, spec tree, SBS
Technical Solution
Design Solution
Implementation
Implementation;
production
prototyping, fabrication, assembly, production
Integration
integration
Product Integration
Verification
System Verification
Functional and Design Verification; Tech reviews
Verification;
[+Chap 9 – IT&E]
Validation
Requirements and End Products Validation
Requirements Validation; Tech reviews, test
Validation;
[+Chap, 9 – IT&E]
Transition
Transition to Use
Product Integration;
Org. Innovation & Deployment; Org. Training
Operation; Maintenance; and Disposal
field support
support
operations, support, maintenance, sustainment, disposal
See other DAG chapters.

23. Example: Apply practices to Technical Planning ISO EIA IEEE CMMI - Dev PM BoK & DoD(AT&L)/SSE Tech Planning considerations

24. ISO 15288 Project Planning activities
“Purpose is to produce and communicate effective and workable project plans”
Identify the project objectives and constraints
Define the project scope as established in the agreement
Establish a WBS based on evolving system architecture
Define and maintain a project schedule based on project objectives and work estimates
Project achievement criteria for the life cycle stage decision gates, delivery dates and major dependencies on external inputs or outputs
Define the project costs and plan a budget
Establish the structure of authorities and responsibilities for project work
Define the infrastructure and services required by the project
Plan the acquisition of materials, goods and enabling system services supplied from outside the project
Generate and communicate a plan for technical mgmt. of the project, including the reviews
Define the project measures to be generated and the associated data to be collected, validated and analyzed
Generate a project quality plan
Source: Summarized from ISO 15288

25. EIA-632 – Technical Planning
Process Implementation Strategy
stakeholders, applicable docs, process approaches, LC phases, integration, reporting requirements, implementation
Technical Effort Definition
Requirement types, db, risk mngt. process metrics, metrics/quality, cost objectives, TPMs, tasks, methods & tools, technology
Schedule & Organization
Event& calendar based schedules, resources, staffing/disciplines, team/ org structure
Technical Plans
Engineering, Risk mngt., Tech Review, V &V, other
Work Directives
Work packages, work authorizations
Source: EIA-632

26. IEEE-1220 Planning the Technical Effort
“Prepare and Implement the technical plans and schedules to guide the project toward accomplishment of its objectives and proper conclusion.”
Engineering plan [example SEMP content]
Master and Detail Schedules
Technical plans
Developmental strategies
Modeling & prototyping
Integrated repository, Data, Tools, and Integrated data package
Hw, SW, Humans
Life cycle processes
Spec and Drawing trees; SBS
Integration the SE effort
Tech Reviews
Quality management
Product & process improvement
Source: IEEE

27. CMMI – Project Planning
Purpose is to supply and maintain plans that define project activities.
Establish Estimates
Estimate scope
Establish Estimates of work products/attributes
Define life cycle
Determine effort & cost estimates
Develop Project Plan
Establish budget & schedule
Identify risks
Plan for data management,
Plan for resources; Needed knowledge & skills
Plan stakeholder involvement
Establish the Plan
Obtain commitment to the Plan
Review plans that affect project
Reconcile work & resource levels
Obtain commitment
Other key process area relationships: – Requirements Development, Monitoring & Control, Supplier Agreement Mngt. Integrated PM, Risk Mngt., Measurement & Analysis, …
Source: CMMI-Dev

28. CMMI Generic Practices for Planning process area
Perform the planning process
Establish & maintain an Org policy for planning process
Plan the planning process
Provide resources
Assign responsibility
Train people
Manage configurations
Identify and involve relevant stakeholders
Monitor and control the planning process
Objectively evaluate adherence to the planning process
Review status with higher level management [PEO?] …

29. PM – Book of Knowledge (PM-BoK) (Project Management)
Scope*
Integration
(charter, scope statement, PMP)
Communication*
Risk*
Quality*
Human Resources*
Time (definition, sequencing, estimation)
Cost (estimation, budgeting)
Procurement (purchase, acquisition, contracting)
* Apply Planning, Execution & Control to each area
DoD PMBoK Extension (2003): SE
SW Acquisition
Logistics
T&E
Manufacturing
Source: ; 3rd Edition, 2004

30. Planning Process example [INCOSE SE Handbook v3]

31. DoD/SSE Technical Planning emphasis
Manage a Comprehensive Set of Requirements
Define project scope w key stakeholders [FoS, SoS]
Formulate, assess, select the preferred system concept
Develop explicit and testable system/project requirements
Develop a WBS [products & process]
Resource & Staffing to the Technical Plan
Organize and staff the project team [TA, PM, LSE, IPTs]
Estimate the time and resource requirements [IMS, EVMS]
Develop a project critical path
Develop a project budget
Develop and Managing Technical Baselines
Identify, manage, and mitigate project risks [technical]
Manage project changes and customer expectations
Managing Event-based Technical Reviews
Integrating Tech Planning into overall Program Planning & Management Context [IMP/IMS, EVMS, program Risks]
Note: DoD is updating DAG/SE, SEP Prep Guide, and DoDI
Source: Dinesh Verma; DoD/SSE Workshop

32. … and in summary
ISO is becoming a SE process ‘reference’ model
IEEE updated per ISO–15288; IEEE ‘adopted’ the w elaboration; further updates planned
EIA is being ‘harmonized’ to the ISO – & CMMI
CMMI used EIA 632, 731 as sources; next version will also look at and 1220
ISO (SW Engineering processes) is being harmonized with 15288; additionally a ”Life Cycle Management” standard being drafted both [in draft review
INCOSE SE Handbook v3.0, 2006 is based on 15288; SE Certification will be based on it; It is to be released as a ISO tech report
Coordination also underway w ISO 9001
DoD SE Revitalization:
DAG SE now being revised; one area being ‘beefed up’ is Software; DAU is implementing a series of SE courses
DoD SoSE Guide being piloted uses the DAG/Chapter 4
CMMI Guide for Acquirers; SE in Acquisition Contracts Guide
NDIA-SE draft Systems Assurance Guide uses ISO and EIA-632
IEEE working on an SE guidance & certification (by specialties)

33. SE Sources Links: ISO 15288: http://www.15288.com/
EIA 632:
IEEE 1220:
SEI-CMMI:
Systems & Software Consortium:
DAU-DAG:
DoD(AT&L) SE:
AF Center for SE;
INCOSE – Standards TC
References:
“SE Standards & Models Compared”; J. Lake (SMi) and S. Sheard (SPC), INCOSE 2004
“Evolution of a Standard EIA-632”; R. Harwell, INCOSE 2006
“Cross-Cultural Issues Associated with the Application of ISO/IEC Standard”; T. Ferris, INCOSE 2006
“INCOSE Insight: Special Feature: Standards in SE”, April 2007; see particularly K. Crowder, D. Kitterman, T. Doran, R. Harwell, and S. Arnold articles.

34. Backup slides

35. Some SE Process References
ISO/IEC 15288: System Engineering – System Life Cycle Processes; 2002
EIA/IS 632: Processes for Engineering a System; 1998 &?
IEEE 1220: Application and Management of the Systems Engineering Process; 2005
CMMI© – Capability Maturity Model Integration for Systems Engineering, Software Engineering, and Integrated Product and Process Development, 1.2 (2006)
Defense Acquisition Guide, Chapter 4 - Systems Engineering; Defense Acquisition University, 2004
CMMI – Acq: Adapting CMMI for Acquisition Organizations: A Preliminary Report; 2006
Understanding and Leveraging a Supplier’s CMMI Efforts; 2007
INCOSE Systems Engineering Handbook, v 3, 2006

36. NDIA Top 5 SE Issues 2003 2006 Lack of awareness of SE importance
Source: NDIA SE Conference 10/06; M. Schaffer DUSD(A&T) SSE
Lack of awareness of SE importance
Lack of adequate qualified resources
Insufficient SE tools and environments
Inconsistent requirements definition
Poor initial program formulation
2003
Inconsistent SE practices across all life cycle phases
Insufficient quantity and quality of SE expertise
Requirements not well managed or translated
Insufficient SE early in the life cycle
Inadequate tools and collaborative environments
2006

37. Taking SE to the Next Level
Source: Honorable James L. Finely, DUSD(A&T), NDIA SE, 10/06
My challenge to you:
Instilling upfront and continuous technical rigor
Understanding Systems-of-Systems SE
Fielding capability to the warfighter sooner
Increasing combat systems fuel efficiency
Addressing growing software concerns
Sincere thanks to our partners
Industry, in particular, NDIA
Service and Agency Engineering Staffs
“THE WILL TO CHANGE …”

38. DUSD(A&T) Vision for SE . . .
Source: Honorable James L. Finely, DUSD(A&T), NDIA SE, 10/06
Systems and Software Engineering
Center of Excellence
Systems Engineering Revitalization
World class leadership
Broaden Software Engineering and System Assurance, expanding to Systems- of- Systems
Responsive and agile, proactive to changing customer needs
Focused technical assistance, guidance, and workforce education and training
Competencies Impaired
Elevated Stature
Raised Awareness
Positive Influence
Delivered Product Suites
Customer focused…
Program Support is number one priority
Credentials…
Domain expertise
Human capital (advanced degrees, DAWIA Level III certifications, lean/six sigma green belts)
World Class…
Outreach to a broad communities base
Recognition and respect; can “compete” in a global marketplace (e.g., NATO, international symposia, advice sought by MODs, etc.)
Organizational stature…
Postured for success (forward-thinking, innovative, etc.)
Credibility and rank
. . . the Technical Foundation
that Enables Acquisition Excellence

39. Sound technical planning to support warfighting capabilities
Systems Engineering
MS A
MS B
MS C CD B1
Joint Concepts
JCD
EOA Rpt
TD & RR
CDD
CPD
SPG
FAA
FNA
EOA
SDD PD
O&S
FRP DR
OSD/ JCS
COCOM
FCB
Operational
Support Capability
Based Assessments
Demonstrate capabilities
meet user needs
OSD/JCS
COCOM
Sound technical planning to support warfighting capabilities
Define relationships with
related capabilities,
architectures (e.g., GIG)
Enterprise
Assess portfolio
performance (CAR)
Identify alternatives;
trade cost, sched, perf
SoS
Integrate SoS;
assess cost, sched, perf
Determine system
performance parameters
and verification plans
System
Assess system
performance against
capability needs
Identify incremental,
system specifications
Components
Integrate and test
SE is the technical foundation for building acquisition knowledge over time
Fully integrated SE approach: technical maturity, cost realism, risk mitigation
Develop, test, and assess
increments of capability
Source: Honorable James L. Finely, DUSD(A&T), NDIA SE, 10/06