1. N65236-ENGOPS-BRIEF-0028-1.2Approved for release to the public - xxx 2007 Systems Engineering Revitalization using CMMI ® Michael T. Kutch, Jr. Director Engineering Operations, Code 09K Department Head, Code 70 Intelligence & Information Warfare Systems SPAWAR Systems Center Charleston (SSC-C) SC SPIN Meeting – Jan. 25, 2007

2. Approved for release to the public - xxx 2007 2 Presentation Outline  Introduction to SPAWAR Charleston  Need for SE Revitalization  Revitalization Effort using CMMI ®  Training  Summary

3. Approved for release to the public - xxx 2007 3 SPAWAR Space and Naval Warfare Systems Command Where We Fit NAVAIR Patuxent River, MD NAVSEA Washington, DC NAVSUP Washington, DC Secretary of Defense President non-DoD CNO Fleet Support ASN (RDA) Acquisition Secretary of the Navy NAVFAC Washington, DC SPAWAR San Diego, CA SYSCEN San Diego, CA SYSCEN Norfolk, VA SFA Chantilly, VA SYSCEN New Orleans, LA NETWARCOM NAVSEANAVAIR MARCOR ADDU for C4I Other DoD SYSCEN Charleston, SC

4. Approved for release to the public - xxx 2007 4 What We Do Mission- We enable knowledge superiority to Naval and Joint Warfighters through the development, acquisition, and life-cycle support of effective, integrated C4ISR Information Technology, and Space capabilities. We are the Principal C4I Acquisition Engineering & Integration Center on the East Coast & Principal C4ISR ISEA for the Navy Connecting the Warfighter to the resources needed to win GWOT Body Worn Variant MWR- MobileNet IR Pocketscope NETCOP-Network Common Operating Picture Connecting the Warfighter Vision- Fully Netted in Three

5. Approved for release to the public - xxx 2007 5 Who We Are The solutions to the global war on terror developed by SPAWAR result from good systems and software engineering. Systems engineering is our core competency. Total workforce of ~ 2,300 employees. A Large Systems & Software Engineering Organization Over 70% of workforce is in an engineering or computer-related discipline Computer Science/ Engineering, 606 Contracts & Supply, 112 Finance & Budget, 77 General Clerical, 51 IT Support, 76 Logistics, 79 Other, 174 Program Mgmt, 106 Science & Engineering, 1049 46% 5% 3% 7% 3% 2% 3% 26% 5%

6. Approved for release to the public - xxx 2007 6 The Need for SE Revitalization  System Failures  Cost and Schedule Overruns  NDIA – Top Issues

7. Approved for release to the public - xxx 2007 7 NASA – SE Rise and Fall Strong Systems Engineering throughout 1960’s/1970’s  An engineering organization focused on making the correct engineering decisions to maximize mission success  Successes – Apollo missions, Viking, and Galileo Shift from Engineering focus to Programmatics  Reduction in Systems Engineering rigor  Budget and Schedule focus System Failures  1986 – Challenger Shuttle – “O” rings  1996 - Ariane 5 launcher exploded – data conversion error  1998/1999 - “Three successive Titan IV mission failures, an Athena failure and two straight mission losses of the …Delta III, …mark the worst string of major U.S. launch accidents in 13 years.” Ricky Butler, NASA –Delta III launch vehicle - control system failure –Titan IV - $1.23 billion - incorrect software  2000 – Mars Polar Lander crash – single line of code  2003 – Columbia Shuttle disintegration – multiple points of failure

8. Approved for release to the public - xxx 2007 8 The Overrun Problems 3 main reasons for overruns  Unstable requirements  Inadequate Systems Engineering  Program budget instability Average overrun in USAF programs36% Average schedule delay2 years Average program length10 -12 years USAF Study - 2004

9. Approved for release to the public - xxx 2007 9 The Need for Early and Persistent Systems Engineering “I see examples of well-run and poorly run projects. The difference becomes clear in the first few minutes of the review. What is also clear is the important role that systems engineering plays in making a project run smoothly, effectively, and efficiently…” Michael Wynne Acting Under Secretary of Defense (AT&L) From 2004 PEO/SYSCOM Keynote

10. Approved for release to the public - xxx 2007 10 Top Five Systems Engineering Issues Lack of awareness of the importance, value, timing, accountability, and organizational structure of SE on programs  Program teams driven by cost and schedule, not disciplined SE  Emphasis on speed; fix in next spiral  SE inadequately considered in decisions  Risk poorly defined and managed Qualified resources are generally not available within government and industry for allocation on major programs Insufficient SE tools and environments to effectively execute SE on programs  System complexity is ever increasing  Need a structured methodology and adequate tools to understand and manage the complexity Poor initial program formulation Requirements definition, development, and management is not applied consistently and effectively From NDIA Study, January 2003 Adapted from “Technical Excellence through Systems Engineering” by Mark Schaeffer - Principal Deputy Director, Defense Systems; Director, Systems Engineering; OUSD (AT&L)

11. Approved for release to the public - xxx 2007 11 SE Revitalization Effort using CMMI ®  Vision  Process Improvement Organization  SE Revitalization Plan  Standard Processes and Guidance  Tools

12. Approved for release to the public - xxx 2007 12 SPAWAR Charleston - Vision Vision  Develop and maintain a World Class Systems Engineering Organization Approach  Achieve Command-wide operational consistency  Based on ISO/IEC 15288 – systems engineering  Based on ISO/IEC 12207 – software engineering  Based on implementing CMMI ® “Staged Respresentation”  Measure using best practices of CMMI ® “Continuous Representation” SSC-C commitment reaffirmed and formalized in Process Improvement Policy, 11 December 2003

13. Approved for release to the public - xxx 2007 13 Which one is World Class? When you want it done right, Who do you want working on it ? Rigorous processes, Skilled resources Cutting corners, undisciplined, untrained

14. Approved for release to the public - xxx 2007 14 Best practices, exceptional operations INTERNAL Develop a World Class Systems Engineering Capability IP1 Develop a World Class Business Operation IP2 Process-Focused Perform Core Work In- House IM1 Enable Capacity for Context Work IM2 Mission-Focused Synchronize Command Awareness of the Customer IC1 Customer-Focused Connected knowledge, champion teams LEARNING & GROWTH Align Competencies with Strategies HC1 Develop Proactive and Visionary Leadership HC3 Accelerate Innovation and Experimentation HC2 Talents Promote a Culture of High Performance, Accountability and Teamwork OC1 Teams Provide Effective Knowledge Transfer INC1 Tools STAKEHOLDER Fully Netted in Three S1 Fully Netted in Three S1 Fully Netted Resources S3 Fully Netted Force S2 Unsurpassed partnerships, invincible national defense CUSTOMERFINANCIAL Optimized efficiencies, exemplary stewardship Provide Composable C4ISR Capability C1 Build Relationships that Last C3 Optimize Financial Health F1 Manage Cost of Operations F2 Maximize Return on Assets F3 Improve the Customer’s Total Experience C2 SSC-C Strategy Map for BSC

15. Approved for release to the public - xxx 2007 15 Process Improvement Infrastructure: Organization Strategy Tactical Implementation Define and Manage Standard Processes Vision AD IPT Engineering Process Office (EPO) SE IPT Enterprise Process Group (Ent PG) Codes 09K / 09A Business Board CM IPT Dept. Code 50 EPG LOG IPT Corporate Engineering Process Group (EPG) WFO IPT Facility IPT RDT&E IPT PPQA IPT Management Steering Group (MSG) Dept. Code 60 EPG Dept. Code 70 EPG Dept. Code 80 EPG Corporate Business Process Group (BPG) TecInn IPT J0R J0M J02 J01 J09W J0C J0A J09C J0E J09F J09T J09A M. Kutch Dir. of Engineering Operations $$$ Staff Team Chairman

16. Approved for release to the public - xxx 2007 16 SSC-C SE Revitalization Plan Elements of SSC-C SE Revitalization Assessment & Support Training / Education Intro to PI WBT SSC-C SE Instruction SSC-C SE Process Manual SSC-C SW-Dev Process Manual Policy / Guidance CMMI ® Level 2 IT Tools ePlan Builder Completed/Ongoing Underway CMMI ® Level 3 SE 101 WBT Integrated Product Teams SE Fundamentals Intro to Software Engr. Certification/Degrees Lean Six Sigma SSC-C SW-Maint Process Manual Architecture Dev. WBT Project & Process Workshop Balanced Scorecard SE for Managers EPO Website Project Reviews

17. Approved for release to the public - xxx 2007 17 ISO/IEC 15288 System Life Cycle Processes SSC-C Systems Engineering Process ISO/IEC 12207 Software Life Cycle Processes SSC-C Software Engineering Process Use CMMI ® to Measure & Assess Processes SPAWAR Instruction 54xx.1 (draft) SYSTEMS ENGINEERING POLICY Industry Standards SPAWAR Policy SSC-Charleston Engineering Processes Improvement Policy (Dec. 2003) Policies mandate the use of ISO/IEC life cycle processes and CMMI ® best practices CMMI ® for SE/SW PPPMCCMREQMPPQAMASAM RDTSPIVerValRSKM IPM DAR OPFOPDOT CMMI ® Supporting Process Areas to SE/SW Processes ML2 ML3 Top Level SSC-C Processes SSC-C Engineering Process Guidance

18. Approved for release to the public - xxx 2007 18 Classic System Engineering “Vee” Diagram Aligning SE with CMMI and Process Improvement Understand User Requirements, Develop System Concept and Validation Plan Develop System Performance Specification And System Validation Plan Expand Performance Specifications into CI “Design-to” Specifications and CI Verification Plan Evolve “Design-to” Specifications into “Build-to” Documentation and Inspection Plan Inspect to “Build-to” Documentation Assemble CIs and perform CI Verification to CI “Design-to” Specifications Integrate System and Perform System Verification to Performance Specifications Demonstrate and Validate System to User Validation Plan Design Engineering Decomposition and Definition Integration and Qualification time Systems Engineering Fabricate, Assemble, and Code to “Build-to” Documentation RD TS PI Ver Val RSKMDARPPPMCCMREQMPPQAMASAM CMMI Process Areas align with SE “Vee”

19. Approved for release to the public - xxx 2007 19 What is expected from SSC-C projects? Execute sound systems engineering  Utilize the organizational processes, tools, guidance available Sound systems engineering requires  Proper Project Planning  Control over the project Configuration Management, Requirements Management, IV&V, Quality Assurance, …  Iterative and robust design and development methodology Decomposition of requirements, Analysis, Alternatives, …  Frequent interaction and reviews by peers, management, and customers Assess project’s process compliance  Balanced Scorecard Level 2/3/4 Self Assessment measure The Command is providing the infrastructure, tools, and training to help the projects succeed

20. Approved for release to the public - xxx 2007 20 Engineering Process Office Website: Portal for SE/SW and CMMI Content

21. Approved for release to the public - xxx 2007 21 ePlan Builder Tool ePlan Builder tool  An interactive, web-based application that leads the user through a structured interview process (like TurboTax ® ) to generate a CMMI ® -compliant plan  Includes standard, consistent text  Generates an initial project-specific document Project Management Plan (with Work Breakdown Structure) Configuration Management Plan Process and Product Quality Assurance Plan Requirements Management Plan Measurement and Analysis Plan Systems Engineering Plan (DoD SEP Format)

22. Approved for release to the public - xxx 2007 22 Training  SE Training Architecture  Process Improvement and CMMI®  Systems/Software Engineering Classroom  Web Based Training (WBTs)

23. Approved for release to the public - xxx 2007 23 What do we need to be world class? All employees need a basic understanding of process improvement All project teams need to fully understand the CMMI ® model (all processes, all levels)  To understand all of the best practices and maturity levels  To comply/prepare for DoD and NAVY policy All project team members and supporting personnel need to know how to perform the standard processes and best practices required  How to do good SE, CM, QA, Planning, Measurement, Risk, … To properly prepare for and complete an assessment or appraisal, key project team members need to map the project work products to the practices assessed. These needs can be depicted in a training architecture

24. Approved for release to the public - xxx 2007 24 PI WBT SEI Intro to CMMI ® 3-day Engineering Project & Process Mgmt Workshop SE Fundamentals Intro to Software Engineering Quality Engineering Appraisal & Assessment Workshop 2-day Configuration Mgmt Requirements Analysis SE for Managers Core SSC-C project and engineering processes (Level 2 and 3) Subject Matter Experts - Use commercially available on-site classes SEMP Workshop Prepare Projects for BSC or SCAMPI Existing Existing - revise New - develop New - buy Foundation of PI and CMMI ® Training Architecture

25. Approved for release to the public - xxx 2007 25 Intro to Process Improvement  Over 950 people trained  Provided via WBT  Now Required for all employees CMMI ®  SEI Intro to CMMI ® Over 300 attendees to date  SSC-C Level 2 Processes overview Project Management/Project Monitoring & Control  730 people trained Process-specific Workshops (CM, QA, REQ, M&A)  450 people trained Over 1600 Individuals Trained Total attendance over 3800 * * This accounts for many employees attending more than one course Process Improvement Training

26. Approved for release to the public - xxx 2007 26 Intro to Process Improvement WBT Originally given as a podium course, converted to Web Based Training in 2004 Now required for all employees

27. Approved for release to the public - xxx 2007 27 SEI Intro to CMMI ® for SSC-C 3-day Introduction to CMMI ® course teaches the full CMMI ® model  Students learn how the best practices build and relate across process areas  Learn the terminology SEI-Authorized instructors are well-versed in our implementation to augment material with SSC-C specific content  Highlight SSC-C tools and resources  Actively involved in projects, teams, and infrastructure Over 300 employees trained  Want to build a cultural foundation within the engineering departments

28. Approved for release to the public - xxx 2007 28 Systems Engineering Fundamentals Classes Teach the Systems Engineering process 3-day on-site, classroom course  Based on SMU SE Masters course  Customized to incorporate SSC-C SE process  Over 300 SSC-C engineers trained 1-day SE for Managers course added “Thought provoking, motivating, and challenging. Learning basic SE caused me to brainstorm many different applications of organized system processes. It motivated me to want to begin organizing its application. It also challenged me to apply GOOD SE practices in order to successfully be more efficient in the process..” “It was extremely beneficial to have a professor with extensive knowledge of the subject matter and one who could apply it to the SPAWAR methods.” Student Feedback

29. Approved for release to the public - xxx 2007 29 Introduction to Software Engineering Similar format to the Systems Engineering Fundamentals  3 days, primarily lecture  Aligned with the SSC-C Software Development Process Manual  Over 60 SSC-C engineers trained Course Outline  Intro to Software Engineering  Roles  Software Engineering Practices  Software Development Process  Software Maintenance  Managing Software Projects  Tailoring

30. Approved for release to the public - xxx 2007 30 Engineering Management Workshop Multi-session workshop-oriented “how to” class What is a good process? Is my process good? How to generate project plans  What makes a good PMP, CM Plan, QA Plan…  How to use ePlan Builder  Hierarchy of plans (Based on level 2 or level 3 goals) Configuration Mgmt  Are my Configuration Items (CI’s) and Change Control adequate? PPQA  How to execute a process review and work product review Measurement and Analysis  Are my measures measurable? Requirements Management  Traceability - simple to complex Monitoring and Control using Reviews

31. Approved for release to the public - xxx 2007 31 SE 101 Web Based Training Introduction to Systems Engineering WBT  10-module web based training  Closely aligned to SSC-C SE Process, SE Fundamentals Course, and ISO/IEC 15288  Includes hotlinks to referenced documentation SSC-C Process manuals, policies, standards  Extensive branching for more detail

32. Approved for release to the public - xxx 2007 32 Topical WBTs Developing web-based training courses in specific topics Architecture Development WBT - completed  Introduction to Architecture Development and DoDAF  Designed to educate and promote value of system architecture to non-architects and new engineers  Tests for understanding Risk Management  Risk identification  Analysis tools and techniques  Mitigation planning  Risk monitoring Requirements Development

33. Approved for release to the public - xxx 2007 33 Summary  Accomplishments  Results and Measures  Lessons Learned  Going Forward

34. Approved for release to the public - xxx 2007 34 What We Have Accomplished Process Focus  Defined Policies and Processes  Aligned with DoD and SPAWAR guidance  Aligned with industry standards and CMMI ® model  Built organization structured around processes and process improvement Training is Critical  Providing Fundamentals of Engineering for new and old professionals  Developed web-based training for “self-paced” and refresher training  Defining a structured technical career development path for engineers Tools for the Engineers  Developed ePlan Builder application to generate planning documents  Developed templates, checklists, and web-based document repositories to link standards and DoD guidance to day-to-day tasks and processes Early and persistent Systems and Software Engineering applied to programs and projects

35. Approved for release to the public - xxx 2007 35 Results and Measures Selected pilot projects  Training and Mentoring of project teams Informal Appraisals, Process Reviews, and Document Reviews to measure progress and identify gaps  Class B/C appraisals of selected projects  Define/review project-specific plans and procedures Project-level Formal SCAMPI SM Appraisals (Class A)  Evaluated compliance with CMMI ® Maturity Level 2 requirements  8 projects appraised between June 2004 and February 2005 Command-wide ML2 appraisal in April, 2005 Immediately began ML3 effort using similar approach Program/Project-level ML3 SCAMPI SM Appraisals completed  5 Successfully Achieved Maturity Level 3  Preparing for Command-wide ML3 appraisal in April, 2007 Continue to internally assess additional projects against Maturity Level 2 and 3 best practices for Balanced Scorecard

36. Approved for release to the public - xxx 2007 36 Is the SE Revitalization Working ? Recognition of SE and CMMI effort  1 st SPAWAR Systems Center to achieve Maturity Level 2  1 st SPAWAR Systems Center to have programs achieve Maturity Level 3 (AP, JTWS, SCN, VIDS, NAVMACS II)  Multiple presenter at NDIA SE and CMMI conferences High interest in Tools, Training, and Implementation Business  SCN: “They see us as a model and want to increase our efforts.”  Automation Program: “We had hundreds of sites and there was a need for a structured organization to put a ‘wrapper’ around that and control it. CMMI became the wrapper.”  CICS: “CMMI was key to achieving the project goal.”  VIDS: “The VIDS failure (2000) motivated implementing CMMI because the team needed to change course or the customer would have no confidence in system development. It was a tremendous success…”

37. Approved for release to the public - xxx 2007 37 Lessons Learned Senior Management support is critical to success Training  Everyone needs to be engaged – “train the masses”  Specific training for process owners/subject matter experts Utilize Teams (IPTs) as champions of specific processes  Multi-department representation  Change agent mentality  Process-focused charters Resource Properly  Implement with projects that want to improve, can benefit from efforts, and that recognize own weaknesses  EPO staff provided skilled coaching, resources, support, and tools  Project members learned by doing and maintaining Goals and Publicity  Keep goals to sizable bites (projects)  Publicize successes; Share best practices

38. Approved for release to the public - xxx 2007 38 Implementation Plan - Future 2007  Conduct interim Maturity Level 3 appraisals on projects  Correct findings and strengthen institutionalization  Conduct Command Maturity Level 3 appraisal in April, 2007  Incorporate new version of CMMI model (V1.2)  Incorporate IPPD (Integrated Product and Process Development)  Push Level 2 projects to Level 3 2008 - 2009  Begin Maturity Level 4/5 implementation  Establish/Refine organizational and project measures  Increase collection of project, process, and organizational measurement data  Conduct interim Maturity Level 4 appraisals on projects

39. Approved for release to the public - xxx 2007 39 Automated Tools  ePlanBuilder  eWBS Training – 1,600+  SE/SW Fundamentals - 365  Web-Based Training courses SSC-C PI; Intro to SE; Arch. Dev. Summary Aggressive SE Program Industry Standards  Systems Engineering  Software Engineering Best Practices Successes  April 2005 Command Achieved CMMI ® Maturity Level 2 as certified by Software Engineering Institute  5 Programs/Projects achieved CMMI Level 3 in 2006  1 st SPAWAR Systems Center to achieve these levels Goals  World-Class SE Program  Support Command Balanced Scorecard  April 2007, Command to achieve CMMI ® Level 3 ISO 9001 corpweb2.spawar.navy.mil/cmmi/ EPO Website

40. N65236-ENGOPS-BRIEF-0028-1.2Approved for release to the public - xxx 2007 Thank you ! Any Questions ? Contact Information: Michael T. Kutch, Jr. SPAWAR Systems Center Charleston Email: michael.kutch@navy.mil Phone: 843-218-5706michael.kutch@navy.mil

41. N65236-ENGOPS-BRIEF-0028-1.2Approved for release to the public - xxx 2007 Backup Slides

42. Approved for release to the public - xxx 2007 42 What is Systems Engineering? SPAWAR Corporation Definition An interdisciplinary engineering management process that evolves and verifies an integrated, life cycle balanced set of system and enterprise-service solutions that satisfy customer needs. Properties: Stages: ComprehensiveConcept IterativeDevelopment RecursiveProduction SequentialUtilization Top-downSupport Product and ProcessesRetirement