Systems Engineering Case Studies Charles M. Garland Air Force Center for Systems Engineering (937) 255-3355 x3368 Charles.Garland@us.af.mil Dr John Colombi Dept of Systems and Engineering Mgt Air Force Institute of Technology October 3, 2008

Agenda Air Force Center for Systems Engineering Case Studies Case Framework/ Approach Learning Principles Teaching Systems Engineering Cases

Air Force Center for Systems Engineering

AF CSE Vision and Mission Vision: Become a national center of excellence for systems engineering, from theory to application, for the defense community. Mission: Shape the future of systems engineering in the AF and DoD to improve our ability to deliver war-fighting capabilities. We will accomplish this by conceptualizing new processes, practices, tools, and resources for the SE workforce through research, education, and consultation.

for Systems Engineering AF CSE Leadership Director Mr. George Mooney Deputy Director Technical Director Col. John Camps Mr. G. Richard Freeman Education & Training Division Applications & Development Division Curriculum Chair for Systems Engineering Maj. Jeffrey Havlicek Mr. Mike Ucchino Dr. David Jacques

Systems Engineering Case Studies Began under the academic oversight of a Subcommittee on Systems Engineering to the Air University Board of Visitors Chaired by Air Force Chief Scientist Dr. Alex Levis Selected four programs for initial case studies (Hubble Space Telescope, Theater Battle Management Core System, F-111, and C-5) AF CSE Strategic Plan for Case Studies Five year plan updated every three years Identified, evaluated, and prioritized candidate programs

Completed Case Studies Hubble Space Telescope GPS (Global Positioning System) F-111 Aardvark The first 4 covered various domains of applying systems engineering and perhaps captured the unique characteristics of those domains C-5 large mobility aircraft F-111 Joint Navy and Air Force multi-role fighter TBMCS is a Command and Control software-intensive systems Hubble examined a precision electronic-optical space systems B-2 C-5 Galaxy TBMCS (Theater Battle Management Core Systems) Peacekeeper Intercontinental Ballistic Missile A-10

International Space Station Ongoing & Future Case Studies International Space Station MH-53J/M Helicopter E-10 on contract underway FY09 Option FY10 Option Global Hawk KC-135 Simulators T-6A Texan II Other domains include commercial aircraft, rotorary wing, simulators and unmanned air vehicles on contract underway FY09 Option underway

Friedman-Sage Framework Based on student heuristics, developed by: Dr George Friedman: University of Southern California Dr Andy Sage: George Mason University Comprised of 9 concept domains (rows) & 3 responsibility domains (columns) Rows represent phases in SE life cycle & necessary process and systems management support Columns depict responsibilities from both sides of the program (industry and government) Derived into matrix - Identifies learning principles Used to organize the case writing First 6 concept domains (rows) deal w/ life cycle, last 3 with SE processes. * Case Studies of Systems Engineering and Management in Systems Acquisition. Systems Engineering, Vol.7, No. 1, 2004

Friedman-Sage Framework

Peacekeeper Learning Principles

Peacekeeper LP1 Development commands must manage their technology base to optimize progress over several programs. Ballistic Missile Office (BMO) developed and managed a technology base that spanned several programs Atlas, Titan, Minuteman, Peacekeeper and Small ICBM This matured technologies such as: Solid rocket propellants Nozzle manufacture Liquid fueled engines Guidance systems During this period of Intercontinental Ballistic Missile (ICBM) development, Ballistic Missile Office (BMO) was able to develop and manage a technology base that spanned several programs. BMO and its predecessors began developing technologies with the Atlas system that continued through the Titan, Minuteman, Peacekeeper and Small ICBM. This included technologies such as: solid rocket propellants, nozzle manufacture, liquid fueled engines, and guidance systems.

GPS Learning Principles The 4 GPS learning principles are mapped in the matrix.

GPS Learning Principle 2 The systems integrator must rigorously maintain program baselines Joint Program Office (JPO) retained the role of managing and controlling the systems specification This allowed control of functional baseline JPO derived and constructed an “agreed-to” set of systems requirements that became the program baseline Performance/Risk/Cost trade studies against functional baseline Interface Control Working Group managed the functional requirements of the allocated baseline Processes gave JPO first-hand knowledge and insight into risks at lowest level JPO had resources to be the system integrator. That might not be the case in today’s environment. Some government programs do not have the staff to be the system integrator. In the case of the prime (or FFRDC) contractor serving as the system integrator, the government must maintain control of the system baseline. It is incumbent upon the government to have keen insight (not just oversight) into the contractor'

Hubble Learning Principles The 4 GPS learning principles are mapped in the matrix.

Hubble Learning Principle For complex programs, the number of players (government and contractor) demands that the program be structured to cope with high risk factors in many management and technical areas Contractors Lockheed (LMSC) and Perkin-Elmer (P-E) “owned” very significant and unique program risk areas LM was the overall integrator P-E was the technical expert in the critical optical system Lack of insight into quality assurance led directly to the primary mirror defects, in spite of substantial evidence otherwise JPO had resources to be the system integrator. That might not be the case in today’s environment. Some government programs do not have the staff to be the system integrator. In the case of the prime (or FFRDC) contractor serving as the system integrator, the government must maintain control of the system baseline. It is incumbent upon the government to have keen insight (not just oversight) into the contractor'

Synopsis of Learning Principles Created as a reference tool for practitioners Categorizes LPs Consider adopting the approach highlighted Problem to be avoided Will be updated as more case studies are completed

Learning Principle Distribution

Case Study Availability http://www.afit.edu/cse/cases.cfm

Case Study Audience? Support teaching of Systems Engineering principles Systems engineering/ programmatic decisions Operational effectiveness Processes, principles, tools Decision material Highlight the importance of skills from multiple functional areas, including multiple engineering disciplines Audience – students in a classroom? Audience – practitioners (engineers/management) throughout the organization? Can a single case address both?

SE Case Study Format Format is different for the audience Students Typical Harvard Business School case Chronologic story emphasizing decision making Shorter length of typically 10-30 pages No “answers”, guided discussion and personal discovery Practitioners Executive Summary Detailed treatment of Systems Engineering activity What to avoid - What to emphasize/apply We attempt to blend both

SE Case Study Format For Practitioners (Part 1) Executive Summary Description of the Learning Principles Students (Case Body Part 2) Systems Engineering Intro/ Appropriate guidance System Description Detailed Chronology Insightful Questions Appendix Material -Trade studies, reviews, Requirements, architecture, other analysis

Case Study Scope Need to understand scope as key controlling factor Time/ Schedule Total Resources Outline/ Page Allocation Scope! Focus on 4-6 LP Apply a framework Assessment Reference

Success Don’t confuse operational system success with systems engineering success C-5 example: Heavy-lift aircraft capable of carrying multiple tanks and related equipment Maximum take-off Gross Weight over 764,000 lbs! Unique front and aft ramps facilitate easy drive-on, drive-off loading of military vehicles and equipment Accomplishes tasks that no other military aircraft can

C-5 Success Synopsis So while a very successful operational aircraft, but LP #2. Total Package Procurement Concept (TPPC) was a fixed-price, incentive fee contract strategy for the design, development, and production of 58 aircraft. Invented to control cost growth – underlying cause for overruns LP #3. A Weight Empty Guarantee was included in the specification and in the contract as a cost penalty for each delivered overweight aircraft. Contract Penalty: $10,000 per pound per delivered aircraft Dominated the traditionally balanced requirements resulting in a major shortfalls in wing and pylon fatigue life Negative effects of forcing (out-of-balance) one system parameter Trend in forcing an aircraft from “nominal” weight LP #1. Systems requirements need to integrate the User (warfighter), planners, developers, and technologists in a coordinated, well-balanced, well-understood set of requirements, which remain unchanged throughout the program. LP #2. Total Package Procurement Concept (TPPC) employed by the Gov’t was a fixed-price, incentive fee contract for the design, development, and production of 58 aircraft. This contract strategy was invented to control costs but it was the underlying cause for the overrun. LP #3. A Weight Empty Guarantee was included in the specification as a performance requirement and in the contract as a cost penalty for each delivered overweight aircraft. This aircraft Weight Empty Guarantee dominated the traditional balanced aircraft performance requirements and resulted in a major shortfalls in the wing and pylon fatigue life. LP #4, Independent Review Teams (IRTs). The Air Force C-5 Systems Program Office employed Independent Review Teams (IRTs) to assemble the national experts to examine the program and provide recommendations to the government. These problem-solving teams were convened to garner the best advice in particular technical areas; structures design, structures technology, and designs to achieve useful service life.

Insight into TPM (Weight) Expected weight trend

Teaching* How best to teach (Systems Engineering)? Wrong question How best do students learn? Student centered Discussion based Active Learning *Participant Centered Learning and the Case Method, Harvard Business School Publishing

Participant Centered Learning Harvard Business School Participant-Centered Learning and the Case Method Typically a class will cover a 2-3 key points (LPs) No Lecturing Students don’t have the LPs… need to discovery as a class Students must apply course material together with experience Listen to other students Professor will elicit student participation “How do you know that …?” “What do you think about …?” “Why?” Professor questions, listens and responds Organize responses, ask more questions, summarize

Summary Feedback suggests these can be very effective to support Graduate and Continuous Learning courses Provide real-world, well-known examples Target an organization / domain DoD examples for DoD students NASA examples for NASA students Provide for practitioners and resident students Use a student-led Case Study pedagogical approach