1. Model-Based System Integration (MBSI) An Instructional Approach Dr. Paul Montgomery Associate Professor of Systems Engineering Naval Postgraduate School May 15, 2012 1

2. Perspectives Calibration  Developmental SE  SI – Different perspective from SEs  DoD – Developing complex systems  Academia – SE professor  MBSE – Evolving and maturing  MBSI – An idea (MBSE from SI’s perspective) 2

3. BLUF  Integration Integration begins at design  Modeling Don’t try to integrate the system until you successfully integrate the system model  Instruction Integration must be experienced, not merely studied 3

4. Essential Concepts of I&Q 1  Integration = ensuring the system comes together Interfaces (connectivity and flow)Interfaces (connectivity and flow) Interactions (also interoperability)Interactions (also interoperability)  Qualification = ensuring the system is acceptable to the customer (aka ‘acceptance’) Building the system correctly (aka ‘verification’)Building the system correctly (aka ‘verification’) Building the correct system (aka ‘validation’)Building the correct system (aka ‘validation’) 1 I&Q = Integration and Qualification 4

5. What’s the Problem?  Many system developments fail at integration & qualification (I&Q) … and fail badly  Added cost, schedule, and needed redesign 1 Hershey Foods Corp. PROJECT: IBM-led installation and integration of SAP, Manugistics Group Inc. and Siebel Systems Inc. software…Hershey sales fell 12% in the quarter after the system went live — down $150.5 million compared with the year before Norfolk Southern Corp. PROJECT: Systems integration with merger target Consolidated Rail Corp…Norfolk Southern lost more than $113 million in business during its 1998/1999 railroad merger with Conrail. Custom logistics software wasn’t tested properly and a dispatcher mistakenly fed bogus test data into the system Tri Valley Growers PROJECT: Oracle Corp. ERP and application integration…Tri Valley bought at least $6 million worth of ERP software and services from Oracle in 1996. None of the software worked as promised; some of it couldn’t even be installed on Tri Valley’s DEC Alpha hardware, the co-op claimed in a $20 million lawsuit filed in February. From: “Top 10 Corporate Information Technology Failures” 5

6. Where are we (DoD) Going? - DoD and SoS/LSI (Gansler)  SoS acquisition and engineering is the norm in DoD  SoS design, integration and qualification (I&Q) is highly complex  DoD engineering workforce not well aligned to LSI responsibilities 6 Government oversight of LSI has been complicated with contractual ambiguities Delineation of “inherently governmental functions” for LSI needs more clarity Private LSIs have inherent conflicts of interests without specific controls SoS integration requires a strong, centralized LSI

7. If SE is Well Defined, Why is I&Q a Challenge?  What’s wrong with this picture? 7

8. INTEGRATION BEGINS AT DESIGN MBSI 8

9. What is a System Model? 9 CORE Model Functional Decomposition (Hierarchy) Functional Flow Model (FFBD) Interface Diagram (N 2 ) Generic Physical Block Diagram Behavior Diagram (Sequence) Functional Process Model (IDEF0)

10. MBSI – The SI’s MBSE Perspective Design Environment Qualification Environment Modeling Environment Integration Environment MBSE? MBSI ? 10

11. SE Activities Should Produce System Definition/Model Functional Model Operational Model Interface Model Physical Model Behavioral Model System Definition (“Model”) 11

12. System Model Underpins I&Q Activities 12

13. Don’t try to integrate the system until you integrate the model System Modeling 13

14. Progressive Integration 14 Different teams in diverse locations

15. Integration and Qualification Considerations from Functional Analysis 15 Complex flows/connectivity may indicate complex interactions and bears special attention for integration and qualification focus (or possible redesign)

16. Integration and Qualification Considerations from Behavior Analysis 16 High behavioral interaction activity bears special attention for integration and qualification focus (or possible redesign)

17. Integration and Qualification Considerations from N 2 Analysis 17 Function 1 Function 2 Function 4 Function 5 Large number of interface content (complex interactions) can warrant special integration and qualification focus (or possible redesign) Function 3

18. INTEGRATION MUST BE EXPERIENCED, NOT MERELY TAUGHT I&Q Instructional Methods 18

19. System Model System model is essential for project success Popular Approaches to Teaching I&Q SE Fundamentals SE Integration Test and Eval Concept Design Build Integrate Qualify Process Approach “Toys” Approach End-to-End Approach Shortfalls: Non-tangible experience Hard to develop I&Q instincts Disjointed learning experience Shortfalls: Cannot design components Interfaces are fixed Interface design may be hidden Shortfalls: Not enough time Not enough student skills Set up for failure 19

20. A PROJECT MBSI Instructional Example 20

21. Overview of Class Project SOH Submarine Detection using Fire Scout (STRAIT SCOUT) 21

22. Customer Problem Statement Problem In the Persian Gulf, we do not have a reliable system to detect submarines that egress and ingress through the SOH by hiding in tanker wakes. Research Questions Can a combination of BAMS and one FireScout be used to provide a high P d of the submarine behavior above? What is shipping traffic density vs. P d performance of such a system? What are some FireScout search strategies for such a system deployment? 22

23. Primary System Assets  BAMS - Persistent surveillance over AOR with surface search Radar  Fire Scout Speed = 0 – 90kts  FireScout Sensor = LIDAR (Light Detection and Ranging) Scanning Sub – inch resolution 23

24. Top-Level STRAIT SCOUT Architecture Concept 24 SensorSensor Red Team FireScoutFireScout C2C2 Test Parameters (Parameters, Scenarios, Results) 1 1 6 6 5 5 4 4 7 7 1 – Ships and sub position data 2 – N/A 3 – Surface track data 4 – Sensor data 5 – Flight path commands 6 - Position data 7 – Environment parameters 8 – Test results 6 6 8 8 8 8 BAMSBAMS 1 1 3 3

25. Team Roles & Responsibilities  LSI Primary stakeholder negotiations Top-level architecture Taxonomy and structure – External systems interfaces – Intra-subsystem interfaces – Functional naming conventions Conop Integration and qualification strategy – Integration strategy – Acceptance goals, objectives, and agreements Leadership  Subsystem teams Subsystem derived requirements Subsystem Design – Functional, interfaces, and generic physical Subsystem integration and qualification  Instructor = Primary customer/stakeholder 25

26. SE Design  Define the problem  Develop functional architecture  Develop physical architecture  Develop operational architecture  Develop interface architecture  Define integration, test, V&V strategy 26

27. Simulation Concept 27 time = t time = t n FireScout sensor scan field Tanker track and direction time Detection? Wake (no sub) Wake (with sub)

28. VBA Project I&Q Environment Excel ™ VBA Excel ™ VBA Excel ™ VBA “Dictator” VBA “Dictator” Excel ™ Subsystem A Subsystem B Subsystem C System LSI Team Subsystem Teams Advantages: Readily available “Office” tools Concept-to-design Interface visibility Team integration = subsystem integration LSI integration = system integration Disadvantages: VBA is not innate SE skill Too much to do in time alloted Integration can still be undisciplined 28

29. MBSI Environment Implementation Physical Modeling Behavioral Modeling Interface Modeling Qualification Modeling Conop Needs Mission Constraints Assumptions Goals Objectives Cell formulas and VBASheets and cells modeling Functional Modeling “Design” “I&Q” 29

30. Primary Class Project Phases  System design  Model integration (CORE 1 )  System development (code)  Subsystem & System integration  System verification (test)  System validation (demonstrate)  System acceptance (grade) 1 CORE 8 (University) Service pack 3 30

31. THE SI PERSPECTIVE MBSI Instructional Example 31

32. SI Challenge Questions  Do you understand your problem and what your subsystem needs to do?  Do you understand enough about your subsystem behaviors to define functions?  How many functions are in your subsystem?  Are the functions “modular” and simple?  How many interactions do you expect?  How many external interfaces do you need to define?  How many internal interfaces do you need to define?  Have you thought of which functions need to be integrated first?  What are the integration and qualification risks that are starting to emerge? 32

33. Simplified Strait Scout Sequence Diagram? 33 C2 FS LSI Red Sensor Setup / Run Sub detected Record Terminate Locations Flight Cmds Fly/ Location loop BAMS Target Data

34. Strait Scout Functional Context Systems 34 Control flow is linear?

35. N2 35 Interface complexity?

36. IDEF 36 Many interfaces?

37. Sequence 37 Triggers? Responses?

38. Student Discoveries  Early requirements clarification is important  Early architecture design imperative (especially functional and interface)  Rushing to development prior to model definition wastes time and effort  Early model integration drives out:  Functional gaps and overlaps  Interface inconsistencies and discontinuities  System behavior misunderstandings  Inter and intra-system interface problems  SI involvement in design can reduce risk during I&Q  Project would have failed without MBSE/MBSI methods 38

39. LESSONS LEARNED MBSI – An Instructional Approach 39

40. Value of MBSI  Successful I&Q requires: Strong LSI / SI Detailed system definition (particularly interfaces and functional interactions) Early taxonomy and structure definition Early SI influence with I&Q success perspective Modeling in order to discipline design efforts Model integration prior to system integration to reduce I&Q risks Diverse and integrated SE/SI support system (i.e. tool sets, etc.)  MBSE tools not yet MBSI tools  “Teach” I&Q using MBSI applied to experiential project 40

41. References  Handbook of Systems Engineering and Management, Sage and Rouse (ed.), Wiley and Sons, 1999, Chapter 14  Systems Engineering Guide for Systems of Systems, Ver 1.0, Aug 2008, Director, Systems and Software Engineering, DUSD (Acq and Tech), OSD (AT&L)  The Role of the Lead System Integrator, Gansler, et.al., NPS-AM-09-005, Jan 2009  Top 10 Corporate Information Technology Failures, www.computerworld.com/computerworld/records/images/pdf/44NfailChart. pdf www.computerworld.com/computerworld/records/images/pdf/44NfailChart. pdf  http://cdn.business2community.com/wp- content/uploads/2012/04/customer-experience-focus.jpg http://cdn.business2community.com/wp- content/uploads/2012/04/customer-experience-focus.jpg  http://www.fas.org/man/dod-101/sys/ac/f-18-ef-usnhornt.jpg http://www.fas.org/man/dod-101/sys/ac/f-18-ef-usnhornt.jpg  http://www.behavioradvisor.com/sbPuzzled.jpg http://www.behavioradvisor.com/sbPuzzled.jpg  http://www.blogspot.com/-hSnr-sGBKko/T2_- WfiI8DI/AAAAAAAAD88/tzcvl7wxgRA/s1600/acellphone.gif http://www.blogspot.com/-hSnr-sGBKko/T2_- WfiI8DI/AAAAAAAAD88/tzcvl7wxgRA/s1600/acellphone.gif 41