1. Resource Analysis Based On System Architecture Behavior Monica Farah-Stapleton Professor Ray Madachy Professor Mikhail Auguston Professor Kristin Giammarco November 2015

2. Overview of Today’s Presentation Pervasive Challenge and Response Research Alignments Applying FP Methodology To MP Model Relating Activities Closing Thoughts 2 We would like to acknowledge Lori Holmes-Limbacher for her contributions as a FPA subject matter expert and the Q/P Management Group, Inc. for the Tee Time System example.

3. Pervasive Challenge and Response Challenge Information Technology (IT) systems are large, challenged by the rate of change of commercial IT, and represent a significant investment in time and resources –Introduction of new system/capability results in unintended or unexpected system/environment behaviors –Operational and financial impacts often assessed after the fact –Resourcing decisions and precise architectural descriptions of system/environment often minimally related Approach Leverage precise behavioral modeling using Monterey Phoenix (MP) to assess architectural design decisions and their impacts prior to, during, and after deployment Relate architectural modeling to resourcing through analysis of behaviors and Unadjusted Function Point Analysis (FPA) counts, leveraging complexity and size metrics, e.g. Data Element Type (DET) and File Type Referenced (FTR) Align activities with Systems Engineering Research Center (SERC) Ilities Tradespace and Affordability Program (iTAP) 3

4. 4 iTAP Research Objectives Total Ownership Cost (TOC) modeling to enable affordability tradeoffs with integrated software-hardware-human factors Current shortfalls for ilities tradespace analysis –Models/tools are incomplete wrt/ TOC phases, activities, disciplines, SoS aspects –No integration with physical design space analysis tools, system modeling, or each other New aspects –Integrated costing of systems, software, hardware and human factors across full lifecycle operations –Extensions and consolidations for DoD application domains –Tool interoperability and tailorability (service-oriented) Can improve affordability-related decisions across all joint services Current Phase 4 Plans – Extract: –Assess Monterey Phoenix (MP) for automatically providing cost information from architectural models –MP will extract software sizing cost model inputs to compute costs, and we will assess mapping MP architectural elements into systems engineering cost model inputs Please See Our Demo At the Tools Fair 5:00-6:30 PM 4

5. Applying FPA Methodology to MP Architecture Model 5 Step 1: Identify problem statement, i.e. typical questions to be answered, determine type of count Step 2: Describe behaviors of system and environment in natural language, identify scope boundary Step 3: Unambiguously represent behaviors using MP, extract use cases/initial views from MP model Step 4: Relate MP system/environment behaviors to Function Point behaviors Step 5: Identify coefficients that inform complexity and scale, identify counting rules Step 6: Determine Unadjusted Function Point (UFP) count Step 7: Assess effort using MP-COCOMO II/III tool and extracting coefficients Step 8: Visualize results in views specific to stakeholders

6. Event - any detectable action in system’s or environment’s behavior Event trace - set of events with two basic partial ordering relations, precedence (PRECEDES) and inclusion (IN) Event grammar - specifies the structure of possible event traces Basic Concepts for Monterey Phoenix (MP) Behavioral Modeling 6 Innovations: Uniform Framework: Describe behaviors and interactions of the system AND environment using the same framework Leverage Small Scope Hypothesis: Exhaustive search through all possible scenarios (up to the scope limit), expecting that most flaws in models can be demonstrated on small counterexamples Separation of System Interaction from System Behavior: Specify behavior of each system’s components separately from interactions between those components

7. What Does “Separation of System Interaction from System Behavior” Mean? 7

8. Function Point Analysis Functionality From User’s Perspective Determine the Type of Count and Boundary : Development Project, Enhancement Project, Application Terminology: External Inputs (EI): Data that is entering a system External Outputs (EO) and External Inquires (EQ): Data that is leaving the system Internal Logical Files (ILF): Data that is processed and stored within the system External Interface Files (EIF): Data that is maintained outside the system but is necessary to satisfy a particular process requirement Function Point Analysis Practice Count Data and Transactional Function Types Determine Unadjusted FP Count Determine the Value Adjustment Factor N/A Calculate final Adjusted FP Count N/A 8 Function Points: Normalized metric used to evaluate software deliverables Measure size based on well- defined functional characteristics of the software system Must be defined around components that can be identified in a well-written specification Sources: IFPUG

9. Relating FPA and MP Unadjusted Function Point is a unit of measurement to express the amount of functionality in a system, and can be used to estimate system cost. Of specific interest are the input/output activities of the system. MP architecture model is based on behavior modeling, providing a bridge between the requirements and high level design, and is supportive of cost estimates early in the design phase. The concept of an event in MP is an abstraction for activity within the system. It is rendered as a pseudo-code, appropriate for capturing the functional aspects of requirements, and supportive of refinement. UFP can be identified in the MP architecture model as an interaction abstraction (i.e. COORDINATE or SHARE ALL constructs). The structure and the complexity of interactions in MP provide a source for assigning weights contributing to the UFP. Since an MP model is precise and formal, FP metrics can be identified by automated tools such as http://csse.usc.edu/tools/MP_COCOMOhttp://csse.usc.edu/tools/MP_COCOMO 9

10. Tee Time Example System Example UFP Count Synopsis 10

11. Relating Activities: Tee Time System -- Golf Courses List, EQ : State Drop Down 11 Transactional Function: EQ (View or Display Retrieval of Data) Click Button from Main Screen, navigate to Golf Courses list. No EP. Exit button: Navigation, no EP. EQ -- View/Display State Drop Down  Click on state arrow  State list display returned  Stop  1 FTR (Golf Courses ILF), 2 DET (Arrow Click, State field) EQ – View/Display City Drop Down  State data entered  Click on City arrow  City list display returned  Stop  1 FTR (Golf Courses ILF), 3 DET (State data, Arrow Click, City field) EQ – View/Display Golf Courses List  Enter State and City  Click Display button  Name displayed back, don’t count state, city.  Stop  1 FTR (Golf Course ILF), 4 DET(State, City, Name, Click Display) EPDescriptionILF/EIFFTR/DETComplexUFP EQState Drop Down Golf Courses (I) (1,2)Low3 © Copyright 2010. Q/P Management Group, Inc. All Rights Reserved.

12. MP Model for EQ State Drop Down Nested COORDINATE 01 SCHEMA EQ_State_Drop_Down 02 ROOT User_GCL: Inquire_on_state_data something_else; 03 Inquire_on_state_data: Click_state_arrow_dropdown Receive_state_list_display; 04 ROOT Golfcourses_ILF: Get_result anything_else; 05 Get_result: Receive_state_arrow_prompt Send_state_list_display; 06 COORDINATE $x: Inquire_on_state_data FROM User_GCL, 07 $y: Get_result FROM Golfcourses_ILF 08 DO 12 Lines 02-05 represent the behaviors of ROOTs (Actors) Lines 06-17 represent the behaviors of EQ: State Drop Down COORDINATE MP Calculation 1 FTR and 1 nested COORDINATE (COORDINATE & 2 ADDs) correspond to 1 FTR and 2 DETs and a functional complexity weighting of 3 EQ State Drop Down = (1 COORDINATE) * 3 UFP/COORDINATE = 3 UFPs Line 01 Schema Name for EQ COORDINATE 17 OD; 09 COORDINATE $xx: Click_state_arrow_dropdown FROM $x, 10 $yy: Receive_state_arrow_prompt FROM $y, 11 $x11: Receive_state_list_display FROM $x, 12 $y11: Send_state_list_display FROM $y 13 DO 14 ADD $xx PRECEDES $yy; 15 ADD $y11 PRECEDES $x11; 16 OD; Lines 09 -16 represent behaviors of nested COORDINATE, DETs and FTRs. The nested interactions (ADD) influence the weight of this EQ COORDINATE

13. EQ State Drop Down Use Case Sequence Diagram Automatically Generated by Firebird, Scope 2 13 Monterey Phoenix and Related Work: http://faculty.nps.edu/maugusto http://faculty.nps.edu/maugusto MP Wiki: https://wiki.nps.edu/display/MPhttps://wiki.nps.edu/display/MP Public MP server with MP editor, trace generator, and trace graph visualization: http://firebird.nps.edu/http://firebird.nps.edu/

14. FPA Calculation External Inquiry (EQ) EPDescriptionILF/EIFFTR/DETComplexityUFP EQState Drop DownGolf Courses (I)(1,2)Low3 EQCity Drop DownGolf Courses (I)(1,3)Low3 EQGolf Course ListGolf Courses (I)(1,4)Low3 EQGolf Course DetailGolf Courses (I)(1,12)Low3 EQScoreboard DisplayScoreboard (I)(1,6)Low3 EQMaintain Golf Course Display (by ID) Golf Courses (I)(1,13)Low3 EQTee Times Reservation Display Tee Times (I)(1,11)Low3 EQTee Times Shopping DisplayMerchandise (E)(1,3)Low3 EQProduct View PictureMerchandise (E)(1,3)Low3 14 Source IFPUG Counting Manual Part 2, p.7-19

15. MP and FPA UFP Calculation EQ State Drop Down UFP Calculation: Extracted From MP 1 COORDINATE interaction associated with State Drop Down EQ behaviors State Drop down EQ COORDINATE contains a nested COORDINATE (2 ADDs) The 2 ADDs relate to 2 DETs ROOT Golfcourses_ILF relates to 1 FTR 0 -1 FTRs and 1-5 DETs correspond to a Low functional complexity rating A Low functional complexity rating corresponds to 3 UFP EQ State Drop Down is equal to 1 COORDINATE with a weight of 3 or 3 UFPs 15 UFP Calculation: FPA Manual Count 1 FTR and 2 DETs identified from the behavior of the State Drop Down EQ 0-1 FTRs and 1-5 DETs correspond to a Low functional complexity rating A Low functional complexity rating corresponds to 3 UFPs EPDescriptionILF/EIFFTR / DET ComplexUFP EQState Drop Down Golf Courses (I) (1,2)Low3

16. MP-COCOMO Tool Collaboration In Progress 16 Source: http://csse.usc.edu/tools/MP_COCOMO/http://csse.usc.edu/tools/MP_COCOMO/

17. Closing Thoughts Resourcing is a socio-technical problem –Methodology and MP Framework synchronize concepts, architectures, system/software implementation, business processes, organizational dynamics –Relevant in the System of Systems problem space –Make the tools and methods user-friendly, enforce across the enterprise Next Steps –Refine weights for each Transactional Function –Refine relationship between steps of a FPA Elementary Process and MP descriptions Nested COORDINATES ILF and EIF behavioral representations in MP –Apply methodology to iTAP UAV case study and IFPUG case study 17

18. Back Up 18

19. FPA Calculation External Input (EI) EPDescriptionILF/EIFFTR/DETComplexityUFP EIScoreboard (ADD)Scoreboard (I)(1,7)Low3 EIScoreboard (CHANGE)Scoreboard (I)(1,7)Low3 EIScoreboard (DELETE)Scoreboard (I)(1,3)Low3 EIMaintain Golf Course (ADD)Golf Courses (I)(1,13)Low3 EIMaintain Golf Course (CHANGE)Golf Courses (I)(1,13)Low3 EIMaintain Golf Course (DELETE)Golf Courses (I) Tee Times (2,3)Low3 EITee Times Reservations (ADD)Tee Times (I)(1,12)Low3 EITee Times Reservations (CHANGE)Tee Times (I)(1,12)Low3 EITee Times Reservations (DELETE)Tee Times (I)(1,6)Low3 19 Source IFPUG Counting Manual Part 2, p.7-19

20. FPA Calculation External Output (EO) EPDescriptionILF/EIFFTR/DETComplexityUFP EOShopping Display (Calculation) Merchandise (1,7)Low4 EOBuy – Send to Purchasing (Calculation) Merchandise(1,15)Low4 20 Source IFPUG Counting Manual Part 2, p.7-19

21. FPA Calculation Data Functions DescriptionILF/EIFRET/DETComplexityUFP Golf CourseILF(1,11)Low7 Tee TimesILF(1,10)Low7 ScoreboardILF(1,5)Low7 MerchandiseEIF(1,3)Low5 21 Source IFPUG Counting Manual Part 2, p.7-19

22. 22

23. 23 iTAP Phase 4 Plans – Task 2 Collaboration with AFIT for a joint Intelligence, Surveillance and Reconnaissance (ISR) mission application involving heterogeneous teams of autonomous and cooperative agents. NPS will provide cost modeling expertise, tools and Monterey Phoenix (MP) modeling support. A focus will be on translations between models/tools in MBSE, specifically mapping architectural elements into cost model inputs. Approach –Develop a baseline operational and system architecture to capture a set of military scenarios. –Transition the baseline architecture to the MP environment. –Utilize the executable architecture modeling framework of MP to perform automated assertion checking and find counterexamples of behavior that violate the expected system's correctness. Operational scenarios will be cycled through the MP modeling process, whereby alternate events are captured for each actor in each scenario. This will produce a superset of scenario variants from the behavior models, suitable for input to tradespace analysis and cost models. –Design and demonstrate an ISR UAV tradespace. –Develop cost model interfaces for components of the architecture in order to evaluate cost effectiveness in an uncertain future environment.

24. 24 iTAP Phase 4 Plans – Task 3 Continue extending the scope and tradespace interoperability of cost models and tools from previous phases. Cost modeling will engage domain experts for Delphi estimates, evolve baseline definitions of cost driver parameters and rating scales for use in data collection, gather empirical data and determine areas needing further research to account for differences between estimated and actual costs. –Prototype cost models and tools will be extended accordingly for piloting and refinement. For tool interoperability we will integrate cost models in different ways with MBSE architectural modeling approaches and as web services. We will also automate systems and software risk advisors that operate in conjunction with the cost models. NPS will provide domain expertise for SysML cost model integration with Georgia Tech and USC to add software cost model formulas and the risk assessment capabilities. –This is also allied with Task 2 where we will assess Monterey Phoenix (MP) for automatically providing cost information from architectural models. MP will extract software sizing cost model inputs to compute costs, and we will assess mapping MP architectural elements into systems engineering cost model inputs.

25. A US Department of Defense University-Affiliated Research Center (UARC) The networked national resource to further systems research and its impact on issues of national and global significance Systems Engineering Research Center The Systems Research and Impact Network 25

26. SERC: 22 Collaborators, Led by Stevens Institute 26