Presentation on theme: "Systems Engineering – A Perspective"— Presentation transcript:
1 Systems Engineering – A Perspective Dinesh Verma, Ph.D.Professor and Associate DeanStevens Institute of Technology
2 Simple Definition…Systems Engineering is “problem solving and solution delivery.” A key prerequisite to good “problem solving” is good “problem definition.” Now this has other prerequisites!Some best practices:Translating customer needs (business and technical) into acceptance criteria - 5 to 7 critical customer requirements agreed to in measurable/testable form.Identifying requirements and then managing them (and tracing them) through the subsequent development, integration, testing phases.Translating the requirements into an “architecture” that becomes a “linkage” between what the customers want and what the developers will build and test.Developing a test architecture, test plans and procedures that are traceable to the requirements for maximum focus and efficiencySounds very simple! A lot of organizations have developed processes that attempt to capture the above intent. But very few are able to execute it…
3 System Design Life Cycle Models: An Automotive Example (VOLVO Car Corporation) 3
4 System Design Life Cycle Models: A Telecom Example (NOKIA Networks) Capability forVolume DeliveriesProgram initiatedProgram proposal readyProgram plan readyReady for integrationReady for verificationReady for Ramp-upE-1E0E1E2E3E4E5DefinePlan andspecifyDesign and implementImplement and integrateVerifyRamp-upProgram allowedto start usingresourcesProgramestablishedCommitment offeatures, resourcesand milestone dates.Specification doneReal HW doneand HW in maintenancemode. HW and SW mainverification starts.SW is module tested and proofon product functionality exist(=SW implementation ready).Volume deliveriescan start"Proof of concept" *HW implemented.Real HW and basic/ lowlevel SW integrated andcore functionality works.Idea of performance exists.First SW build made.Proof of productarchitecture.Traditional Pilotdeliveries start. HW and SWhave been tested togetherand released as a productProgram main contents frozen for program planning purposes (optional)Requirementsspecs doneProductDesignfrozen(optional)Programcompleted(optional)Trial deliveries can start (Optional)Functional tests done and HW fullfillslegal type approval requirementsE0.5E1.5E 3.5E 5.5Optional Milestones can be moved.I.e. E1 and E1.5 dates can be the same.* Core functionality can be I.e. control plane,signal goes through (typically not call yet). Exact contentsof core functionality is need to be defined in E14
5 System Design Life Cycle Models: A Workstation Example (SUN Microsystems) 5
6 Architecture/Component The IBM AMS Systems Engineering Process defines deliverables and a series of Reviews (I)Need / OpportunityIdentificationConceptualSystem SpecificationComponentArchitectureDetailed DesignCustomerBaselineSystemBaselineArchitecture/ComponentBaselineDesignBaselineBusinessRequirementsReview (BRR)SystemsRequirementsReview (SRR)PreliminaryDesignReview (PDR)CriticalDesignReview CDR)SystemsReq’mentSpecsComponentReq’ment SpecsComponentRTVMBusinessRequire.Specs.SystemLevelArchitect.ComponentLevelArchitectureComponentDesignComponent Test PlanTestArchitectureRTVMCustomer ProvidedSystems Engineering ProvidedComponent Developer Provided6
7 The IBM AMS Systems Engineering Process defines deliverables and a series of Reviews (II) Detail DesignDevelopmentTest and ProductionSystem UpdateNew ProductionSystemDesignBaselineTestBaselineProductionBaselineCDRTestReadinessReview (TRR)ProductionReadinessReview (PRR)SystemTestStrategySystemTest Plan /Test CasesReleaseContentDeploymentPlanComp.DesignTest PlanSystem TestDataTestTraceabilityMatrix.Move toProd.PlanDataMigrationPlanCustomer ProvidedSystems Engineering ProvidedComponent Developer ProvidedSystem Test ProvidedService Delivery / Managed Ops Provided7
9 Key Concepts… The notion of Systems Thinking and Contextual Setting The notion of BaselinesFacilitates requirements definition and management, change management, and scope controlThe notion of Functional (Dynamic) And Non-Functional (Static) RequirementsA holistic view that addresses new or modified Business Processes, while also addressing concerns pertaining to Performance, Availability and Reliability, Scalability and SecurityThe notion of Baseline Reviews and Objective MetricsAn in-process review to validate completeness of the baseline and identify risks and defectsThe notion of the System Life Cycle (Define – Design – Build – Test – Deploy – Operate – Upgrade/Retire)Systems Engineering should have “ownership” and accountability through the various phases of the life cycle.
10 “I try to be very clear in separating the “problem space” from the “solution space”.” Chief Architect, NOKIA Technology Platforms “I try to design at three levels: my level, the level above me, and the level below me.” Chief Architect, L3 Communications “I architect and design in two iterations. In the first, my focus is on feasibility and completeness, while in the second, my focus is on “goodness” and other refinements and embellishments such as scalability, modularity, and such.” Chief Architect, IBM Global Services10
11 Systems Engineering – Expectation Successful implementation of proven, disciplined systems engineering processes results in a total system solution that is:Robust to changing technical, production, and operating conditions;Adaptive to the needs of the users; andBalanced among the multiple requirements, design considerations, design constraints, and program budgets.11
12 Systems Engineering – Expectation Successful implementation of proven, disciplined systems engineering processes results in a total system solution that is:Robust to changing technical, production, and operating conditions;Adaptive to the needs of the users; andBalanced among the multiple requirements, design considerations, design constraints, and program budgets.12
13 DoD Systems Engineering Shortfalls* Root cause of failures on acquisition programs include:Inadequate understanding of requirementsLack of systems engineering discipline, authority, and resourcesLack of technical planning and oversightStovepipe developments with late integrationLack of subject matter expertise at the integration levelAvailability of systems integration facilitiesIncomplete, obsolete, or inflexible architecturesLow visibility of software riskTechnology maturity overestimatedMajor contributors to poor program performance* DoD-directed Studies/Reviews13
14 Some Inhibitors to Good Systems Engineering: Based on a survey of IT architects and project managersCustomer Related Input:Isolation from real “user”Customer requirements and (even) identity not clearCustomer doesn’t know what they wantScope creep; Undocumented system scope and functionalityUser/buyer too distantDon’t understand the customer value systemManagement Related Input:Executive management doesn’t buy inLack of teamworkProgram Managers not empoweredProgram manager and capture managers are differentUnstable funding stream; Lack of upper management supportOrganizational/Cultural Input (Some Perceptions):SEA only adds to the Project CostSEA often seen as an “outside” team or “project reviewer” roleWe would like you to build us a lawn mower please!14
15 Theory versus (Virtual) Reality… Primary Reasons for Dysfunctional Behavior – My Opinion Confusion between “What you NEED” versus “What you WANT”Also called Gold-PlatingIt is the moral duty of a systems engineer to articulate the resulting cost and schedule deltaConfusion with regard to the SYSTEM BOUNDARYThis is more difficult for legacy systems with undocumented and implied interfaces; and even more so for “network-centric systems” and “SoS”Confusion (?) with regard to fidelity between the technical project scope and its allocated budget and scheduleThe result is cynicism and complacency, along with other negative behavioral patternsLack of Leadership
16 Leadership at the Individual Level… Leadership and FocusHonestyOpenness – Lexicon (Key Issue)Domain Knowledge – This is the key ingredient missing in most systems engineers today, resulting in significant impact to the credibility of the disciplineAbility for Systems ThinkingA very strong mental model, in the face of:A diverse lexiconNeed for rough estimates, and to identify outliersChanging standards (are they really changing?)New fads (6-sigma, IPPD, Concurrent Engineering, Simultaneous Engineering)Changing life cycle models (incremental, spiral, V, etc.)16
17 Deploying Systems Engineering within a Commercial Global Leader: Some Results
18 Systems Engineering Has Been Applied to Both Internal and Commercial Accounts 200020011st qtr2nd qtr3rd qtr4th qtr1st qtr2nd qtr3rd qtr4th qtr1st project uses SE principlesSE organization introducedSE Reviews / Scorecards introducedDirective to use SE on projects >$1M‘Fundamentals of SE’ course introduced1st commercial account uses SEDirective to use SE on projects > $500KFormal SE dept created200220031st qtr2nd qtr3rd qtr4th qtr1st qtr2nd qtr3rd qtr4th qtrSE process integration - GS Method13 projects using SESE process integration - AMS MSSEI introduces CMMI 1.1‘SE Design’ Class introducedSE team grows to 3012 completed and 13 active projects using SESE deliverables templates provided17 completed and over 50 active projects using SEOver 230 trained in SE FundamentalsSE team grows to 14SE process updated for CMMI
19 Architecture/Component Systems Engineering Process defines deliverables and a series of Reviews (Part I)Need / OpportunityIdentificationConceptualSystem SpecificationComponentArchitectureDetailedDesignCustomerBaselineSystemBaselineArchitecture/ComponentBaselineDesignBaselineBusinessRequirementsReview (BRR)SystemsRequirementsReview (SRR)PreliminaryDesignReview (PDR)CriticalDesignReview CDR)SystemsReq’mentSpecsComponentReq’ment SpecsComponentRTVMBusinessRequire.Specs.SystemLevelArchitect.ComponentLevelArchitectureComponentDesignComponent Test PlanTestArchitectureRTVMCustomer ProvidedSystems Engineering ProvidedComponent Developer Provided
20 Systems Engineering Process defines deliverables and a series of Reviews (Part II) Detail DesignDevelopmentTest and ProductionSystem UpdateNew ProductionSystemDesignBaselineTestBaselineProductionBaselineCDRTestReadinessReview (TRR)ProductionReadinessReview (PRR)SystemTestStrategySystemTest Plan /Test CasesReleaseContentDeploymentPlanComp.DesignTest PlanSystem TestDataTestTraceabilityMatrix.Move toProd.PlanDataMigrationPlanCustomer ProvidedSystems Engineering ProvidedComponent Developer ProvidedSystem Test ProvidedService Delivery / Managed Ops Provided
21 Successful implementation of System Engineering needs… A “productized” process for efficient implementationGlobally consistent templates and processes,Uniform and consistent metrics and lexicon (part of the SE culture)Focus must be on the “necessary” and critical subset of the overall methodology and theory (Flexibility and Adaptability)Tailoring for time-to-market considerationsTailoring for schedule and resource considerationsRisk tolerance must be explicitly considered in the tailoring processImplementation must be organizationally supported and nurturedLinkage to strategic organizational goals is keyA well managed competency development program and a “community of practice”
22 ISM delivered 5% under budget and with higher quality in production The charts here are based on data collected from a recent study analyzing project defects by type and phase. Here ISM defects by phase is compared to 46 similarly sized projects not utilizing SE.Total defect counts for non-SE projects exhibited 53.4% of total project defects during the Test Phase of the project. On ISM defects were detected earlier in the project life-cycle. In fact 56% of ISM detects were detected in Plan Phase.The chart on the left illustrates the cost implications of early defect detection as found with ISM 2.0.In effect ISM 2.0 expended 2.4 times less than what would have normally been required for the non-SE oriented projects compared to in the study.
23 IGA Metrics show 8% cost avoidance when comparing SE&A projects to non-SE&A projects = 15% of $10.7MM Baseline= 7% of $10.7MM Baseline= 8% Cost Avoidance
24 Academic Perspective: Discipline and Domain Centric Systems Engineering Programs Discipline Centric Systems Engineering Programs: These are programs where the major is designated only as Systems EngineeringDomain Centric Systems Engineering Programs: These are programs where the major is designated as X and Systems Engineering; or Systems and X Engineering.In this case, the most common instances of “X” Engineering are:Industrial EngineeringManufacturing EngineeringElectrical EngineeringManagement EngineeringComputer EngineeringThe primary source of this data is: Fabrycky, W.J., “Systems Engineering Education in the United States”, Proceedings, Conference on Systems Integration (CSI), Stevens Institute of Technology, New Jersey, March 2003.
25 Academic Perspective: Universities with Discipline Centric Systems Engineering Programs - 30 Air Force Institute of Technology California State University Colorado School of Mines Cornell University George Mason University George Washington University Iowa State University Johns Hopkins University National Technological University Naval Postgraduate School Oakland University Polytechnic University - Farmingdale Portland State University Purdue University Rochester Institute of TechnologySouthern Methodist University Stevens Institute of Technology University of Alabama - Huntsville University of Arizona University of Idaho University of Illinois at Urbana-Champaign University of Maryland University of Massachusetts University of Minnesota University of Missouri - Rolla University of Pennsylvania University of Rhode Island University of Southern California University of Virginia VPI and State UniversityThe list in the primary reference contains 35 records. Four of these referred to Universities with only an Undergraduate Program in Systems Engineering, and one to a University with only a Doctoral Program in Systems Engineering.
26 Academic Perspective: Universities with Domain Centric Systems Engineering Programs - 38 Auburn UniversityBoston UniversityCalifornia State University - FullertonCase Western Reserve UniversityGeorgia TechMassachusetts Institute of TechnologyNew Jersey Institute of TechnologyNorth Carolina A and T UniversityNortheastern UniversityOhio State UniversityOhio UniversityPolytechnic UniversityPurdue UniversityRensselear Polytechnic UniversityRutgers, The State UniversitySan Jose State UniversityStanford UniversityTexas Tech UniversityUniversity of Alabama - HuntsvilleUniversity of ArizonaUniversity of Central FloridaUniversity of ConnecticutUniversity of FloridaUniversity of HoustonUniversity of IllinoisUniversity of MemphisUniversity of Michigan Ann ArborUniversity of Michigan-DearbornUniversity of MinnesotaUniversity of PittsburghUniversity of Rhode IslandUniversity of South FloridaUniversity of Southern CaliforniaUniversity of Southern ColoradoUniversity of St. ThomasVirginia TechWichita State UniversityYoungstown State University
27 Industry/Government Perspective: Systems Engineering Education and Training 30+38~15Definition of Relevant:Relevance of the curriculum – orientation to DoD projects and programsPortability and flexibility of the delivery format – distributed organizationsHybrid – credit and continuing educationBasis:The Boeing SE Education Program (50 > 15 > 6 > 2)A DoD Component SE Education Program (80 > 25 > 11 > 2)Available…Relevant…
28 Industry/Government Perspective: Systems Engineering Education and Training 30+38~15?Definition of Critical Mass:Number of Tenured or Tenure Track FacultyNumber of Faculty with DoD/Aerospace Relevant Project/Program ExperienceBench Strength…Why?Available…Relevant…Critical Mass…
29 Industry/Government Perspective: Systems Engineering Education and Training 30+38~15?Definition of Critical Mass:Number of Tenured or Tenure Track FacultyNumber of Faculty with DoD/Aerospace Relevant Project/Program ExperienceBench StrengthAvailable…Relevant…Critical Mass…It is my opinion that we do not have a critical mass in graduate SE education in the US…However, we do have a very mature base and recognition within industry and government to facilitate the building of this critical mass…
30 A Fundamental DilemmaWe’ve forgotten (and are continuing to forget) much of the systems engineering we once knew.Reversing this trend has become an imperative for both Government and commercial industry.and yet…What we once knew may be inadequate for solving the systems engineering problems we now face.30
31 Security in the Port of NY & NJ Ten threat scenarios were defined What do I mean…Security in the Port of NY & NJTen threat scenarios were definedEach is low-likelihood, high-consequenceWhat is the eleventh?The Paradox of Ownership and Boundaries…The Driver and the Driven – The System and the Organization…Top Down…