Presentation is loading. Please wait.

Presentation is loading. Please wait.

INCOSE International Symposium 2008

Similar presentations


Presentation on theme: "INCOSE International Symposium 2008"— Presentation transcript:

1 INCOSE International Symposium 2008
Factors Related to the Implementation of ISSE: A Quality Perspective Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAP Doctoral Candidate, Information Assurance University of Fairfax Vienna, VA (USA) Director, IS Risk Management The Children’s Hospital of Philadelphia Philadelphia, PA (USA)

2 ISSE in the Acquisition of Secure IT
Introduction Background Research Proposal Research Review and Synthesis Preliminary Conclusions Preliminary Recommendations

3 Introduction Technical Risk Impacts the Corporate “Bottom Line”
Individual Companies Estimate Annual Losses of up to $5M (Deloitte, 2006) Loss of Mobile Devices Identity Theft Considered “Number Two Hot-Button” and “Crime of the 21st Century” Viruses/Worms Phishing/Pharming Spyware/Malware Increasing Cost of “Compliance” Sarbanes-Oxley (SOX) Federal Information Security Act (FISMA) Health Insurance Portability and Accountability Act (HIPAA) Payment Card Industry (PCI)

4 Introduction Technical Risk Generally Attributed to System Design, Development, and/or Configuration System Design & Development Incorrect System Inadequate Understanding of User Needs System Specification Doesn’t Reflect User Needs System Incorrect System Not Designed to Specifications System Not Built to the Design System Configuration Unnecessary Services Incorrect Settings Inadequate Patching

5 Introduction Ensures Requirements are Addressed Early in (and throughout) the System Development Life Cycle (SDLC) INCOSE SE Hdbk ver. 2a

6 Introduction Return on Investment (ROI) Related to Stage “Defect” is Identified and Subsequently Corrected INCOSE SE Hdbk ver. 3.1

7 Background Federal Law National Policy DoD Policy, Regulations
Service Policy, Regulations

8 Background DoD Rules and Regulations
CJCSI D, IA & Computer Network Defense (CND) Policy & guidance for IA and CND IA architecture C&A MAC levels Defense-in-Depth “Incorporate … IA … into [IS] during all phases of [the] system design life cycle including analysis, design, development, test and operation …”

9 Background DoD Rules and Regulations (Cont)
CJCSM , Defense-in-Depth: IA and CND Guidance & procedures for implementing Defense-in-Depth strategy and standards Defense-in-Depth elements:

10 Background DoD Rules and Regulations (Cont)
CJCSM D, Interoperability and Supportability of IT and National Security Systems NR-KPP mandatory IA compliance one of 4 NR-KPP requirements Specifies use of Information Systems Security Engineering (ISSE) IA component of the GIG Throughout system lifecycle

11 Background DoD Rules and Regulations (Cont)
DoDD , Data Sharing in a Net-Centric DoD “… data assets shall have associated [IA] and security metadata, and an authoritative source for the data …” DoDD , IA in the Defense Acquisition System Describes required & recommended levels of IA activities related to acquisition Describes the IA strategy Prescribes an IA strategy submission and review process Captures the acquisition-related IA policies of and

12 Background

13 Background DoD Rules and Regulations
DoDI , Information Assurance Establishes IA policy and assigns responsibilities Requires a defense-in-depth approach to IA DoDI , IA Implementation Prescribes DoD IA Requirements via Minimum Baseline IA Controls Additional Controls Specified in Related Documentation

14 Background DoD Implementation
Controls-based C&A Since 2003 (DoDI ) Security Technical Implementation Guides (STIGs) predate DoDI PMs Required to Perform Information Systems Security Engineering (ISSE) (DoDD , DoDI , DoDI ) Ensure Security is Incorporated into Overall Design Support Successful Certification of System Security Design and Configuration

15 Background Continuing Perception Security is Not Adequately Addressed in System Design/Configuration General increase in number and types of exploitable vulnerabilities (Deloitte, 2006) Excessive Technical Risk Often Identified During Security/Certification Testing (Cline, 2007) Need to Incorporate Security in Development Practices (Mouratidis, 2007; Davis 2004)—Limited Practice, Limited Research (Siponen, 2007; Woon, 2007) Problems/Issues with Specification/Incorporation of Security Requirements (Mayer, 2005; Mead, 2005; Wilander, 2005)

16 Background Impact Program Managers: Scope, Cost, & Schedule
Certifiers: Cost & Schedule DAAs: Cost, Schedule, Acceptance of Excessive Residual Risk Users: Scope, Cost, Schedule, & Acceptance of Excessive Residual Risk

17 Research Proposal Problem Statement Opportunity for Research
Research Objective

18 Problem Statement Barriers to the successful implementation of ISSE in DoD acquisition organizations result in IS/IT with excessive residual security risk—as determined by the DAA based on the certifiers recommendation—that potentially impacts: Project Scope (Requirements / Capabilities) Project and/or System Cost Project Schedule Technical Security Risk

19 Opportunity for Research
NSA Responsibility Provides: Technical guidance for protecting information and infrastructures Prescribes a Defense-in-Depth approach Defines a process for system development … Information Assurance Technical Framework (IATF)

20 Opportunity for Research
Two Principal Disciplines in System Development* (Cline, 2007) Program Management Systems Engineering Intersection forms Technical Management aka Engineering Management aka Systems Engineering Management *in addition to specialty engineering Systems Engineering Technical Management Program Management

21 Opportunity for Research
Two Principal Disciplines in System Security Risk Management (Cline, 2007) Information Assurance IT/IS Compliance Intersection forms Certification & Accreditation Certification & Accreditation Information Assurance IT/IS Compliance

22 Opportunity for Research
Information Systems Security Engineering (ISSE) … Systems Development meets Security Risk Mgmt Four Domains in the “ISSE Rose” (Cline, 2007) Security Engineering Technical Management C&A Rules & Regulations Technical Management Systems Engineering Program Management Information Systems Security Engineering Security Engineering Rules & Regulations Information Assurance IT/IS Compliance Certification & Accreditation

23 Opportunity for Research
Information Systems Security Engineering (ISSE) System development process described by the IATF Tightly integrated into the systems engineering model 1. Discover Information Protection Needs 2. Define System Security Requirements 3. Design Architecture 4. Develop Detailed Design User Involvement 6. Assess Information Protection Effectiveness 5. Implement

24 Opportunity for Research
SE Activities ISSE Activities Discover Needs The systems engineer helps the customer understand and document the information management needs that support the business or mission. Statements about information needs may be captured in an information management model (IMM). Discover Information Protection Needs The ISSE helps the customer understand the information protection needs that support the mission or business. Statements about information protection needs may be captured in an Information Protection Policy (IPP). Mission/Business Function Information Management Functions

25 Opportunity for Research
SE Activities ISSE Activities Define System Requirements The systems engineer allocates identified needs to systems. A system context is developed to identify the system environment and to show the allocation of system functions to that environment. A preliminary system Concept of Operations (CONOPS) is written to describe operational aspects of the candidate system (or systems). Baseline requirements are established. Define System Security Requirements The ISSE allocates information protection needs to systems. A system security context, a preliminary system security CONOPS, and baseline security requirements are developed.

26 Opportunity for Research
SE Activities ISSE Activities Design System Architecture The systems engineer performs functional analysis and allocation by analyzing candidate architectures, allocating requirements, and selecting mechanisms. The systems engineer identifies components or elements, allocates functions to those elements, and describes the relationships between the elements. Design System Security Architecture The ISSE works with the systems engineer in the areas of functional analysis and allocation by analyzing candidate architectures, allocating security services, and selecting security mechanisms. The ISSE identifies components or elements, allocates security functions to those elements, and describes the relationships between the elements.

27 Opportunity for Research
SE Activities ISSE Activities Develop Detailed Design The systems engineer analyzes design constraints, analyzes trade-offs, does detailed system design, and considers life-cycle support. The systems engineer traces all of the system requirements to the elements until all are addressed. The final detailed design results in component and interface specifications that provide sufficient information for acquisition when the system is implemented. Develop Detailed Security Design The ISSE analyzes design constraints, analyzes trade-offs, does detailed system and security design, and considers life-cycle support. The ISSE traces all of the system security requirements to the elements until all are addressed. The final detailed security design results in component and interface specifications that provide sufficient information for acquisition when the system is implemented.

28 Opportunity for Research
SE Activities ISSE Activities Implement System The systems engineer moves the system from specifications to the tangible. The main activities are acquisition, integration, configuration, testing, documentation, and training. Components are tested and evaluated to ensure that they meet the specifications. After successful testing, the individual components—hardware, software, and firmware—are integrated, properly configured, and tested as a system. Implement System Security The ISSE participates in a multidisciplinary examination of all system issues and provides inputs to C&A process activities, such as verification that the system as implemented protects against the threats identified in the original threat assessment; tracking of information protection assurance mechanisms related to system implementation and testing practices; and providing inputs to system life-cycle support plans, operational procedures, and maintenance training materials.

29 Opportunity for Research
SE Activities ISSE Activities Assess Effectiveness The results of each activity are evaluated to ensure that the system will meet the users’ needs by performing the required functions to the required quality standard in the intended environment. The systems engineer examines how well the system meets the needs of the mission. Assess Information Protection Effectiveness The ISSE focuses on the effectiveness of the information protection—whether the system can provide the confidentiality, integrity, availability, authentication and non-repudiation for the information it is processing that is required for mission success. 1. Discover Information Protection Needs 2. Define System Security Requirements 3. Design Architecture 4. Develop Detailed Design User Involvement 6. Assess Information Protection Effectiveness 5. Implement

30 Opportunity for Research
ISSE Activity Assess Information Protection Effectiveness Task Discover Information Protection Needs Present an overview of the process. Summarize the information model Describe threats to the mission or business through information attacks Establish security services to counter those threats and identify their relative importance to the customer. Obtain customer agreement on the conclusions of this activity as a basis for determining system security effectiveness. Define System Security Requirements Ensure that the selected solution set meets the mission or business security needs. Coordinate the system boundaries. Present security context, security CONOPS, and system security requirements to the customer and gain their concurrence. Ensure that the projected security risks are acceptable to the customer. Design System Security Architecture Begin the formal risk analysis process to ensure that the selected security mechanisms provide the required security services and to explain to the customer how the security architecture meets the security requirements. Develop Detailed Security Design Review how well the selected security services and mechanisms counter the threats by performing an interdependency analysis to compare desired to effective security service strengths. Once completed, the risk assessment results, particularly any mitigation needs and residual risk, will be documented and shared with the customer to obtain their concurrence. Implement System Security The risk analysis will be conducted/updated. Strategies will be developed for the mitigation of identified risks Identify possible mission impacts and advise the customer and the customer’s Certifiers and Accreditors.

31 Opportunity for Research
ISSE and C&A

32 Opportunity for Research
Little in the Formal Literature on ISSE Implementation General Failure to Implement ISSE Davis (2004), Mouratidis (2007) Literature Focused on Methods, Processes, Tools Application of Quality Construct to ISSE Security is a Non-Functional or Quality Requirement SSE-CMM is ISSE Maturity (Quality) Model Further Indicates Applicability of Quality Concepts Indicates ISSE both Technical & Mgmt Innovation Allows Synthesis of Process & Factor Models / Frameworks (Adoption Literature)

33 Opportunity for Research
Construct Proposed by Sebastianelli & Tamimi (2004) Barriers Based on Factor Analysis: Human Resources Development & Mgmt, Planning, Leadership, Resources, Customer Focus Difficult to Interpret from Survey Items Construct modified by Cline (2007) Factors Based on a Synthesis of the Literature (Hill, 2006) Training, Planning, Leadership, Resources, Culture Factors Formally Defined

34 Research Objective The research seeks to determine if the implementation of ISSE in support of acquired DoD IS/IT encounters the same impediments or barriers to implementation as traditional quality programs.

35 Research Review and Synthesis
Approach to the Review Empirical Research in Information Security and Assurance Empirical Research in Relevant Disciplines

36 Approach to the Review Purpose of Methodical Review
Understand Research Topic and Justify the Research Identify Research Problem and Gaps in the Literature Provides Suitable Methodologies Relevance to Research Problem is Key Discriminator “Each literature piece should constantly be evaluated on how applicable it is to the proposed study. Thus, only the applicable literature articles that are relevant to build the theoretical foundations for the validity of the theories, constructs, and measures should be noted.” (Levy & Ellis, 2006, p. 188) ISSE is “an engineering process … that captures and refines information protection requirements [SRE] and ensures their integration into IT acquisition processes [implementation] through purposeful security design and configuration [development]”

37 Non-Functional Requirements
Approach to the Review Primary Bodies of Knowledge Quality Management Hypothesized relationship Demonstrated relationship SRE Implementation Development (Design & Configuration) Capability Maturity Factors Non-Functional Requirements ISSE Adoption Theory

38 ISSE / Quality Implementation
Approach to the Review Based on IS/ISSE Research Models (Lowry et al., 2002; Bjorck, 2001) Rigorous Relevant Established Emergent ISSE Quality Management Development (Design & Configuration) SRE ISSE / Quality Implementation Adoption Theory

39 Approach to the Review InfoSec Research Model (Siponen & Oinas-Kukkonen, 2007) Levels of Abstraction Organizational, Conceptual, Technical Information Security Area / Topic / Contribution Access to IS Secure Communication Security Management Development

40 Approach to the Review Applicable Categories (Italicized Most Relevant to Research Problem) Access to IS / Conceptual (e.g., modeling of access control policy) Secure Communication / Conceptual (e.g., modeling in distributed systems) Security Management / Organizational (e.g., organizational ISSE implementation policies and guidelines) Development / Organizational (e.g., ISSE methods, processes)

41 Empirical Research in Information Security and Assurance
ISSE Methodology ISSE Implementation

42 ISSE Methodology Organizational-level Literature on Development of Secure IS/IT Landwehr (1981) Assurance-based Security Design (circa ) Development Using Formal Models (TCSEC, ITSEC, CC) Baskerville (1993) Checklist-based Security Design (circa ) Design Based on Available (Known) Controls Use of Risk in Cost-Benefit Analysis (Controls Selection) Mechanistic Engineering Security Design (circa ) Design Based on Requirements Continued Use of Risk in Cost-Benefit Analysis Formative Logical Security Design (circa ) Design Based on Abstract Models (Functional/Organizational) Limited or No Use of Risk in Cost-Benefit Analysis)

43 ISSE Methodology Weiss et al. (1996)
Discussed Integration of Engineering and Risk Management Straub & Welke (1998) Discussed Systems Risk in Organizational Risk Management Security Design a Component of Systems Risk Assessment Chivers (2004) Traditional Methods Ineffective for Distributed Systems Integrated Risk/Engineering Security Design Methods (circa ) Failure Mode Analysis, Fault Tree Analysis Increased Complexity Degrades Abstraction

44 ISSE Methodology Chivers (2004) (Continued)
Need to Integrate Security Analysis in the Systems Engineering Process Most Relevant Work in Requirements Engineering (circa ) Functional Requirements: Use Cases, Goals Non-Functional Requirements: Patterns, Abuse Cases & Misuse Cases, Anti-Goals Need for Integration of Functional and Non-Functional Requirements Object-Oriented Patterns; Goal-Based Refinement Most Promising Area is Goal-Based Refinement

45 ISSE Implementation Organizational-level Literature on Security Management (ISSE) Marmor-Squires & Rougeau (1988) (cited in Ashby et al.,1989) Early Paper Addressing Security Engineering in Systems Engineering Did Not Address ISSE Implementation Issues Froscher et al. (1990) Specific to C&A But Addressed Secure IS/IT Development Recommended Certification Activity Early in Development Recommended Specific Interactions: CA, DAA, PM, Developer Weiss et al. (1996) Discussed Integration of Engineering & Security Risk Mgmt

46 ISSE Implementation Peter & Schleipfer (2004)
Recommended Addressing Security Engineering Early in Development Did Not Address ISSE Implementation Issues Davis (2004) Advocated Security Engineering Throughout Development Lifecycle

47 ISSE Implementation Mouratidis (2007)
Security Engineering and IS Engineering Communities Work Separately Little Integration of Security and IS Engineering Principles/Practice Research in Security Engineering Primarily Technical Issues Little Consideration of Social Aspects / Social Context Recommended New Discipline: Secure Information Systems Engineering (Note: No Significant Literature on DoD ISSE) Did Not Address ISSE Implementation Issues

48 Empirical Research in Relevant Disciplines
Quality Management Models Quality Implementation

49 Quality Management Models
Seminal Authors (Hill, 2006) Crosby Deming Feigenbaum Ishikawa Juran Shewart Taguchi Quality Management Models (Hill, 2006) Quality Control Quality Trilogy Quality Assurance Quality Management TQM Six-Sigma CMM

50 Quality Implementation
Cline (2007) Sebastianelli & Tamimi (2003) Nwabueze (2001) 1 Deming (1987) 2 Amar & Zain (2002) 3 Juran (1993) 4 Williams et al. (2004) 5 Leonard & McAdam (2001) 6 Chin & Pun (2002) 7 Kwak & Anbari (2006) 8 Culture Lack of customer focus No focus on customer satisfaction 5 No understanding of cultural change 1 Culture or relationships between departments 3 Resistance to change 6 No long-term management (mgmt) commitment due to rewards based on short-term successes 5 Inappropriate organizational culture 7, 8 Incompatibility of attitudes between management and workers 1 Attitudes towards quality 3 Employee ‘buy-in’ 4 Insufficient cooperation 7

51 Quality Implementation
Cline (2007) Sebastianelli & Tamimi (2003) Nwabueze (2001) 1 Deming (1987) 2 Amar & Zain (2002) 3 Juran (1993) 4 Williams et al. (2004) 5 Leonard & McAdam (2001) 6 Chin & Pun (2002) 7 Kwak & Anbari (2006) 8 Leadership Inadequate human resources (HR) development and management HR issues 3 No focus on employee satisfaction, innovation, or quality systems 5 Failure of management to recognize employee importance 7 Lack of leadership for quality Lack of management commitment 1 Ineffective mgmt 1, 2 Lack of management commitment 3 Unstable management due to high turnover 3 Lack of long-term management commitment due to rewards based on short-term successes 5 Lack of management leadership 7 Lack of realism for what TQM can do 7 Planning Lack of quality planning Poor planning 1 Lack of goals 1 Implementation without regard to context 1 Use of improper framework 1 Divide between strategic & operational goals/planning 6 Lack of strategic planning 8

52 Quality Implementation
Cline (2007) Sebastianelli & Tamimi (2003) Nwabueze (2001) 1 Deming (1987) 2 Amar & Zain (2002) 3 Juran (1993) 4 Williams et al. (2004) 5 Leonard & McAdam (2001) 6 Chin & Pun (2002) 7 Kwak & Anbari (2006) 8 Resources Inadequate resources Lack of resources 1 Insufficient raw materials 3 Lack of machines & equipment 3 Training Lack of training 1 Inadequate training 3 Inadequate training 7 Training in leadership & project mgmt 8

53 Preliminary Conclusions
Limited Research on Implementation in the ISSE Literature (Mouratidis, 2007; Siponen, 2007) Applicability of quality concepts to ISSE (Cline, 2007) Non-Functional Requirements (Chung, 1995) Capability Maturity (SSE-CMM) (ISSEA, 2007) Factors Related to Quality Implementation (Hill, 2006) Factors Related to the Implementation of ISSE by DoD Acquisition Agencies are Similar to the Implementation of Traditional Quality Management Models (Cline, 2007) Culture Leadership Planning Resources Training

54 Preliminary Recommendations
In Addition to Mandating Use of ISSE in System Acquisition, the DoD Should Provide a Level-of-Effort Similar to the Implementation of Traditional Quality Programs Recommendations Specific to Each Barrier: Culture—Stress the Importance of ISSE through Policy; Provide Penalties for Non-Compliance and Enforce the Penalties Leadership—Ensure Leadership/Management Understand their Compliance Requirements and the Penalties for Non- Compliance

55 Preliminary Recommendations
Recommendations Specific to Each Barrier (cont): Planning— Formally Incorporate ISSE in Systems Engineering Plans, Processes, Procedures, Standards and Guidance Fully Integrate ISSE in All Engineering & Program Reviews Implement the SSE-CMM as part of CMMI in Acq Agencies Resources— Mandate ISSEs for Acquisition Programs Program/Budget Needed Tools (e.g., Mgmt and Assessment) Training Require ISSE Awareness Training for Systems Engineers and PMs Require Properly Trained and Certified ISSEs

56 Preliminary Recommendations
Senior Management Responsible for Fielding Secure IS/IT (e.g., DAA, CA, PM, & User Rep) Should: Receive Briefings on the Role of ISSE Apply Lessons Learned From Prior Quality Model Implementations to ISSE Be Tracked to Determine Rate & Level of Adoption Subsequent Impact on Residual Risk Technical Vulnerabilities Associated with System Development and Configuration

57 Recommended Reading DS Herrmann (2002). A practical guide to security engineering and information assurance. Boca Raton, FL: Auerbach Publications. 57

58 Questions … that haven’t already been asked?
Bryan S. Cline, CISSP-ISSEP, CISM, CISA, CPP, CAP University of Fairfax The Children’s Hospital of Philadelphia

59 Additional Information

60 Justification for Research
Answering the research questions will allow DoD acquisition agencies to: Address the most critical issues facing program and project managers Provide targeted application of scarce resources (dollars and personnel) Improve overall system security Reduce risk to project scope, cost and schedule

61 Theoretical Foundation
Extensive Literature on ISSE Methodology Little or No Relevant Literature on ISSE Implementation No Construct for Barriers to ISSE Implementation Applicability of Quality Concepts to ISSE Nonfunctional Requirements (Engineering) Capability Maturity Models (Management) Quality Construct – Barriers to Implementation Sebastianelli and Tamimi (2003) Exploratory Quantitative Research Correlational Survey Study Barriers Based on Interpretation of Optimal Factor Analysis Cline (2007) Barriers Based on Synthesis of Relevant Literature

62 Research Methodology Theoretical Framework Research Design Approach
Context of Study Feasibility Analysis and Design Selection

63 Theoretical Framework
Research Hypotheses Operational Definition of Variables Rival Hypotheses Plausibility of Rival Hypotheses

64 Research Questions Question 1: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s culture? Question 2: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s leadership? Question 3: Is there a relationship between the successful implementation of ISSE and the acquisition organization’s planning? Question 4: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s resources? Question 5: Is there a relationship between the successful implementation of ISSE and the DoD IS/IT acquisition organization’s training?

65 Research Hypotheses H1 H10 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s culture. H11 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s culture. H2 H20 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s leadership. H21 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s leadership. H3 H30 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s planning. H31 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s planning. H4 H40 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s resources. H41 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s resources. H5 H50 (Null): Implementation of ISSE is independent of the DoD IS/IT acquisition organization’s training. H51 (Alternative): Implementation of ISSE is dependent on the DoD IS/IT acquisition organization’s training.

66 Operational Definition of Variables
Independent Variable: ISSE implementation is defined as the implementation of ISSE within a DoD IS/IT acquisition organization, where ISSE is defined as “an engineering process … that captures and refines information protection requirements and ensures their integration into IT acquisition processes through purposeful security design and configuration” (DoD, 2006, p. 19). Dependent Variable 1: Culture is defined as “a set of shared assumptions, values, and behaviors that characterize the functioning of an organization” (Schwalbe, 2006, p. Glossary 8). Dependent Variable 2: Leadership is defined as the actions of an individual, usually in a formal position of authority in an organization (management), “who focuses on long-term goals and big-picture objectives, while inspiring people to reach those goals” (Schwalbe, 2006, p. Glossary 6).

67 Operational Definition of Variables
Dependent Variable 3: Planning is defined as the activities and processes used to devise and maintain a workable scheme to ensure organizational needs are met (adapted from Schwalbe, 2006, p. Glossary 8). Dependent Variable 4: Resources are defined as “skilled human resources (specific disciplines either individually or in crews or teams), equipment, services, supplies, commodities, materiel, budgets, or funds” (Program Management Institute, 2004, p. 372) but not the management of personnel as defined by leadership. Dependent Variable 5: Training is defined as “the level of learning required to adequately perform the responsibilities designated … and accomplish the mission” (DAU, 2005, p. B 170).

68 Rival Hypotheses Use of integrated commercial off-the-shelf (COTS) or non- developmental items (NDI) Burbank & Kasch (2004); Chung & Cooper (2002); Steves (1997) Use of a rapid acquisition process

69 Plausibility of Rival Hypotheses
Both type of development (NDI vice developmental) and type of acquisition (rapid vice traditional) will be accounted for and controlled as mediating variables.

70 Research Design Approach
Post-Positivistic with Hermeneutic (Constructivist) Elements Applied Explanatory (Confirmatory?) Quantitative Correlational (Predictive?) Survey

71 Context of Study Setting Population Limitations
Sample Design and Selection

72 Setting Product Group Twelve (PG-12), Marine Corps Systems Command
Multiple government and contractor facilities and workspaces

73 Population Theoretical Study Population
All military, government civilian and civilian contractor acquisition and engineering personnel supporting DoD IS/IT acquisition programs on behalf of a DoD acquisition agency Target Study Population Restricted to Marine Corps Systems Command

74 Limitations Facility Access Duty Days, Duty Hours
Formal Approval and/or Escort Required Personnel Access Not-to-Interfere Basis Information Restrictions on Use of Classified and SBU

75 Sample Design and Selection
Random Selection Infeasible Research Setting Restricted Small Target Study Population Non-Random Alternative Grounded Theory of Generalized Causal Inference (Shadish et al., 2002) Purposive Sample of a Typical Instance (Theoretical Study Population) Marine Corps Systems Command Product Group Twelve

76 Sample Design and Selection
Principles of Generalization (Framework for Validity Claims) Surface Similarity Categorization of Phenomena Based on Similar Characteristics (Homogeneity) Ruling Out Irrelevancies Ignoring Characteristics of Phenomena Not Relevant to Categorization (Heterogeneity) Making Discriminations Discarding Phenomena That Do Not Fit Categorization Interpolation and Extrapolation Generalizing Between or Beyond Sampled Values Causal Explanation Attributing Causation Based on Theorized Structural Similarity

77 Feasibility Analysis and Design Selection
Assumptions Limitations Delimitations Design Approaches Resources

78 Assumptions Additional technical risk discovered during IV&V is due to a failure of one or more aspects of the ISSE implementation Theoretical framework provided by identified barriers to quality implementations is applicable to ISSE Target study sample is assumed extensible to the target and theoretical study populations Perceptions of surveyed personnel are a valid indicator of the variables measured

79 Limitations Threats to Validity
External Validity of the Purposive Sample Addressed Though Five Principles of Generalization Construct Validity of Barriers to Implementation Addressed Through Analytical Comparison with Original Construct (Factor Definitions / Item Loadings) Internal Validity due to Construct Modifications Addressed Through Grounded Interpretation of Factors (Factor Definitions / Item Loadings) Statistical Conclusion Validity Addressed Through Use of Generally Accepted Statistical Procedures and Statistical Software Package Threats to Predictive Analysis Non-Experimental Design; Small Sample Size Addressed Through Full Disclosure of the Issues

80 Delimitations Research Restricted to: DoD IS/IT
DoD Acquisition Organizations

81 Design Approach Operational Constraints
Limited Invasiveness / Interference Suggests Non-Experimental Design Site Constraints Small Population Limited Sample Size Suggests Non-Random Selection Suggests Purposive Sample Foundational Constraints Partial Replication of Prior Study Suggests Exploratory or Confirmatory Research Requires Correlational Design Requires Survey Study (Approach to Data Collection)

82 Resources Limited Requirements
General Computing and Printing Resources Statistical Software Package Web Hosting (Survey Instrument) Direct (e.g., Books, Literature Sources) Indirect (e.g., Communications, Travel) All Costs Born by the Author

83 Research Design Specification
RDS Completed and Approved Study has IRB Approval; Permission to Collect Data Received Next Steps … Survey Instrument Completed / Hosted on SurveyMonkey Data Collection to Start as Early as First Week in June 2008

84 Backup Slides

85 Clinger-Cohen Act Clinger-Cohen Act (CCA) IA-related requirements:
IT Registration DoD IT Registry DoN IT Registration Database USMC IT Registration Database ?? IA Strategy Stand-alone document Written at program inception More comprehensive for higher levels of assurance Supplemental/supporting documents: ISP (Policies, Standards, & Architecture) SSAA (Certification & Accreditation)

86 National Policy, Regulations
National Security Directive 42 (NSD-42) Established the National Security Telecommunications and Information Security Committee (NSTISSC) Chaired by the DoD Provides security guidance for National Security Systems (NSS) Evaluates the status of security for NSS Note: NSTISSC is now the Committee on National Security Systems, CNSS)

87 FISMA Federal Information Security Management Act (FISMA)
Federal Agencies must: Inventory systems (IT Database) Identify & provide appropriate security protections (IA Controls / C&A) Institute an INFOSEC program NIST provides security standards & guidelines DoD provides own standards & guidance OMB oversees FISMA (and CCA) compliance Authorizes annual program reviews Non-compliance can result in severe delays or elimination of funding!

88 FISMA FISMA-compliance IA Controls C&A INFOSEC Program Reporting
DODI C&A DoDI & DoD M INFOSEC Program DoDD & DoDI Reporting Handled by MCSC IA Division

89 OMB Circular A-130 Office of Management and Budget (OMB) Circular A-130, Appendix III Provides security guidance and defines responsibilities Defines the use of “adequate security” “Commensurate with risk and magnitude of harm” Implements risk management vice risk avoidance

90 NSTISSP 11 National Security Telecommunications and Information Systems Security Policy (NSTISSP) No. 11 “National Policy Governing the Acquisition of IA and IA-Enabled IT” Replaced “Orange Book” Requires use of National Information Assurance Program (NIAP)-certified products Based on the Common Criteria (CC) Products evaluated against: Protection Profile (PP) Evaluation Assurance Level (EAL)

91 NSTISSI 1000 National Security Telecommunications and Information Systems Security Instruction (NSTISSI) No. 1000 “National IA C&A Program (NIACAP)” Standardizes C&A for Federal Agencies Implements C&A using FISMA guidelines Basis for revision of current DoD C&A process

92 Key Performance Parameters
KPPs establish requirements (including security) specific to a particular system Objective: Preferred capability Threshold: Minimum capability Failure to meet a threshold may result in a reevaluation, re-assessment or termination of the program, or a modification of the production increments Example: Time to restore full functionality into a degraded or corrupted system Objective: 5 minutes Threshold: 15 minutes

93 DoD R SE Process

94 IEEE Std SE Process

95 Relevant Terms Accreditation – Formal declaration by a DAA that an IS is approved to operate at an acceptable level of risk, based on the implementation of an approved set of technical, managerial, and procedural safeguards. Acquisition – The conceptualization, initiation, design, development, test, contracting, production, deployment, logistics support, modification, and disposal of systems to satisfy DoD needs intended for use in, or in support of, military missions. Authority to Operate (ATO) – The authorization granted by a DAA for a DoD IS to process, store, or transmit information. Certification – A comprehensive validation of actual IA capabilities and services of a DoD IS to establish compliance with assigned IA Controls.

96 Relevant Terms Certifier – Individual responsible for making a technical judgment of the system’s compliance with stated requirements. Certification Determination – A certifier’s validation of the systems compliance with IA controls, identifying and assessing the risks with operating the system, and the cost to correct or mitigate the IA security weakness. Designated Accrediting Authority (DAA) – Official with the authority to formally assume responsibility for operating a system at an acceptable level of risk. DoD Information System (IS) – Set of information resources organized for the collection, storage, processing, maintenance, use, sharing , dissemination, disposition, display, or transmission of information.

97 Relevant Terms Information Assurance Control – An objective IA condition of integrity, availability, or confidentiality achieved through the application of specific safeguards or through the regulation of specific activities. Independent Verification and Validation (IV&V) – An independent review … [of a system] performed by an organization that is technically, managerially, and financially independent of the development organization. Information Systems Security Engineering – An engineering process that captures and refines information protection requirements and ensures their integration into [IS] acquisition processes through purposeful security design and configuration. Residual Risk – Risk due to a partial or unsatisfactory implementation of assigned IA controls.

98 Relevant Terms Risk – Possibility that a particular threat will adversely impact an IS by exploiting a particular vulnerability. Technical Controls – Security controls (i.e., safeguards or countermeasures) for an IS that are primarily implemented and executed by the IS through mechanisms contained in the hardware, software, or firmware components supporting the system. Test & Evaluation – Process by which a system or components are exercised and results analyzed to provide performance-related information. Threat – Any circumstance or event with the potential to adversely impact an IS through unauthorized access, destruction, disclosure, modification of data, and/or denial of service.

99 Relevant Terms Validation – Activity applied throughout the system lifecycle, to confirm or establish by testing, evaluation, examination, investigation, or competent evidence that a DoD information system’s assigned IA Controls are implemented correctly and are effective in their application. Vulnerability – Weakness in an IS, system security procedures, internal controls, or implementation that could be exploited.


Download ppt "INCOSE International Symposium 2008"

Similar presentations


Ads by Google