1. 1 Towards Total Security Quality Management (TSQM): Definition and Measurement WORK IN PROGRESS 8 September 2005 (Major changes to slides 43-47) MIT TEAM Yang Lee Stuart Madnick Michael Siegel Diane Strong Venkataramana Thummisi (Venkat) Richard Wang

2. 2 Academic Literature Overview of Project Comprehensive List of Aspects of Security Industry Literature Extended Enterprise Hypotheses Key Dimensions And Aspects Stakeholders And Roles Survey 1 and 2 Survey 3 Gap Analysis Instrument Key Findings Gap Analysis Interim Results Gap related hypotheses - Extensive Gap Analysis - Case Studies - Best Practices - Benchmarking - Metrics - Security Methodology - Security Maturity Model

3. 3 Brief Description of Surveys Survey 1 Open-ended: What does holistic Security mean to you? Survey 2 Semi-structured: What does holistic Security mean to you? Similar to Survey 1, but starts with 20 security aspects. Survey 3 13 semi-structured questions regarding Extended Enterprise security covering issues such as Security Return on Investment, Benefits of Security, and Extended Enterprise Security.

4. 4 Academic Literature Overview of Project Comprehensive List of Aspects of Security Industry Literature Extended Enterprise Hypotheses Key Dimensions And Aspects Stakeholders And Roles Survey 1 and 2 Survey 3 Gap Analysis Instrument Key Findings Gap Analysis Interim Results Gap related hypotheses - Extensive Gap Analysis - Case Studies - Best Practices - Benchmarking - Metrics - Security Methodology - Security Maturity Model

5. 5 Comprehensive List of Aspects of Security Ability to effectively use data acceptance inspection Access access control mechanism access level access list Access modes access period access port access type Accountability accreditation accreditation authority add-on security administrative security Alert handling Antivirus Asset classification & control assurance attack audit trail authenticate Authentication authenticator authorization automated information system (AIS) automated information system security automated security monitoring availability of data Availability of service back door backup plan Bell-La Padula model benign environment between-the-lines entry Brand equity “is tied to customer’s perception about security Breach of confidentiality Breach of Security (BOS) Breach of integrity (BOI) browsing Buffer overflow Business loss Cache overflow call back capability category certification closed security environment communications security (COMSEC) Company preparedness compartment compartmented security mode Competitive edge Compliance compromise compromising emanations computer abuse computer cryptography computer fraud computer security subsystem concealment system confidentiality configuration control configuration management confinement confinement channel confinement property Connection contamination contingency plan control zone controlled access controlled sharing Controls Cookies cost-risk analysis countermeasure covert channel covert storage channel covert timing channel Credibility Criteria crypto-algorithm Cryptosecurity Customer confidence Customer loss Customers system Customized access Data control Data encryption Data Encryption Standard (DES) Data reliability dedicated security mode default classification Degausser Products List Denial of Service

6. 6 Comprehensive List of Aspects of Security (continued) Descriptive Top-Level Specification Designated Approving Authority Detection Directory services disaster plan Discretionary Access Control domain Dynamic passwords Ease of use Eaves dropping Encryption Endorsed Tools List end-to-end encryption Evaluated Products List fail safe failure access False alarms False indicators False pages fault Fear factor fetch protection file protection file security Financial loss Firewall flaw hypothesis methodology formal access approval formal security policy model Formal Top-Level Specification Formal Top-Level Specification (FTLS) Fraud front-end security filter functional testing granularity Gypsy Verification Environment Hacking handshaking procedure 'humanizing' online presence Identity management Identity server Identity validation impersonating Incident handling information flow control Information protection Information System Security Information security policy development Officer Integrity interdiction Intrusion Invisible re-routing isolation least privilege loophole maintenance hook malicious logic Mandatory Access Control (MAC) multilevel secure multilevel security mode Multiple levels of authentication multiuser mode of operation Network administration Non-repudiation object reuse open security environment Operation modes Operations Security Organizational security overt channel overwrite procedure Password penetration signature periods processing permissions Personal information Phishing Physical damage piggyback Policy management Preferred Products List Prevention print suppression privileged instructions procedural security Protection information protection philosophy protection ring protocols pseudo-flaw Reaction Reliability Reputation Risk Gross Risk/Residual Risk Safety Scanners scavenging Secure access secure configuration management … AND MORE …

7. 7 Academic Literature Overview of Project Comprehensive List of Aspects of Security Industry Literature Extended Enterprise Hypotheses Key Dimensions And Aspects Stakeholders And Roles Survey 1 and 2 Survey 3 Gap Analysis Instrument Key Findings Gap Analysis Interim Results Gap related hypotheses - Extensive Gap Analysis - Case Studies - Best Practices - Benchmarking - Metrics - Security Methodology - Security Maturity Model

8. 8 Dimensions of Security Technology Resources For Security Financial Resources For Security Business Strategy For Security Policy & Procedures Security Culture AccessibilityConfidentiality Vulnerability

9. 9 Good Security Good Security provides Accessibility to data and networks to appropriate users while simultaneously protecting Confidentiality of data and minimizing Vulnerabilities to attacks and threats. Good Security Practice goes beyond technical IT solutions. It is driven by a Business Strategy with associated Security Policies and Procedures implemented in a Culture of Security. These are practices are supported by IT Resources and Financial Resources dedicated to Security.

10. 10 Academic Literature Overview of Project Comprehensive List of Aspects of Security Industry Literature Extended Enterprise Hypotheses Key Dimensions And Aspects Stakeholders And Roles Survey 1 and 2 Survey 3 Gap Analysis Instrument Key Findings Gap Analysis Interim Results Gap related hypotheses - Extensive Gap Analysis - Case Studies - Best Practices - Benchmarking - Metrics - Security Methodology - Security Maturity Model

11. 11 Stakeholders of Extended Enterprise Security Enterprise General Public Extended Enterprise Ring 1: Enterprise Ring 2: Extended Enterprise Ring 3: General Public

12. 12 Stakeholders & Roles Domain/Role Level/RankGeneral business IT OrganizationGeneral security/ physical security Partners (Extended Enterprise) Top exec CEO, CFO, …Top IT Mgt/CIOTop Security Mgt / CSO Line/middle manager Business unit manager IT non-security managers ------------------------- IT security manager Security managers Workers Business personnel IT non-security personnel ------------------------- IT security personnel Security personnel (e.g., guard)

13. 13 Academic Literature Overview of Project Comprehensive List of Aspects of Security Industry Literature Extended Enterprise Hypotheses Key Dimensions And Aspects Stakeholders And Roles Survey 1 and 2 Survey 3 Gap Analysis Instrument Key Findings Gap Analysis Interim Results Gap related hypotheses - Extensive Gap Analysis - Case Studies - Best Practices - Benchmarking - Metrics - Security Methodology - Security Maturity Model

14. 14 Top Level Hypotheses H1: Addressing security can have a positive Return on Investment (ROI) H2. Security Practice, if not careful, can lead to counter-productive impacts H3. Security is not just about technology but requires correct corporate policies and incentives H4. Public Policies and Regulations regarding Security are important H5. Security must be understood holistically in the context of the Extended Enterprise

15. 15 H1. Hypotheses on ROI H1: Addressing security can have a positive Return on Investment (ROI) H1.1: Security is not necessarily an added cost H1.2: Benefits of security can outweigh the cost H1.3: Resilient enterprises are able to seize opportunities from security problems H1.4: Security can be a competitive advantage H1.5: Security needs to be designed into organizational strategy and translated into organizational performance

16. 16 H2. Hypotheses on Productive & Counter-productive Impacts H2. Security Practice, if not careful, can lead to counter- productive impacts H2.1 Security must be designed into all products and processes. H2.2 Security should be a part of organizational process design. H2.3. Security is not just an IT function. H2.4. Security must be based on statistical and economic models for investment and risk. H2.5. Software and hardware products are often not secure. Security updates are not well managed. H2.6. Security can be perceived to be a burden to users.

17. 17 H2. Hypotheses on Corporate Policies & Incentives H2. Security is not just about technology but requires correct corporate policies and incentives H2.1. Managing security of distributed heterogeneous systems and technologies need to catch up with rapid use and advancement of this technology. H2.2. Public not well-trained on Security. H2.3. Lessons learned from security are often not shared. Confusion between security and secrecy.

18. 18 H2. Hypotheses on Public Policies & Regulations H2. Public Policies and Regulations regarding Security are important H2.1 Rules of law and regulations needs to catch up with global digital security. H2.2 Security crimes need clear and appropriate punishment.

19. 19 H5. Hypotheses on the Holistic Nature of Security in the Extended Enterprise H5. Security must be understood holistically in the context of the Extended Enterprise H5.1 Solving local security problems can lead to larger global security problems [“Stopping small fires may lead to big fires.”] H5.2 Security and Privacy are closely related with complex interdependencies H5.3 Security problems at disparate parts of the enterprise (or members of the extended enterprise) can ripple throughout the extended enterprise H5.4 Different stakeholders within the organization (and within partner organizations) can have different perceptions and requirements for security

20. 20 Academic Literature Overview of Project Comprehensive List of Aspects of Security Industry Literature Extended Enterprise Hypotheses Key Dimensions And Aspects Stakeholders And Roles Survey 1 and 2 Survey 3 Gap Analysis Instrument Key Findings Gap Analysis Interim Results Gap related hypotheses - Extensive Gap Analysis - Case Studies - Best Practices - Benchmarking - Metrics - Security Methodology - Security Maturity Model

21. 21 Purpose of Gap Analysis Purpose of Gap Analysis is to understand perceptions of Differences between factors such: (A) Security Status Assessment and Security Importance (B) views of diverse Security Stakeholders within Enterprise and across Extended Enterprise

22. 22 Purpose of Gap Analysis (cont.) Gaps represent Opportunities for Improvement within the Enterprise and across the Extended Enterprise (A) When Status is below the Needs, these represent Areas for Improvement (B) When Status among Stakeholders show differences, these represent areas for Investigating sources of the differences Gaps may represent misunderstandings Gaps may represent differences in local knowledge and needs

23. 23 Three Types of Gaps 1.Performance Gaps 2.Role Gaps 3.Inter-Enterprise Gaps For today, we will focus on Performance Gaps, insufficient data for analyzing Role and Enterprise Gaps at this time Issue: Gathering of enough data from same organization and partner data

24. 24 Performance Gap 1. Gaps between Security Assessment and Security Importance Example: High importance for Confidentiality vs. Low assessment status of Confidentiality

25. 25 Role Gap 2. Gaps among Enterprise Roles Example: Business Managers vs. IT managers Example: Business Executives vs. Technology Executives Example: Executives vs. Line Managers

26. 26 Enterprise Gap 3. Gaps between Enterprise and Extended Enterprise partners Example: Internal (IT or Line managers) vs. Suppliers Example: Internal (IT or Line managers) vs. Customers

27. 27 Gap Analysis Questionnaire 1.Questionnaire respondents are comprised of the diverse roles (IT, IT security, Users, Business managers, Executives, etc.) within the enterprise and across (suppliers, customers, collaborators, etc.) the extended enterprise. 2.Each respondent reports his/her view of actual assessment and importance of each aspect for both his/her organization and a partner organization.

28. 28 Gap Analysis Questionnaire (cont.) 3.Questions on the questionnaire cover the 8 constructs of security: Accessibility Vulnerability Confidentiality Financial resources for security Technology resources for security Business strategy for security Security policy and procedures Security culture 4. To ensure construct validity, 5 questions are included for each construct.

29. 29 Extended Enterprise Security Survey Form # 01-20-____________ Towards Total Security Quality Management (TSQM) MIT’s Extended Enterprise Security Survey Introduction The following survey is part of a research project at MIT to develop a holistic framework to study enterprise security within and between organizations. Your responses to the following survey will provide us valuable insight about extended enterprise security. The extended enterprise includes an organization and its suppliers, customers, partners, and competitors. Extended enterprise security is concerned with security both within and between these organizations. The survey should take you about 20 minutes to fill out. Note about confidentiality: Your responses to questionnaire items will not be revealed to your organization or to any other organization. Only aggregate results will be used in our analyses. If you would like to receive a copy of our research results, please provide your email address at the bottom of the survey. General Instructions 1. What does it mean by “assessment” and “importance”? The survey asks you to give your impression of the “assessment” and “importance” of various security issues. “Assessment,” means your view of how well your organization is doing on these issues. “Importance” means your view of how important this issue is to you. 2. There is no right or wrong answer to any question. We are asking for your view. You may not know exact details about your company’s security. We are not asking for these details, but asking for your views. Please give your best estimate. 3. What is “Partner Organization”? The survey also asks you to give your impressions of “assessment” and “importance” for ONE partner organization. This partner organization should be one of your suppliers, if feasible. Alternatively, please select a customer or a collaborator organization. 4. There is no right or wrong answer about a partner’s security. We are asking your views of the partner organization’s security, you do not need to know exact details. Please give your best estimate. If you have no knowledge at all of an aspect of your partner security, you may leave that question blank. Thank you MIT TSQM team

30. 30 Your Organization & Partner Extended Enterprise Security Survey Section 1: Your Organization Your Organization/Company Organization Name__________________________________________________________ Industry____________________________________________________________________ Approximate total number of employees in your entire organization: ________________ Your Job Title and Work Role ________________________________________________ ___________________________________________________________________________ Department/Division/Group___________________________________________________ In my organization, I am a: _____(1) Executive (CEO,CFO, VP etc.) _____(2) Functional or Line Manager _____(3) Professional (Consultant, Engineer, In-house Expert, etc.) _____(4) Other Organizational Member In my organization, I work in the area of: _____(1) Business Security Policy and Management _____(2) IT Security _____(2) IT but not in Security, _____(3) General/Physical Security, _____(4) Not in Security or in IT. Section 2: Your Partner Organization Pick one partner organization for answering these questions. The survey administrator may give you additional instructions about picking a partner origination. All answers about your partner organization should be about ONE specific organization. Your Partner Organization/Company Partner Organization’s Name (optional)__________________________________________ Partner’s Industry_____________________________________________________________ Approximate total number of employees in your partner organization: ________________ Your Partner Organization is your organization’s: _____(1) Supplier ____(2) Customer ____(3) Collaborator ____(4) Competitor Major Group/Division/Department you usually work with: _______________________________________________________________________

31. 31 Security Questions (40)

32. 32 Lots of Survey Data (more to come)

33. 33 Gap Analysis Procedures 1.Assess Construct Validity Compute Cronbach Alphas Check inter-item correlations Delete and revise questions as needed 2. Form Constructs Aggregate questionnaire items into constructs Check inter-construct correlations 3. Compute Gaps: Performance Gap, Role Gap, Enterprise Gap 4. Test Gaps for significance 5. Interpret the results 6.Further analysis of interesting results

34. 34 Gap Analysis Preliminary Findings Performance Gaps Explore: at item level (yet not construct level) - Data just now being received - Only very limited analysis so far - All Findings that follow are preliminary

35. 35 Gap Analysis Findings Accessibility Question 40 Gap = 1.40 Example data: 5.40 (Assessment) vs. 6.80 (Importance) Availability of data and network when needed

36. 36 Gap Analysis Findings Vulnerability Question 1 Gap = 1.20 Example data: 4.60 (Assessment) vs. 5.80 (Importance) Tampering with data and networks is rare.

37. 37 Gap Analysis Findings Confidentiality Question 24 Gap = 1.20 Example data: 5.40 (Assessment) vs. 6.60 (Importance) Protects privacy of personal data.

38. 38 Gap Analysis Findings Financial Resource for Security Question 2 Gap = 1.17 Example data: 5.50 (Assessment) vs. 6.67 (Importance) Security is adequately funded.

39. 39 Gap Analysis Findings IT Resource for Security Question 5 Gap = 2.00 Question 17 Gap = 1.33 Example data: 4.33 (Assessment) vs. 6.33 (Importance) 5.00 (Assessment) vs. 6.33 (Importance) Business managers are involved with IT security policies. Adequate technology for supporting security.

40. 40 Gap Analysis Findings Business Strategy for Security Question 4 Gap = 1.50 Question 19 Gap = 2.00 Example data: 5.00 (Assessment) vs. 6.50 (Importance) 4.00 (Assessment) vs. 6.00 (Importance) Security strategy sets directions for security practices. Well-defined and communicated security strategy.

41. 41 Gap Analysis Findings Policy and Procedures for Security Question 25 Gap = 0.60 Question 30 Gap = 1.00 Example data: 5.20 (Assessment) vs. 5.80 (Importance) 5.20 (Assessment) vs. 6.20 (Importance) Adequate procedures for physical security. Procedures for detecting and punishing security violations.

42. 42 Gap Analysis Findings Security Culture Question 11 Gap = 1.83 (3.67 vs. 5.50) Question 18 Gap = 1.83 (3.83 vs. 5.67) Question 26 Gap = 1.60 (4.40 vs. 6.00) Question 39 Gap = 2.40 (4.20 vs. 6.60) People are knowledgeable about IT security tools and practices. People carefully follow good security practices. People can be trusted not to tamper with data and networks. People are aware of good security practices.

43. 43 Recent Activities (since last meeting) Developed web-based survey instrument Developed secure (https) web-based survey instrument Collected more data –About triple –Considerable “partner” company data –Both “miscellaneous” and two companies Valuable for intra-company stakeholder gap analyses Preliminary analysis of increased pilot data –Some sample analysis follows …

44. 44 Gap Analysis Findings - Updated Security Culture Question 39: People are aware of good security practices. Gap between Assessment and Importance – for your company Complete = 1.83 (4.71 vs. 6.54) Miscellaneous* = 2.40 (4.20 vs. 6.60) Company C= 1.83 (5.00 vs. 6.83) Company W= 1.89 (4.61 vs. 6.50) * Original sample: –diverse array of companies –many middle-managers

45. 45 Gap Analysis Findings - Updated Security Culture Question 39: People are aware of good security practices. Gap between Assessment and Importance – for your company Complete = 1.83 (4.71 vs. 6.54) Company C= 1.83 (5.00 vs. 6.83) Gap between Assessment and Importance – for partner company Complete = 0.99 (4.82 vs. 5.81) Company C= 1.40 (4.80 vs. 6.20)

46. 46 Next steps: Phase 2 (near-term) Collect much more data – especially for intra-company stakeholder analysis –Complete Round 1 Survey (within Sponsor) –Conduct Round 2 Survey (4-6 Sponsor partners) Perform construct analysis Analysis of pilot data Refine stakeholder and dimensions Refine questionnaire items Revise gap analysis instrument

47. 47 Next steps: Phase 3 (longer-term) Large-scale Gap Analysis Study Extensive Gap Analysis Results Pursue other hypotheses, through –Other Survey Instruments –Case Studies –Best Practices –Benchmarking –Security Methodology –Security Maturity Model

48. 48