1. Towards Total Security Quality Management (TSQM): Enterprise Perception Measurement and the “House of Security” May 15, 2006 Professor Stuart Madnick, Dr. Michael Siegel {smadnick, msiegel}@mit.edu

2. Copyright © 2006, MIT 2 FACULTY Yang Lee Stuart Madnick Michael Siegel Diane Strong Richard Wang Chrisy Yao STUDENTS Wee Horng Ang Dinsha Mistree Venkataramana Thummisi MIT TEAM

3. Copyright © 2006, MIT 3 Dimensions of Security Technology Resources For Security Financial Resources For Security Business Strategy For Security Policy & Procedures Security Culture AccessibilityConfidentiality Vulnerability “House of Security”

4. Copyright © 2006, MIT 4 Good Security Good Security provides Accessibility to data and networks to appropriate users while simultaneously protecting Confidentiality of data and minimizing Vulnerabilities to attacks and threats. Good Security Practice goes beyond technical IT solutions. It is driven by a Business Strategy with associated Security Policies and Procedures implemented in a Culture of Security. These are practices are supported by IT Resources and Financial Resources dedicated to Security.

5. Copyright © 2006, MIT 5 Stakeholders: Ranks & Roles Domain/Role Level/RankGeneral business IT OrganizationGeneral security/ physical security Partners (Extended Enterprise) Top exec CEO, CFO, …Top IT Mgt/CIOTop Security Mgt / CSO Line/middle manager Business unit manager IT non-security managers ------------------------- IT security manager Security managers Workers Business personnel IT non-security personnel ------------------------- IT security personnel Security personnel (e.g., guard)

6. Copyright © 2006, MIT 6 Differing Perceptions Picture of old lady or young lady ? Perceptions are as important as “reality”

7. Copyright © 2006, MIT 7 Purpose of Gap Analysis Purpose of Gap Analysis is to understand Differences in Perceptions between factors such: (A) Security Status Assessment and Security Importance (B) views of diverse Security Stakeholders …within the Enterprise and across the Extended Enterprise Types of Gaps (examples) Performance Gaps: Current Status v. Importance Role Gaps: Business Managers v. IT staff Inter-Enterprise Gaps: Internal Manager v. Supplier

8. Copyright © 2006, MIT 8 Purpose of Gap Analysis (cont.) Gaps represent Opportunities for Improvement within the Enterprise and across the Extended Enterprise (A) When Status is below the Needs, these represent Areas for Improvement (B) When Status among Stakeholders show differences, these represent areas for Investigating sources of the differences Gaps may represent misunderstandings Gaps may represent differences in local knowledge and needs

9. Copyright © 2006, MIT 9 Gap Analysis Questionnaire 1.Questionnaire respondents are comprised of the diverse roles (IT, IT security, Users, Business managers, Executives, etc.) within the enterprise and across (suppliers, customers, collaborators, etc.) the extended enterprise. 2. Each respondent reports his/her view of actual assessment and importance of each aspect for both his/her organization and a partner organization.

10. Copyright © 2006, MIT 10 Gap Analysis Questionnaire (cont.) 3.Questions on the questionnaire cover the 8 constructs of security: Accessibility Vulnerability Confidentiality Financial resources for security Technology resources for security Business strategy for security Security policy and procedures Security culture 4. To ensure construct validity, (approx) 4 questions are included for each construct.

11. Copyright © 2006, MIT 11 Extended Enterprise Security Survey Form # 01-23 Towards Total Security Quality Management (TSQM) MIT’s Extended Enterprise Security Survey Introduction The following survey is part of a research project at MIT to develop a holistic framework to study enterprise security within and between organizations. Your responses to the following survey will provide us valuable insight about extended enterprise security. The extended enterprise includes an organization and its suppliers, customers, partners, and competitors. Extended enterprise security is concerned with security both within and between these organizations. The survey should take you about 20 minutes to fill out. Note about confidentiality: Your responses to questionnaire items will not be revealed to your organization or to any other organization. Only aggregate results will be used in our analyses. If you would like to receive a copy of our research results, please provide your email address at the bottom of the survey. General Instructions 1. What does it mean by “assessment” and “importance”? The survey asks you to give your impression of the “assessment” and “importance” of various security issues. “Assessment,” means your view of how well your organization is doing on these issues. “Importance” means your view of how important this issue is to you. 2. There is no right or wrong answer to any question. We are asking for your view. You may not know exact details about your company’s security. We are not asking for these details, but asking for your views. Please give your best estimate. 3. What is “Partner Organization”? The survey also asks you to give your impressions of “assessment” and “importance” for ONE partner organization. This partner organization should be one of your suppliers, if feasible. Alternatively, please select a customer or a collaborator organization. 4. There is no right or wrong answer about a partner’s security. We are asking your views of the partner organization’s security, you do not need to know exact details. Please give your best estimate. If you have no knowledge at all of an aspect of your partner security, you may leave that question blank. Thank you, MIT TSQM team

12. Copyright © 2006, MIT 12 Your Organization & Partner Extended Enterprise Security Survey Section 1: Your Organization Your Organization/Company Organization Name__________________________________________________________ Industry____________________________________________________________________ Approximate total number of employees in your entire organization: ________________ Your Job Title and Work Role ________________________________________________ ___________________________________________________________________________ Department/Division/Group___________________________________________________ In my organization, I am a: _____(1) Executive (CEO,CFO, VP etc.) _____(2) Functional or Line Manager _____(3) Professional (Consultant, Engineer, In-house Expert, etc.) _____(4) Other Organizational Member In my organization, I work in the area of: _____(1) Business Security Policy and Management _____(2) IT Security _____(2) IT but not in Security, _____(3) General/Physical Security, _____(4) Not in Security or in IT. Section 2: Your Partner Organization Pick one partner organization for answering these questions. The survey administrator may give you additional instructions about picking a partner origination. All answers about your partner organization should be about ONE specific organization. Your Partner Organization/Company Partner Organization’s Name (optional)__________________________________________ Partner’s Industry_____________________________________________________________ Approximate total number of employees in your partner organization: ________________ Your Partner Organization is your organization’s: _____(1) Supplier ____(2) Customer ____(3) Collaborator ____(4) Competitor Major Group/Division/Department you usually work with: _______________________________________________________________________

13. Copyright © 2006, MIT 13 Security Questions (34)

14. Copyright © 2006, MIT 14 Evaluating Statistical Significance Significant at 99.99% level 28 Significant at 99% level 11 Significant at 95% level 0 Significant at 90% level 1 Less than 90%0 Total40 Gap significance notation: *** Significant at the 99.99% level; ** Significant at the 99% level; * Significant at the 95% level; ~ Significant at the 90% level. MA vs MI Gaps:

15. Copyright © 2006, MIT 15 Gap Analysis Findings – Different Organizations Gap between Assessment and Importance – for your company Overall = 1.28 (5.04 vs. 6.32) Miscellaneous 1 = 2.40 (4.20 vs. 6.60) Company X 2 = 1.83 (5.00 vs. 6.83) Company W 2 = 1.89 (4.61 vs. 6.50) Company I 3 = 0.44 (5.33 vs. 5.78) 2 High-tech organizations 3 Non-USA company Question 39: People are aware of good security practices. 1 Original pilot sample: diverse array of companies many middle-managers MA Gap MI

16. Copyright © 2006, MIT 16 Gap Analysis Findings – Compared with Partner Organization Gap between Assessment and Importance – for your company Overall = 1.28 (5.04 vs. 6.32) Gap between Assessment and Importance – for partner company Overall = 0.70 (5.25 vs. 5.95) Question 39: People are aware of good security practices. General conclusion: - View partner as “better” in assessment - But it is also “less important” -> So Gap is much less But not exactly true for all organizations … MA Gap MI PA PI Gap

17. Copyright © 2006, MIT 17 Gap Analysis Findings – Different Roles/Areas Your OrganizationPartner Organization Question 39: People are aware of good security practices. Some observations: Not huge difference in gaps for “your organization” - More significant gaps in views of partner organization IT Security people perceive much less “gap” in partner - And much lower “importance” for partner

18. Copyright © 2006, MIT 18 Dimensions of Security Technology Resources For Security Financial Resources For Security Business Strategy For Security Policy & Procedures Security Culture AccessibilityConfidentiality Vulnerability “House of Security”

19. Copyright © 2006, MIT 19 Average Construct Values

20. Copyright © 2006, MIT 20 Construct Gaps: Average Values

21. Copyright © 2006, MIT 21 Construct Gaps: Absolute Values

22. Copyright © 2006, MIT 22 Some Preliminary Insights 1Highest assessments and gaps in Accessibility indicates that construct has been major focus … and success. 2Low assessment in Security Culture, indicates that security management have yet to mature to the same level of security awareness and depth. 3High standard deviations in Security Policy indicates there is a disparity between the various companies/ industries. 4The large MI-MA gap, and PI -PA gap in Security Culture, shows companies are beginning to understand the need to achieve further improvement, highlighting an important area of potential growth. 5Partners assessment lower than self assessment indicates the aura of "invincibility" -- each company believes that it is safer than its partners. [Of course, everyone is someone else’s partner.] 6Partners importance of security lower than self security also reiterates the point that each company believes it rate these qualities more importantly on its agenda than would its partners.

23. Copyright © 2006, MIT 23 Company Assessment: Values

24. Copyright © 2006, MIT 24 Company Assessment: Gaps

25. Copyright © 2006, MIT 25 Role Assessment: Values

26. Copyright © 2006, MIT 26 Role Assessment: Gaps

27. Copyright © 2006, MIT 27 Area Assessment: Values

28. Copyright © 2006, MIT 28 Area Assessment: Gaps

29. Copyright © 2006, MIT 29 Next steps: Larger-scale Gap Analysis Study –Continue to seek more individual participants and company-wide participation Understand reasons for differences Understand implications of the differences -> Prescriptive actions –Need for more education –Need for more security –Need for more appropriate security –Etc … Others

30. Copyright © 2006, MIT 30 What is “Good Security?” It can be a matter of opinion (perception) Stuart Madnick; T 617-253-6671; E-mail: smadnick@mit.edusmadnick@mit.edu Exec Summary: http://ebusiness.mit.edu/research/Briefs/Madnick_Siegel_Security_Brief.pdfhttp://ebusiness.mit.edu/research/Briefs/Madnick_Siegel_Security_Brief.pdf I-SEE Overview: http://web.mit.edu/smadnick/www/Projects/I-SEE%20CeB.pdfhttp://web.mit.edu/smadnick/www/Projects/I-SEE%20CeB.pdf TSQM Survey: http://web.mit.edu/surveys/tsqm/http://web.mit.edu/surveys/tsqm/