Presentation is loading. Please wait.

Presentation is loading. Please wait.

Ian M. Harper, CISA, CISM, CISSP Chief Technology Officer Dealing with Your IT Process Failures Improving Security with Root Cause.

Similar presentations

Presentation on theme: "Ian M. Harper, CISA, CISM, CISSP Chief Technology Officer Dealing with Your IT Process Failures Improving Security with Root Cause."— Presentation transcript:

1 Ian M. Harper, CISA, CISM, CISSP Chief Technology Officer Dealing with Your IT Process Failures Improving Security with Root Cause Analysis Lisa Johnson, CPA, CISA, CISM, CITP

2 Agenda Control Environment Survey Risk Environment Survey Control Weaknesses as Critical Process Failure Root Cause Analysis Tools & Techniques General Application of Root Cause Analysis to Control Weaknesses Common Pitfalls War Stories

3 The Controls Equation Increased complexity of control drivers Increased complexity of systems Dramatically increased complexity of IA controls environment +

4 Explosion of IA Drivers General Business US Sarbanes-Oxley (SOX) EU 8 th Company Law Directive Japanese Financial Instruments and Exchange Law Australia’s Corporate Law Economic Reform Program Specific Market Regulations US Gramm-Leach-Bliley (GLB) EC Markets in Financial Services Directive (MiFID) US Health Insurance Portability and Accountability Act (HIPAA) Market Forces (the real driver) Protection of business, shareholders and other market players’ interests per best practices

5 Organizational Control Frameworks May be home-grown or based off of internationally recognized frameworks COSO, COBIT, etc. From the perspective of RCA, all implemented control frameworks are generally equally valid Unless the framework is the cause for the weakness!

6 The Role of Risk A necessary evil "If the primary aim of a captain were to preserve his ship, he would keep it in port forever." –Thomas Aquinas "Risk is what separates the good part of life from the tedium." –John “Johnny Zero” Foley But one that must be controlled “From others' slips some profit from one's self to gain.” –Publius Terentius Afer

7 Risk Lifecycle UNKNOWN RISK ACCEPTED RISK IDENTIFICATION OF RISK ConceptPlansComponents Initial System Operational System Initial Risk Assessment Risk Assessment Developmental ST&E Inspection & Acceptance Certification Monitoring & Periodic Assessment SYSTEM DEVELOPMENT

8 What Risks? Financial – risks that directly cost us money Strategic – risks that affect our ability to meet our various goals and objectives Operational – risks that introduce inefficiencies in achieving our goals Compliance – risks that may put us in jail or may cost us in fines Reputation – risks that undercut the trust in us held by others

9 Control Weakness Identification Risk Management Process Allow us to fix before the weakness is exploited Incident Pressure-cooker environment Personal and business credibility is at stake

10 Control Weaknesses Identification of presence of unacceptable risk Critical Process Failure (Key Point) In manufacturing: a defect Ultimately, a quality issue in the manufacture of controls

11 “Weakness” in Context Control Framework Controls Business desires to produce quality controls Continued commitment to manage risks WeaknessControl framework or controls have failed (execution or quality) ObjectDenotes

12 Control Weakness Observations Usually occur in batches Typically affect various areas of the organization Pareto Principle 80% of cost of control issues can be attributed to 20% of the total population of defective controls Key Point:These observations typically indicate systemic causes

13 Root Cause Analysis Enter Root Cause Analysis

14 Root Cause Analysis Definition: A structured approach to identify factors causing an undesirable event Goal is to prevent recurrence A toolbox of tools and techniques tailored to a case, not a rote process Each team/practitioner will use different techniques and tools Same team/practitioner may use different approaches on different cases

15 RCA Roots & Applicability Manufacturing Focus on quality production processes Causes related to workforce, materials, methods, and machinery IT Need to view control framework as an assembly line Have to shift focus to those aspects that deal with IT and with virtual objects Machinery is never the cause

16 RCA is Not New to IT Auditors, remember the Cause statement of standard findings? So why don’t we do this in practice? Blame game and rejection of findings by management Lack of resources and return on investment Environment that does not promote creativity

17 RCA Stages Information Gathering & Problem Identification Process Analysis and Problem Understanding Brainstorming Data Collection Root Cause Determination Root Cause Elimination

18 Information Gathering/ Problem Identification Goals: Gather information about the observed weakness Identify the actual problem Tools & Techniques: Timeline Used throughout the RCA effort Needs to identify pertinent data as a reference point

19 Information Gathering/ Problem Identification Tools & Techniques (continued): WWWWH What?equipment involved, what was wrong. Who?who took what action, may be unknown When?day, date, time of occurrences, SDLC stage, etc. Where?physical, logical locations involved How?rate of occurrence, scope of effect

20 Information Gathering/ Problem Identification Concerns: Integrity starts here Data needs to be attributable Data preservation is key here Identified problem must be accepted by the team and management Problem definition should not jump to conclusions (avoids “no” or “lack of” statements)

21 Process Analysis/ Problem Understanding Goals: Understand the basic function of applicable business processes and controls Understand the relationship of the problem to processes and controls Tools & Techniques: Document review Interview of personnel Need to focus on developing rapport and furthering openness

22 Process Analysis/ Problem Understanding Concerns: Continued need to preserve data Need to be resourceful in finding sources of information Trouble tickets, security plans, development artifacts, audit logs, helpdesk statistics, etc. Need to focus on developing rapport and furthering openness of the process Remember people develop personal worth from their work

23 Brainstorming Goals: Identify the full set of possible causes for the defined problem based on information obtained Prioritize possible causes for further review Tools & Techniques: Unstructured Brainstorm Session Effective in symbiotic teams All personnel provide possible causes while one member records ideas Trades risk of domination for more freedom

24 Brainstorming Tools & Techniques (continued): Structured Brainstorm Session Effective in stratified teams (personality or position) Round Robin – Each individual gets a chance to provide ideas in turn Secret Ballot – Ideas are turned in on chits secretly Trades uninhibited expression for fairness of process

25 Brainstorming Tools & Techniques (continued): Comparison Tables Used to resolve conflicts on prioritization Priority of each possible cause is voted on by each team member The cause with the most votes tallied is prioritized over the least number of votes Can only be used for small sets of data

26 Comparison Table Example –3 causes, 10 team members –Resulting Prioritization: A, C, B A A/B7 A/C6 B 3 C 4 B/C28 Total13512

27 Brainstorming Concerns: At this point do not attempt to limit factors based on feasibility or data collected to this point Prioritization is key to making sure resources are spent wisely Prioritization should focus on the best fit of the causes to the observed effect The team must be able to deal with strong personalities or perceived positions of power

28 Data Collection Goal: Obtain any additional information needed to document applicability of possible causes Tools & Techniques: Interview and Document Review Physical Inspection Concerns: This step may not be needed if all necessary data was collected in Information Gathering Corroborative information may be necessary to assess potential causes identified during Brainstorming

29 Determination of Root Causes Goal: Organize and analyze potential causes to identify the systemic root causes of an effect

30 Determination of Root Causes Tools & Techniques 5-Whys A technique used by Toyota Motor Corporation Based on the concept that by answering why 5 times, you get to the cause (may take less) Prone to rote memorization of rationalized causes, so need to ensure proper mindset Also limited to the set of knowledge of the questioner

31 Determination of Root Causes Tools & Techniques (continued) Pareto Charting Based on 80/20 rule Provides a visual way to distinguish the “vital few” from the “important many” Need to collect the occurrence of potential causes related to an effect Usually more effective with larger sets of data

32 Pareto Chart (Example) Potential Causes A and B are the “vital few” Example Pareto Chart 0 0.1 0.2 0.3 0.4 0.5 0.6 ABCDE Potential Cause Occurence (%) 0 0.2 0.4 0.6 0.8 1 Cumulative Percentage Cumulative

33 Determination of Root Causes Tools & Techniques (continued) Fishbone Diagramming Based on Kaoru Ishikawa’s approach Provides a graphical representation of causes in a causal factor chain Uses cause groupings for default causes: Policy – Unclear or non-existent policies Procedure – Lack of sufficient, clear procedures Organization – Ineffective business organization People/Culture – Performance of personnel or groups Machines are not causes in IA control failures Acceptance is an issue within other process groupings

34 Fishbone Diagram (Example) Weak Passwords Procedure Password procedures not implemented Procedures drafted but not approved Password management system being revamped Organization People/Culture Policy

35 Determination of Root Causes Tools & Techniques (continued) Event Causal Factor Chart Based on Max Ammerman’s approach Provides identification of potential and root causes based on a workflow methodology Standard charting methodology Circles – Start and end conditions Rectangles – correctly-functioning events Diamonds – Incorrectly functioning events Light Ovals – Secondary causes Dark Ovals – Root causes

36 Event Causal Factor Chart (Example) Baseline Established XYZ Application Down 1/1/05 Helpdesk Creates Trouble Ticket System Moved to Production Helpdesk Assigns Trouble Ticket to Admin Change Not Documented Admin Finds Problem/ Applies Fix Helpdesk Contacts Admin To Close Helpdesk Closes Trouble Ticket Baseline Differs From Production Insufficient Resources Helpdesk Not Trained Elevated Helpdesk Privileges Admin Reassigned Helpdesk Avoiding Notification

37 Determination of Root Causes Concerns: Go beyond the “pilot error” knee-jerk defense Make sure you don’t stop until fundamental causes are identified Be creative in the techniques used, use the techniques that are most effective “To thine own self be true”

38 Root Cause Elimination Goals: Suggest controls or changes to existing controls to eliminate the root cause of undesired events Identify metrics to ensure that proposed recommendations resolve root causes within the field

39 Root Cause Elimination Tools & Techniques: Prioritized potential fixes are identified with effort represented with all costs For each root cause, metrics are defined to allow measurement of the effectiveness of an implementation The findings of analysis and recommended fixes are provided to management for decision A champion is identified to pilot the implementation of recommended actions

40 Root Cause Elimination Concerns: Effectiveness should be provided as a function of return cost of future events against up-front and ongoing costs for each recommendation Credibility of the team in front of management is key Typically the best champion is a member of the RCA team with sufficient organizational power to effect change

41 Common Pitfalls Human nature Team dynamics Lack of organizational knowledge Lack of necessary skills (technical and professional) Lack of team credibility Organizational dynamics “Gotcha” mentality Blame game/personnel-rating Insufficient resources “Drive-by” RCA

42 War Stories

43 Conclusions IT organizations should consider effectiveness of IA controls as a key measure of quality for the processes used to develop or acquire systems Root cause analysis is directly applicable to and can provide great benefits to IT organizations, and specifically within the IA controls environment

Download ppt "Ian M. Harper, CISA, CISM, CISSP Chief Technology Officer Dealing with Your IT Process Failures Improving Security with Root Cause."

Similar presentations

Ads by Google