Presentation is loading. Please wait.

Presentation is loading. Please wait.

Application Auditing Scope, Approach, & Execution January 2009 Michael Kirk, CIA, CISA Risk-based.

Similar presentations

Presentation on theme: "Application Auditing Scope, Approach, & Execution January 2009 Michael Kirk, CIA, CISA Risk-based."— Presentation transcript:

1 Application Auditing Scope, Approach, & Execution January 2009 Michael Kirk, CIA, CISA Risk-based

2 7 Mike Kirk, CIA, CISA –Application experience includes: Oracle, PeopleSoft, JD Edwards, and industry-specific and customized applications for healthcare, insurance, energy/utility, manufacturing, construction, and environmental industries. –Board member of the Central Ohio Chapter of ISACA Introduction

3 8 JD Edwards

4 9 Risk Considerations for Application Auditing Defining Your Scope Developing Your Approach Execution Q & A Overview

5 10 Where do you start? 1.Understand the environment 2.Understand business & technical processes 3.Assess risks & controls 4.Develop improvements Scope Audit Methodology

6 11 1.Understand the business environment the application supports Develop a complete understanding of the business process flow (inputs, processing, & outputs), and, the data flow, including interfaces Defining Your Scope

7 12 2.Understand the applications technical environment –IT control environment –Off-the-shelf or customized –Legacy or web-based –Housed internally or external provider –Developed in-house or 3PP –User population –Dispersion of users Defining Your Scope

8 13 Understand the business process and technical environment Assess business information risks Defining Your Scope Start Notif y shipp ing Confirm customer order Ite m out -of- sto ck ? Log revenue A Information System End Confirm receipt Creat e invoic e Send info to Acct A Close books # Critical Risk / Control Points Critical Process MapsData-Flow/Interface Diagrams

9 14 Business & Information Risks: –Regulatory compliance –F/S integrity –PCI-related –Integration of an acquisition –Data privacy –Integrity of the application –Process control validation Defining Your Scope

10 15 Lessons Learned on Scope Critical dependencies: personnel, technical, external providers [BCP] Canned still seems to get modified Dont forget spreadsheets & report-writers! Leverage existing flow diagrams Invest the time to understand the process flow... –Adds value to your internal client –You may find that you are now the expert!

11 16 Top-down Approach: Auditing around the application… Information Technology General Controls (ITGCs) Bottom-up Approach: Auditing the insides… Application Controls Testing Developing Your Approach

12 17 Relationship between ITGCs and application controls 1.Development 2.Change Management 3.Security Administration 4.Operations Top-down Approach

13 18 Top-down Approach Source: COBIT 4.1 Framework Boundaries of Business, General, and Application Controls

14 19 Development AI2 Acquire and Maintain Application Software AI2.7 Development of Application Software Ensure that automated functionality is developed in accordance with design specifications, development and documentation standards, QA requirements, and approval standards. Top-down Approach Source: COBIT 4.1 Framework

15 20 Change Management AI6 Manage Changes AI6.1 Change Standards and Procedures Set up formal change management procedures to handle all requests (including maintenance and patches) for changes to applications, procedures, processes, system and service parameters, and the underlying platforms in a standardised manner. Top-down Approach Source: COBIT 4.1 Framework

16 21 Security Administration AI2 Acquire and Maintain Application Software AI2.4 Application Security and Availability Address application security and availability requirements in response to identified risks and in line with the organisations data classification, information architecture, information security architecture and risk tolerance. Top-down Approach Source: COBIT 4.1 Framework

17 22 Top-down Approach Change Mgt Network Security Development Application Security Database Security Process Controls Application Controls

18 23 Security roles and functionality Software Development Life Cycle –Test plans & supporting documentation –Testing to break vs. testing to pass Lessons Learned on Top-down

19 24 Dispersion and diversity of the business, processes, and technology compounds the effort Dont underestimate the value of a thorough general controls review! Lessons Learned on Top-down

20 25 Auditing the functionality and control effectiveness of the application can only be determined based on the maturity level and effectiveness of the ITGCs Auditing the insides… Application Controls Testing Bottom-up Approach

21 26 Bottom-up Approach Business Management Reporting Controls Application Data Processing Controls Access and Data Source Origination Setup Controls InputProcessingOutput

22 27 Understand the business process and technical environment Assess business information risks Bottom-up Approach Start Notif y shipp ing Confirm customer order Ite m out -of- sto ck ? Log revenue A Information System End Confirm receipt Creat e invoic e Send info to Acct A Close books # Critical Risk / Control Points Critical Process MapsData-Flow/Interface Diagrams

23 28 Understand the transaction process related to the application… identify key transactions Assess Application Controls Bottom-up Approach

24 29 Bottom-up Approach Transaction and Data Flow Detail Screens to Test

25 30 Bottom-up Approach Screens to Test Application Walk Through: Key Screens

26 31 Bottom-up Approach Source: COBIT 4.1 Framework Boundaries of Business, General, and Application Controls

27 32 Bottom-up Approach Application Controls AC1 Source Data Preparation and Authorisation –Ensure that source documents are prepared by authorised and qualified personnel following established procedures… AC2 Source Data Collection and Entry –Establish that data input is performed timely AC3 Accuracy, Completeness and Authenticity Checks –Ensure that transactions are accurate, complete and valid. Validate data that were input, and edit or send back for correction as close to the point of origination as possible. Source: COBIT 4.1 Framework

28 33 Bottom-up Approach Application Controls AC4 Processing Integrity and Validity –Maintain the integrity and validity of data throughout the processing cycle. AC5 Output Review, Reconciliation and Error Handling –Establish procedures and associated responsibilities to ensure that… verification, detection and correction of the accuracy of output occurs. AC6 Transaction Authentication and Integrity –When sharing data between internal applications and business/operational functions… maintain integrity. Source: COBIT 4.1 Framework

29 34 Bottom-up Approach Testing Documentation –Application Function (screen mapping) –Function Description –Testing Procedures –Control Observations (testing results) & Recommendations –Key Risks & Impact Lots of screen shots!

30 35 Bottom-up Approach Application Function Description Testing Procedures Control Observations & Recommendations Key Risks & Impact Order Entry – Override Pricing Screen #: OE480 Screen allows user to override pricing information for the order. Examined updateable fields for data validation edits. 1. Gross Price 2. Sales, Inventory, and Expense G/L Codes Noted existence of adequate validation checks of the following: 1. Table Look-up (Sales Inventory, and Expense G/L Codes) 2. Existence Check (Sales Inventory, and Expense G/L Codes) 3. Completeness Check (Sales Inventory, and Expense G/L Codes) Noted the lack of the following validation checks: 4. Limit Check (Gross Price) 5. Reasonableness Check (Gross Price) Noted that limit and reasonableness checks are not in place for the gross price field to restrict the price of items ordered. This screen will also allow the gross price of the ordered item to be zero. We recommend that the appropriate limit and reasonableness checks be implemented for the gross pricing field to prevent blank or unreasonable data entry. Noted the following control weakness: Access to Override G/L Accounts Noted that the sales, expense, and inventory general ledger code fields could be edited through the override pricing screen. User access to edit critical fields should be minimal. We recommend that only appropriate and essential users be granted the permission to edit the G/L codes. Reports that rely on the gross price field or perform calculations based on this field have the potential to contain incorrect information based upon the observed control weakness. Modifying the G/L codes in the order screens could result in the posting of orders to inappropriate G/L accounts and subsequently impact updates to the Sales General Ledger and the Sales by Branch/Product Group (SLS042) report.

31 36 Bottom-up Approach Testing Procedures –Good code vs. not-so-good code Sequence checks Limit checks Range checks Validity checks Reasonableness checks Table lookups Existence checks Completeness checks Duplicate checks Logical relationships

32 37 System-Based Data Entry Integrity Controls EditsDescription Sequence Checks The control number follows sequentially and any control numbers out of sequence or duplicated are rejected or noted on an exception report for follow-up purposes. For example, invoice numbers are systematically and generated sequentially generated and cannot be overridden. Limit Checks Data should not exceed a predetermined amount. For example, it may be determined that quantities ordered should not exceed a maximum of 500 each, or that a sales commission cannot exceed 12% without an override. If a quantity exceeds the limit, the data would be rejected for further verification or authorization. Range Checks Data should be within a predetermined range of values. For example, date ranges. Once books are closed for the current month, entries are not permitted for the previous months date range. Validity Checks Programmed checking of the data validity in accordance with predetermined criteria. For example, department & expenses relationships, account codes, company numbers, vendor numbers, customer numbers, invoice types, etc. Reasonableness Checks Input data are matched to predetermined reasonable limits or occurrence rates. For example, the tolerances exist within purchasing application to allow for quantity variations or unit price variations based on preset tolerable limits. Table Lookups Input data are agreed to predetermined criteria maintained in a data table of possible values. Essentially the same type of controls as validity – any pull-down menu, pre-defined field… an employee must exist within the EMPMSTR table for payroll processing. Existence Checks Data are entered correctly and agree to valid predetermined criteria. For example, a picking pack slip can only be generated for an existing order number, or in order to receive against a P.O., it must exist. Completeness Checks Data fields should always contain data and not zeros or blanks. A company number, accounting unit, and account code for a general ledger entry are required fields prior to the record being added to the system. A hard stop is received by the user if the required fields are not completed. Completeness check controls can be utilized by individual field or by data entry screen. Duplicate Checks New transactions are matched to those previously input to ensure that they have not already been entered. For example, accounts payable invoices cannot be entered twice for the same vendor invoice. Logical Relationship Checks If a particular condition is true, then one or more additional conditions or data input relationships may be required to be true and consider the input valid. Company, accounting unit, & account relationships exist. For example the marketing department cannot charge an expense to lease expenses, only to accounts logically tied to that department.

33 38 Bottom-up Approach Testing Procedures contd. –Field formats are defined & locked –Grayed fields… better yet, linked to user and displayed only if applicable to functionality –On screen, visual feedback –Not-so-good… better have monitoring reports as mitigating controls!

34 39 Applications pre-dating the current climate of control Baselining Value of an implementation project audit Application functionality… best if conducted throughout testing stage End user training & desktop procedures Lessons Learned on Bottom-up

35 40 Monitoring controls… issues typically arise from the one $1,000,000 transaction, not from the million $1 transactions Please remember to conduct audit work in a Test environment Dont underestimate the time commitment! Emerging technology trends… web-based, PDAs, now smartphones Lessons Learned on Bottom-up

36 41 Risk considerations for application auditing Risk-based approach drives increased efficiency + decreased costs = organizational value Summary

37 Questions & Discussion Mike Kirk t: e:

Download ppt "Application Auditing Scope, Approach, & Execution January 2009 Michael Kirk, CIA, CISA Risk-based."

Similar presentations

Ads by Google