1. Automação de Auditoría Ryan Teeter, Ph.D. Student, Rutgers University 18 th World Continuous Auditing Symposium 6 th CONTECSI São Paulo, Brasil – June 4, 2009

2. 18 th WCARS – June 4, 2009 Aiding the Audit Outline 1. Introduction –Continuous Controls Monitoring (CCM) and COSO Guidance 2. Automating the IT Audit –Evaluating monitoring software platforms 3. Implementation of CCM at Siemens PLM –Classifying audit requirements into degrees of automation –Creating rules from Audit Action Sheets –Reengineering audit processes –Feedback loop 4. Preliminary Results –Time and resource commitments –Successes & Challenges 5. Conclusion 2

3. 18 th WCARS – June 4, 2009 Aiding the Audit 1. Introduction Continuous Controls Monitoring (CCM) –Evaluating control settings on business processes that provide compliance with regulation and/or internal management objectives. –Proof of concept expands existing research in continuous audit streams (see Brown, Wong, and Baldwin 2007, Alles et al 2006) COSO “Guidance on Monitoring Internal Control Systems” (2008): –Effective monitoring involves (1) establishing an effective foundation for monitoring, (2) designing and executing monitoring procedures that are prioritized based on risk, and (3) reporting the results, and following up on corrective action where necessary. 3

4. 18 th WCARS – June 4, 2009 Aiding the Audit 4 2. Automating the IT audit Why the IT audit? –Cost considerations and effectiveness –Internal audit team spends approximately 70 days manually checking tables, authorizations, and documentation Vasarhelyi et al (2004) indicate firms are likely to adapt existing internal audit programs Alles et al (2006) suggest utilizing the expertise of experienced audit professionals Verifiability of automated controls against results from the manual audit

5. 18 th WCARS – June 4, 2009 Aiding the Audit CCM aids compliance for SOX sec. 201 and 404 Large accounting firms unable to source CCM Third-party platforms installed as monitoring and control layer –Minimal impact on performance –See Vasarhelyi, 2004 2.1 Evaluating monitoring software platforms 5 TypeERP TopographyExample System-specificHomogenousApprova BizRights Modular and mappingHomo/heterogenousACL IDEA, OverSight CustomHomo/heterogenousSiemens’ e-Audit Alles, et al 2006

6. 18 th WCARS – June 4, 2009 Aiding the Audit Siemens’ Current SAP Audit Model Use text file output and transaction checks on line to audit SAP Report findings and recommendations for remediation Use follow-up audits to assure appropriate controls are in place and remain in place 6 Company A SAP SYS. PD2 Company B SAP SYS. P88 Company C SAP SYS. P51 Company D SAP SYS. P40 Common –“E -Audit” Extractions on a request basis. Text File Store Text File Store Text File Store Text File Store

7. 18 th WCARS – June 4, 2009 Aiding the Audit 3. Implementation of CCM at Siemens PLM Rules were created in Approva BizRights based on ~300 audit action sheets provided by Siemens Siemens Corporation wants universally- adaptable sets of rules and control tests for use in different divisions 7

8. 18 th WCARS – June 4, 2009 Aiding the Audit 8 AAS Audit Program Test of Effectiveness: 1. /nSA38 report RSUSR002, user SAPCPIC. 2. Check whether SAPCPIC is used as a dialogue user (>>eAudit: 1.02.060_2 SAP* data in USR02 – last login date, UFLAG <<) 3. Check which profiles have been assigned (>>eAudit: 1.02.060_3 profiles of SAP* in USR04<<)

9. 18 th WCARS – June 4, 2009 Aiding the Audit 9

10. 18 th WCARS – June 4, 2009 Aiding the Audit 3.1 Classifying rules by degree of automation Authorization –users with access to screens or functions –Approx 30% of audit effort Baseline Separation of duties Transaction –Frequency of code use User Activity Insight –Timeliness and correctness Configuration –ERP settings Manual 10

11. 18 th WCARS – June 4, 2009 Aiding the Audit 3.2 Creating rules from Audit Action Sheets Low-hanging fruit –Authorization requests –Separation of duties checks –Example: See who can create and approve purchase orders Partial automation –Example: See who has access and whether that is appropriate Non-automatable –Evaluation of documentation –Interviews with managers 11

12. 18 th WCARS – June 4, 2009 Aiding the Audit 3.3 Reengineering audit processes Creation of custom rules in Approva InsightStudio Combination of existing controls tests Partial automation of manual controls –“Gain an understanding of X process. Verify Y function isn’t allowed.” 12

13. 18 th WCARS – June 4, 2009 Aiding the Audit 3.4 Feedback loop Rule descriptions were added to aid the audit Rules were tested and compared to results from the manual audit Adjustments were made based on results 13 Rule TypeAAS Ref # Short DescriptionDescription of rule to be made Conditions used Status Authorization1.02.XUnauthorized access to SAP system – emergency user concept Test these authorizations: 1. S_TCODE=SM18, S_ADMIN_FCD=AUDA 2. … AI rulesRule Built 1.02.X Configuration1.02.XSystem admin/completeness verification Set up 3 rules to test the following: 1. parameter rdisp/vbdelete=0 2. parameter rdisp/vbreorg=0 3. … Parameters are listed in report RSPFPAR Rule Built 1.02.X

14. 18 th WCARS – June 4, 2009 Aiding the Audit 4. Results Time and resource commitments Successes Challenges Firm characteristics 14

15. 18 th WCARS – June 4, 2009 Aiding the Audit 4.1 Time and resource commitments Time commitments: –70 days for the manual audit –3 months preparation Platform installation AAS classification Resource commitments –Travel, lodging, etc. 3-5 researchers – 3 Full-time equivalent 2 internal auditors at PLM 2 IT auditors from Siemens 1 support staff from Approva 15

16. 18 th WCARS – June 4, 2009 Aiding the Audit 4.2 Successes Initially approximately 63% of controls automated Rules were used to provide support for the IT audit Initial evaluation of cost savings (A&D PL specific) 1 : – For 3 of every 4 years, eliminate ~ 500 man-hours of IT GCC and application control testing (@ $137/hr = $68,750/year) – With system certified, 80% reduction in 500 man- hours of annual external IT audit hours (@ ~$200/hr, $80,000/year) 1 Siemens IT audit pool billing rate is $137/hour; Approx $200/hr Big 4 blended rate. 16

17. 18 th WCARS – June 4, 2009 Aiding the Audit 4.2 Successes Module Total Controls Authoriz ations Controls Percent Automated (%) Business Process Controls Percent Automated (%) Overall Percent Automated (%) Basis System (BC)10420100%8444%55% Financial Accounting (FI)558100%4751%58% Asset Accounting (AA)264100%2264%69% Sales and Distribution (SD)215100%1650%62% Materials Management (MM)328100%2454%66% Project System (PS)329100%2370%78% Human Resources (HR)14 100%N/A 100% Total28468100%21652%63% 17

18. 18 th WCARS – June 4, 2009 Aiding the Audit 4.3 Challenges Audit priority –Non-applicable rules ignored because of time constraints CCM platform issues –Bugs or unimplemented features –Identified when comparing automated with manual results –Vendor vs. auditor priorities –Issues addressed in future releases Properly functioning controls –Control failure resulted in lack of support for the audit 18

19. 18 th WCARS – June 4, 2009 Aiding the Audit 4.4 Firm characteristics Siemens PLM –Technology firms generally have better IT controls –Already using SAP R/3 Degree of success may depend on the amount of IT systems and support. 19

20. 18 th WCARS – June 4, 2009 Aiding the Audit 5. Conclusion The IT audit is a feasible starting point for CCM implementation –Existing audit plan –Knowledge of experienced auditors –Real-time performance comparison 63% of audit controls automated –100% of authorizations, which comprise 30-35% of audit commitment –vs. 75% proposed by Alles et al. (2006) 20

21. 18 th WCARS – June 4, 2009 Aiding the Audit Expanding this paper Weighting control risk? –(Cushing 1974, Cash et al, 1977, Vasarhelyi 1980, Srindini and Vasarhelyi 1986, Vasarhelyi and Srindini 1989) Cost savings reallocation to auditing rulebooks? 21