Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Philip Elsas ComputationalAuditing.com Newark, New Jersey November 6-7, th World Continuous Auditing and Reporting Symposium

ComputationalAuditing.com Introduction Since 2003: Company - Canada, Netherlands : Deloitte. with intermezzo at Bakkenist Management Consultants, sold to Deloitte : PhD Computational Auditing - Principal, chief architect & inventor of Smart Audit Support - Smart Audit Support: since 1994 key in Deloitte’s worldwide audit practice. Currently integrated in ‘The Deloitte Audit’ - System blueprint in chapter 5 of … - PhD in Mathematics & Computing Science, on Financial Auditing - In parallel to Smart Audit project, 30% part-time, Vrije Universiteit - Directly after appearance awarded with the biennial Alfred Coini Prize for the best publication in Auditing Offering software and consultancy services to innovate audit practices and audit software firms 1 The Dutch Tax Office used Computational Auditing in as frame of reference to compare Big 4 planning and decision-support models & systems to investigate how to improve audit productivity (57 page report); considers Smart Audit Support ‘leader of the pack’

ComputationalAuditing.com Agenda Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: “What is the content?” Aggregation mechanisms: quantitative, qualitative & confidence Web platform for audit support: “How to use that content?” 2 Managing the use of aggregation & classification Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation

ComputationalAuditing.com Web platform for audit support: What’s the content? by auditors, for auditors ACL AuditExchange (AX 2), Business Assurance Platform Interactive Audit Documentation * Audit pack: a bundle of interrelated forms, specific for an industry, or sector 3 Deloitte’s ‘Builder Player Platform’-architecture CaseWare Open Engagement & CaseWare IDEA Audit support architecture of a big audit firm, or of a shared back-office of a group of smaller audit firms Audit repository: data, scripts for analytics (CM), findings Working paper templates & scripts, DMS & KMS, partially organized per type of industry (website building system) Platform of audit packs* with check lists & audit planning templates,organized per type of industry All mentioned + capturing context to offer guidance in determining & configuring scripts for data analysis, addressing the key questions: - “When to do which test?” - “What to do with the test results?”

ComputationalAuditing.com p p.337 Specified Audit Methods drive integral Planning, Execution & Documentation Proven Architecture ‘Correctness by Construction’ Deloitte’s Smart Audit Support: Interactive Audit Documentation published in Word and browsers, World’s Strongest Audit Support* * Dutch Tax Office InstantaneousAdequate Flexible Questionnaire integrated in Web Forms: By making explicit what is needed to answer “When to do which audit test?” & “What to do with the test results?” you articulate a body of multiple-choice questions, tables, etc., connected by choice- labeled relevancy links, embodying an approach, a method, or even, if possible, a workflow process, to guide how to achieve assurance Effective: don’t miss relevant issue Efficient: no access to less relevant issues Drives & Captures the ‘Story of the Audit’ Optimal mitigation of litigation risk Conditional Relevancy

ComputationalAuditing.com Smart Audit Support’s document index related to Deloitte’s International Audit Approach (1990’s) p p.62 All planning docs are smart forms All planning docs are smart forms with built-in Conditional Relevancy Example audit pack In addition to $200M yearly cost reduction ROI is: - Relevant Doc & Planning, no more no less - Comfortable & stringent way to get it Yearly ROI guess : 20K man-yrs/yr x $10K cost reduction/man-yr = $200M Deloitte’s approach

ComputationalAuditing.com Interactive Audit Documentation: Dedicated Functionalities for the Audit Team Filling out a web-based questionnaire with multiple-choice questions: “The Auditor’s New Clothes”, 2008, Tom Koning & the ‘Audit Navigator’, translation into English is pending Capturing the ‘Story of the Audit’, ISA Functionalities for audit workflow operators Activates dedicated support to indicate how to: –Specify a norm for an entity-level control –Specify a fraud risk, including a description of who is able & how to do it –Specify a norm for initial numerical analysis; when within norm, no extra tasks –Specify or configure a script for a data analysis tool –Decide to involve an external specialist in your audit team (e.g. forensic, EDP) Activates relevant, more detailed questions & de-activates irrelevant Aggregates audit risk/audit evidence, according to a prescribed processing scheme, as captured in risk summarization tables Plans and configures audit tasks to constitute an audit plan, for example, based on accumulated risk: –To be able to rely on a specific assertion level control –To further investigate the risk by planning substantive procedures Shows when to stop investigating an account, a process or an assertion Sets a risk classification to ‘significant inherent risk’ Documents and guides: –“What has been done?” & “What has to be done?” –“What information has been found?” & “What’s the impact on the audit?”

ComputationalAuditing.com Agenda Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: “What is the content?” Aggregation mechanisms: quantitative, qualitative & confidence Web platform for audit support: “How to use that content?” 7 Managing the use of aggregation & classification Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation

ComputationalAuditing.com Web platform for audit support: How to use that content? ‘business wise’ by auditors, for auditors Interactive audit documentation & business positioning: 8 Successfully positioned within Deloitte “Audit Software: From Bench Warmer to Star Player”, Royal NIVRA, “de Accountant”, March 2009, pp , Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com Professional bodies of CPAs and standard setters upload high-level guidance packs à la ISA & strict forms à la Tax. Basis to be refined upon, but not overruled Building & uploading by fee-earning expert auditors Downloading & use by fee-paying engagement teams Broker-fee for the hosting platform provider Trade in audit packs between member firms External auditors develop tailored packs & on-line services for client’s internal audit department. Why? Marketing strategy of ‘vendor lock-in’

ComputationalAuditing.com Web platform for audit support: How to use that content? ‘society wise’ by auditors, for auditors Interactive audit documentation & ‘open pack’-platform: 9 “Audit Software: From Bench Warmer to Star Player”, Royal NIVRA, “de Accountant”, March 2009, pp , Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com Invitation to CaseWare & ACL: do you want to contribute to proposing a tailored version to AICPA & CICA? Uploading by content providing expert auditors, using a dedicated content builder Downloading by engagement teams, using a generic player to apply content Content is certified, published & hosted by A.an audit firm’s global and national office (layers) B.a professional body of auditors C.a standard setter or regulator each granting access rights to their members, ideally with ‘content overlaying’ (A on top of B, B on top of C)

ComputationalAuditing.com 10 Recap ‘Builder Player Platform’-architecture “How to get the data?” is not the challenge anymore. Today, audit analytics fully focuses on: “How to use the data?” & “How to manage that use?” Aggregation & classification are key methods of using data, so let’s have a look into how to manage aggregation & classification “What keeps audit leaders up at night?”, ACL, 2008 Support in capturing audit methods Support in applying audit methods Support in classifying audit methods Goal of the PlatformGoal of the BuilderGoal of the Player BuilderPlayer “Audit Automation as the Foundation of Continuous Auditing”, Michael Alles, Alexander Kogan, Miklos Vasarhelyi & Donald Warren, 16th WCAS, 2008

ComputationalAuditing.com Agenda Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: “What is the content?” Aggregation mechanisms: quantitative, qualitative & confidence Web platform for audit support: “How to use that content?” 11 Managing the use of aggregation & classification Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation

ComputationalAuditing.com 12 Aggregation scheme for risk assertions (cf 20) Yahoo! SiteBuilder + own plug-ins to specify, visualize & interact with aggregation links (W3C SVG) What do the arrows mean? E.g. Table A1.2.1 accumulates risks regarding the assertion ‘Systems that retain …’ based upon underlying feeding questions such as E1.6 & classifies & propagates the accumulated risk to Table A1.2 & A1 to contribute to driving the configuring, via table S2, of audit tasks constituting the audit plan Expressible, in a similar way, in Deloitte’s Smart Audit Support, see: ‘Computational Auditing’, p.328 Experiments with Adobe Flex, MXML & Google Open Docs, considering CaseWare’s Open Engagement Website Building System Risk summarization tables capturing assertion-based aggregation schemes Managing the use of aggregation & classification The arrow is an Audit Workflow operator

ComputationalAuditing.com Aggregation, Process Mining & Workflow 13 Input: event log with journals, e.g. SAP Output: smart flowchart Analyzing 3232 cases, classi- fying casualties (red arrows): A. Invoice receipt without prior approval (2537x) B. Approval acquired after pur- chase completion (261x) C. Purchase order established for rejected request (9x) D. Handled order status skip- ping receipt (875x), etc. Managing the use of aggregation & classification Based on: “Towards a Computer- Assisted Audit Analysis of Business Processes: Process Mining as Tool for IT Auditors”, Maria Bezverhaya, Emiel Caron & Piet Goeyenbier, ‘de EDP-Auditor’, NOREA, 2009 D A C B Design-time workflow vs. run-time workflow Push signal from Technical University of Eindhoven, ProM, Fluxicon & Anne Rozinat Pull signal from audit practitioners & IT audit educators Computational Auditing: - focus on discovery of supercycle - framing ‘stand alone’ workflows - connecting to 80 years of audit theory

ComputationalAuditing.com M: Majority Owner-Manager S: Sales department B: Buy/Purchase department F: Financial department T: IT department W: Warehouse manager L: Labor/salary accounts P: Planning department C: Creditor accounts D: Debtor accounts A: Application Agent Legend C b f t F m d D s t A t L f t P t W t A t S A A L F M M D F D C B F W P P P P W A A A A C m D f t S t A t F t B f t P t W t L f , Agent’s access is associated to: 1. Transactions 2. States 3. Flows Capital letter: authorized, legitimate access Small letter: illegitimate access 14 Ernst & Young’s Smart Flowchart Pilot Study Case by Hans Verkruijsse & EY team, More on integrated audit analytics: “Enterprise-level Process Documentation incorporating Automatic Audit Analytics”, 2008 Deloitte/KU Symposium & follow-up with Raj Srivastava & EY CARAT Approach: Powerful and easy system to support practice, founded in theory World’s strongest ‘business process’-oriented auditing theory: classical Dutch auditing theory (80+ years) & its best-fitting rigorous process theory: Petri nets tailored to the auditing domain Dynamic: Transaction Profit & Loss Item T Static: State Balance Item S Top-level is Supercycle, or Top-cycle. Connects traditional cycles Case in Efrim Boritz’ CAATTs class, Fit recognized by Jagdish Gangolly, EY’s evaluation report: - Clarifying. Refreshing. - Systematic framework guides input preparation process (2009: new style) - Quantitatively motivated process decomposition Managing the use of aggregation & classification New in 2009: Process mining; pilots by a Big 4, UvA.nl & CWI.nl  Focus on top- cycle discovery Output: 1. ‘As Is’ diagram (‘Ist’) 2. Identify ‘To Be’ (‘Soll’) 3. Built-in audit analytics Input: event log

ComputationalAuditing.com Typology of Top-cycles 15 Scientific foundation: rationally rigorous. With mathematical & computational formalization. Superbly suited for the digital age. Recognized as such in accelerating pace. Easy by new tech Limperg, Starreveld, Frielink, Blokdijk & Veenstra Managing the use of aggregation & classification Top-cycle: normative backbone of the ‘business process’-oriented audit approach previous slide: example supercycle Top-cycle concept & typology: Central result of integral evolution. Of ‘business process’-oriented Auditing Theory, Auditing Practice & Auditing Education. Over years Typology/classification of top-cycles: ordered by the strength of the backbone Unfortunately hardly translated into English

ComputationalAuditing.com 16 Starreveld et al. Typology of Top-cycles Frielink et al. Supercycle- backboned Audit Approach Volumes 1, 2a, 2b, etc. Managing the use of aggregation & classification ‘Industry classification’- based auditing concepts, norms & methods Decisive advantage of these concepts, norms & methods: no need to prove again in practice, since practice was part of the evolution process

ComputationalAuditing.com Agenda Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: “What is the content?” Aggregation mechanisms: quantitative, qualitative & confidence Web platform for audit support: “How to use that content?” 17 Managing the use of aggregation & classification Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation

ComputationalAuditing.com Mechanism for quantitative aggregation 18 2 Receivables 3 Inventories += See: “On Positioning XBRL Assurance Business Rules in a Computational Infrastructure for Modern Auditing”, 2009, University of Kansas, Annual International Conference on XBRL Aggregation in XBRL: - Calculation linkbase - XBRL Formula Plug-in ‘type polymorphism’ mechanism (transferable) from programming language into XBRL Assurance Builder & Player Domain-Specific Language (DSL) for auditing: Pacioli, developed by Dutch software partner in cooperation with national research center for mathematics and computer science in the Netherlands (CWI) & University of Amsterdam 5 Assets 5 Current Assets At least one noncurrent inventory All three inventories are current { XBRL US GAAP Taxonomy or Articulate XBRL Assurance functionality using a dedicated website builder (plug-ins) instead of handcrafting XBRL Formula’s Type Polymorphism: Least Upper Bound in the Taxonomy

ComputationalAuditing.com For reasons of efficiency: establish a full aggregation as early as possible in the audit process (observation by William Kinney) Mechanism for qualitative aggregation: 19 Irreplaceable in the sense that there is no way for an external auditor to compen-sate its lacking or failing, while it is indispensable for a rationally justifiable approval “X-Raying Segregation of Duties: Support to Illuminate an Enterprise’s Immunity to Solo-Fraud” with discussions & response, IJAIS, June 2008 Solo-fraud free? Design, Implementation & Operation Continuous auditing web service (hosted via external auditor?) intercepts every Authorization Change Request to signal: refuse human intervention required OK Efrim’s proposal (2008): Large-scale introductory study for this science-based method. As for new medicine. New method on top of Dutch auditing theory as incarnated in computational process theory. Collaboration with Canada. Identification of budget doubling when large audit firm steps in. Current status: pilots by Big 4 Dutch member firm Method locating who has too many authorizations in one hand creating a dangerous opportunity for traceless embezzlement, jeopardizing the integrity of financial statements How to aggregate weak spots in the Internal Control that are both irreplaceable and indispensable, e.g. weak spots in Segregation of Duties? “Get it right at entry level” Focal point in modern auditing? Launched at Accountant.nl by Jules Muis, Oct Directly endorsed by Hans Blokdijk, Marc van Hilvoorde and others. Berry Wammes, CEO Royal NIVRA, directly stated the intent to position “Get it right at entry level” as the theme for the NIVRA spring 2010 debate series Top-of-iceberg solo-frauds: 1. Madoff 2. Stanford 3. Kerviel, etc. Clarifies why & how weak spots in the SoD require a hot-line direct-top-level aggregation mechanism

ComputationalAuditing.com 20 Mechanism for confidence-level aggregation (cf 12) Based on: Sun, Srivastava & Mock, 2006 “An Informa- tion Systems Security Risk Assessment Model”, pp This can be realized in Deloitte’s Smart Audit Support with a plug-in for Dempster-Shafer-Srivastava confidence-level computations

ComputationalAuditing.com Agenda Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: “What is the content?” Aggregation mechanisms: quantitative, qualitative & confidence Web platform for audit support: “How to use that content?” 21 Managing the use of aggregation & classification Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation Early Warning System as Killer App for XBRL Assurance & Continuous Auditing: speeding up getting their ‘Place & Future’ into ‘Here & Now’ “The PCAOB and the Social Responsibility of the Independent Auditor” Douglas Carmichael, Founding Chief Auditor of the PCAOB ‘Golden Opportunity’ Jan Helderman, President Royal NIVRA, Accountant.nl, Sept. 2009

ComputationalAuditing.com Proposed Solution 1.An off-the-shelf system for tracking-and-tracing bar-coded products, configured for, and populated by ‘XBRL tagged’ financial products 2.A regulator-mandated auditor attests internal controls for the XBRL reporting channel to the new governmental systemic risk agency. Allowing for a continuous data stream—further subjected to audit tests, sampling & monitoring—with on-the-fly automatic aggregation into systemic risk indicators (release 1.0: ‘Bookstaber’ indicators) How far away? XBRL Assurance is closer than ever 22 ‘Golden Opportunity’ Royal NIVRA: “Preparing for an Audit Mandate to Contribute to Systemic Risk Anticipation”, magazine, web & adopted in ‘Sharing Knowledge’-project 1.Instead of expecting more from XML, start expecting more from the builder-based approach to XBRL & continuous auditing 2.Release 1.0: matter of weeks or months, not years Jumpstart by cooperation of top-specialists Rick Bookstaber, Miklos Vasarhelyi, Raj Srivastava & Charlie Hoffman, and preferably in cooperation with a Big 4 audit firm Small step for XBRL & Continuous Auditing, quantum leap for the financial world Bailing out inflates moral hazard, early warning deflates More rigor on macro, more rigor on micro: use Dutch auditing Limperg’s Theory of Rationalized Confidence Dutch Auditing Day, hosted by Royal NIVRA, November 25, 2009, agenda’s keynote & key discussion: “risk systems & systemic risk”