1. Information Fusion in Continuous Assurance Johan Perols University of San Diego Uday Murthy University of South Florida UWCISA Symposium October 2, 2009 1

2. Introduction and Motivation Drivers for continuous assurance (CA) have been in place for some time  A number of CA implementations have been described in prior research (Groomer & Murthy 1989; Vasarhelyi & Halper 1991; Alles et al. 2006)  Technology vendors providing CA functionality (SAP, ACL) Existing continuous assurance architectures focus on the detection of exceptions and do not address the important problem of processing detected exceptions. 2

3. Introduction and Motivation CA systems can lead to information overload, with a large number of detected exceptions Dealing with detected exceptions requires aggregating, processing, and analyzing exceptions Behavioral research in psychology and auditing has shown that humans are not good at these tasks 3

4. Research Objective We introduce Continuous Assurance Fusion (CAF): an architecture for CA that uses the concept of information fusion for aggregation and analysis of detected exceptions. CAF is grounded in prior CA literature and computer science information fusion research. 4

5. Information Fusion Information fusion involves gathering data about an object of interest from multiple sources and integrating these data in order to arrive at a holistic conclusion about the object. Application fields: defense, geoscience, robotics, medicine, and industrial engineering Goals of information fusion—  Dimensionality reduction  Improving precision / reducing uncertainty  Improving robustness when some data are noisy 5

6. CAF Overview The data gathering component of the architecture draws from prior CA research REA ontology concepts are used in the first data integration step Artificial intelligence and machine learning algorithms are used in the subsequent integration and evaluation steps 6

7. Background: Exception Detection Detection of exceptions can be data-oriented or control- oriented Two architectures have been proposed to accomplish the exception detection task  Embedded audit modules (EAM)  Monitoring and control layer (MCL) 7

8. Embedded Audit Modules Embedded audit modules (EAM) Examine transactions and controls for exceptions in real-time Relatively intrusive as detection code is implemented at the application or database level Offers better detection capabilities, but consumes more resources, so EAMs are better suited for control- oriented exception detection, in real-time Control over EAM resides with transaction processing system owners 8

9. Monitoring and Control Layer Monitoring and control layer (MCL) Examines transactions and controls for exceptions on a periodic basis Implemented as a stand-alone system that periodically queries transaction and control data Better suited for large-volume data-oriented exception detection, since it does not affect operational TPS Control over EAM resides with EAM owners: auditors 9

10. Alarm Overload Both data exceptions and control exceptions can result in alarm overload Dealing with alarm overload  Manually turning off groups of controls to minimize the degree of exception flooding (Alles et al. 2006)  Adjust CA monitoring parameters such that fewer alarms are generated 10

11. Continuous Assurance Fusion (CAF) CAF provides a method for aggregating and analyzing detected exceptions to draw audit-relevant conclusions Basic idea: It is possible to get a more complete and accurate assessment of objects and situations if data from many sensors are combined and multiple models are used to evaluate these data. 11

12. CAF Architecture Accounting Information System Aggregation Layer Monitoring Layer External Data Business Process Evaluation Layer Internal Data Information Users Figure 1 - CAF Conceptualization CAF CAF Data Decision Layer DAI-DAO DAI-FEO FEI-FEO FEI-DEO DEI-DEO 12

13. Layers in CAF Architecture Monitoring layer – same functionality as extant CA architectures Aggregation layer – generates object features by grouping exceptions based on their association to specific objects and computing additional object features Evaluation layer – invokes classifiers (e.g., logistic regression, ANN) on object features to make decisions about objects’ class membership Decision layer – combines the individual classifier object classifications into an overall CAF object class membership decision. 13

14. Aggregation Layer Data In  Feature Out fusion Use McCarthy’s REA ontology as the basis for aggregation layer processing  Automatically discerning features of exceptions Processing done is primarily grouping of exceptions in terms of severity, from Level 1 to Level 5 14

15. REA Ontology 15

16. Purchasing subsystem REA model 16

17. Aggregation Layer Level 1: Exceptions that relate to changes affecting a single R, E, or A object. Level 2: Exceptions grouped in terms of event-resource relationships. Level 3: Exceptions grouped in terms of event-agent relationships. Level 4: Exceptions grouped in terms of event-event relationships. Level 5: Exceptions grouped in terms of resource-event- agent relationships. 17

18. Exceptions – Purchasing Scenario Level 1: Invalid date/time stamp on purchasing order (exception involving a single ‘E’). Level 2: Purchase order for an inventory item with excessive quantity on hand (exception involving ‘E’ – ‘R’ relationship). Level 3: High dollar value purchase order placed by low level purchasing agent (exception involving ‘E’ – ‘A’ relationship). Level 4: Purchase order placed without existing purchase requisition (exception involving ‘E’ – ‘E’ relationship). Level 5: P.O. placed by low level purchasing agent for high value item (exception involving ‘R’ – ‘E’ – ‘A’ relationship). 18

19. Evaluation Layer Feature In  Decision Out fusion Use artificial neural network (ANN) technology  Input = features from aggregation layer  Output = probabilistic decision regarding “state” of the subsystem Purchasing business process example  Based on Level 1 – Level 5 features input, is the purchasing subsystem “in control” or “out of control”? 19

20. Decision Layer Decision In  Decision Out fusion Could use Bayesian control concepts, ANN, or RBES  Input = decisions regarding the state of individual subsystems from evaluation layer  Output = probabilistic decision regarding “state” of the overall information system Level of analysis for Evaluation and Decision layers is a design choice – modules / subsystems / system 20

21. Future Work Development of a functioning CAF prototype following architecture presented in this paper Use of agent-based technologies for implementing CAF Empirically evaluate and compare the utility of different classification algorithms at the Evaluation Layer Use of information fusion concepts in areas such as fraud detection 21

22. Summary and Conclusion Using concepts from information fusion, CAF is an architecture for dealing with audit exceptions CAF is not a functioning system but an approach that represents a way forward in addressing the problem of dealing with detected exceptions Aggregation layer applies REA ontology concepts to group exceptions by severity Evaluation and decision layers apply ANN or Bayesian concepts to draw conclusions about the state of a subsystem or the system as a whole 22