Data Testing Techniques Auditor obtains data file and uses program to examine the contents of the file and to produce reports Common functions of computer software Reading the contents of electronic data files Calculation and summarization Comparison - fields, records or files Sorting and reorganizing files Selection/Extraction data based on attributes Selection of rep. samples - using stat sampling Printing

GAS - generalized audit software Only limited by lack of auditor familiarity Most are easy to use Read data files, convert to std format file for processing Edit for unacceptable output and display control totals, browse data and analyze Advantages cheaper than custom better than manually Disadvantages reliability must be proven equipment/file constraints

Embedded audit modules Build audit routines or modules into regular processing e.g., extend a program that prints aged a/r listing to do confirmations Good for paperless systems, tags transaction for analysis Need to be involved in original design of system

Utility and custom written programs Utility Special function programs that can be performed by computer manufacturer library of programs Main limitation is that not designed for audit Include: sort, merge, copy, print, backup (speed backups), edit, recover, compression, virus detection, file transfer Custom written by auditor can perform exactly what is needed have to be written specifically for certain hardware Disadvantage - time cost and expertise to develop program, limited use and higher risk of error due to insufficient testing

Expert Systems Computer programs that emulate problem solving knowledge and skill of human experts Three components knowledge base - conceptual equivalent of the data base, made up mostly of rules invoked by pattern matching (thousands) inference engine - "heuristics...rules of good judgement that characterize expert-level decision making in the field activated by patterns in the knowledge base... this enables or fires some rules and disables others...problem processor finds the enabled rules and decides which one applies to obtain a solution...problems arise that every possible path is traced user interface - how information is asked for and given

Neural networks Try to solve problems by modeling the trial and error process of learning from experience They can be trained to solve certain problems or identify a set of specific patterns Learning algorithm adjusts the network by adjusting the weights among network components until it can correctly classify all presented facts (i.e., it becomes trained) Major advantages ease of system development, no need for if…then logic conditions an easier alternative to rule based expert systems where pattern recogn and classification involved Disadvantages - no facilities to determine how conclusion reached

Profile of data testing application Plan audit objectives and execution steps Select an appropriate technique Analyze accounting systems, data retention practices Arrange processing facilities and personnel Obtain and analyze data files Reformat / sort files if needed Select, summarize or extract data based on criteria Perform calcs on some/all of the data Select representative sample from sub-population Sort selected data into logical print sequence Print reports Review results for reasonableness Document results

Analytical Procedures Ratio analysis Statistical ratio analysis - id unusual and systematic fluctuations in ratios outside normal range of fluctuations; sets a confidence interval stat ratio - calculate expected value and std deviation using past data; confidence interval based on acceptable range of variation limitation - reliance on a normal dist'n as reference; if not normally distributed results may be incorrect or indefensible

Analytical Procedures Pattern analysis - use several ratios to signal problem, construct confidence levels around each and determine whether significant fluctuations represent pattern better screens out chance of fluctuations to avoid inappropriate conclusions limitation reliance on normal distributions poor choice in pattern may prevent systematic shifts from getting to auditors att'n or lead to excessive investigation Regression analysis - used to estimate relationships based on known values

Control Testing Stat. selection from transaction stream for subsequent checking of attributes (approvals etc.); lots of manual time involved Could use integrated audit facilities to provide evidence that key prog. procedures (edit and validation etc.) are being done

Detailed Testing Key item testing Cut-off work Calcs etc. A/R - select confirms, ageing, cash receipts Inv - counts/price test selection, clerical accuracy, key items, negatives, cutoff A/P - confirmation, footing, cutoff etc.

Def'n, design, execution and control of CAATs Feasibility Considerations significance of audit effort and time required more efficient? may be the only way to achieve audit objective availability and sequence of data files make sure available in readable form, watch retention policies

Def'n, design, execution and control of CAATs Feasibility Considerations availability of facilities make sure enterprise facilities are available, data files are readily usable, compatibility of software availability of qualified staff appropriate experience is needed economic considerations - need to be able to control audit costs, develop cost estimates, consider life cycle of the CAAT

Def'n, design, execution and control of CAATs Definition, design, execution and control of CAATs define audit objectives what to prove - indicate type of test and test data to be used selection of appropriate technique based on objective, degree of audit assurance required, techniques that are available, cost/benefit factors consider training, support, time constraints design of input, processing and output requirements expand conceptual ideas into detailed descriptions of application features (flowcharts, etc.)

Def'n, design, execution and control of CAATs Design of input, processing and output requirements audit software application flowcharts understanding of inputs, file interaction and outputs details of application logic - define before coding reports and other output requirements code tables - describe general purpose of codes, list each code and meaning and indicate procedures performed if an undocumented code is encountered control points - make sure audit software works, control totals etc.

Def'n, design, execution and control of CAATs Formulate test plans to make sure it works critical step use desk checking - detects logic problems computer testing using prior or audit data to review and compare Review and challenge application design

Execution and Control of CAATs Select and arrange facilities and resources determine whether audit software can be readily installed on client computer id version of software needed determine availability of supplies (disks etc.) Desk Checking and Testing coded specs should be tested determine data availability Running the applications auditor should control processing and control outputs reconcile control totals etc.

Execution and Control of CAATs Audit documentation update working papers to show final status most GAS is self documenting to a degree Application summary report summarize whether objectives are met, significant problems encountered, special software features and other comments File retention client data should be maintained for at least year to permit year to year comparison

Audit Control over CAATs Maintain to reduce errors such as data records not match documentation dropping intermediate files before finished processing audit software not working properly application logic not correct Auditor should ensure results of the application will be reliable by choosing adequate controls to guarantee reliability Control techniques selected will have a direct impact on work plans

Factors Affecting Control Environment Nature of CAAT - integrity of software, methodology for testing is well documented watch reliance on any auditee software concern over manipulation test and do alternative testing to ensure integrity Inherent risk more stringent testing for higher risk items Internal control will impact extent of computer audit procedures that can be used Assurance to be derived more reliance on primary evidence more stringent CAAT controls

IDEA Case Study Bonus Program –Background –Audit Objectives –Documentation Company Policy Software Flowcharts –Input (record layout) –Process (black box) –Output Organizing Audit Software Approach How to run the program When to stop What to hand in