1. ~ Testing ~ General & Automated Controls
Chapter [4]
~ Testing ~
General & Automated Controls
Created By Manish Mathur

2. Created By Manish Mathur
Testing : Definition
Testing is the process of assessing Correctness, Completeness and Quality of system.
Testing is a process to determining whether the controls are adequately protect the system.
Types –
SUBSTANTIVE : To prove the integrity of the actual processing and To ensure that processes work to produce reliable results.
COMPLIANCE : To ensure that system controls adhere to management directives.
Created By Manish Mathur

3. Created By Manish Mathur
Phases –
PLANNING : Here auditor determines the way to collect the evidence to achieve objectives IS audit.
Testing : Here auditor tests the effectiveness of IS controls.
Reporting : Here auditor concludes and reports the result of audit to the management.
Created By Manish Mathur

4. Created By Manish Mathur
Audit Planning
Planning occurs throughout the audit and includes the following activities –
Obtain an understanding of the entity and its operations.
Obtain an understanding of internal controls.
Assess the risk.
Design the nature, extent and timing of audit procedure.
Created By Manish Mathur

5. Created By Manish Mathur
Auditor uses the concept of Materiality and Significance.
According to these concepts –
Auditor is not required to spend resources on item that are not material and significance i.e. those that would not affect the judgment of users of audit report.
Created By Manish Mathur

6. Created By Manish Mathur
Audit Testing
Some important decisions before testing begins ~
Testing Methodology-
Auditor must find testing methods to determine that controls are effective. This may include reviewing documentary evidence, conducting personnel interview and personal observation.
File interrogation-
Auditor must browse directories of PC to investigate use developed application files.
Created By Manish Mathur

7. Created By Manish Mathur
Test pack-
Auditor uses valid and invalid data to test the ability to prevent, detect, and correct errors.
The intensity and extent of testing depends upon importance of the application.
Automated tools –
Audit team can use GAS (Generalised Audit Software) to do sampling, data extraction, summarizing and reporting.
Created By Manish Mathur

8. Created By Manish Mathur
Tasks –
Understanding the entity and key business process.
Understanding entity’s network structure.
Identify key area of audit interest.
Assessing IT risks.
Identify critical control points
Understanding of IS controls.
Performing other audit procedures.
Created By Manish Mathur

9. Types of Control Audit Test
Financial Audit –
If IS control audit is performed as a part of financial audit, the auditor understand the controls over financial reporting to assess the risk of misrepresentation.
Performance Audit –
If IS control audit is performed as a part of performance audit, the auditor should evaluate the design and operating effectiveness of all the controls.
Created By Manish Mathur

10. Created By Manish Mathur
The following factors assist the auditor to determine which audit procedure to use to collect audit evidences –
The extent to which internal controls are to be tested.
The availability of evidences outside the system.
The relationship of system controls and data reliability.
Audit Objectives.
Created By Manish Mathur

11. Key areas of Audit interest
Key areas are those applications and files that are critical in achieving audit objectives.
For financial audit : key financial applications
For performance audit : all key system applications
For each key area, auditor should document –
Operational location
Significant components (h/w, s/w)
Other support systems/resources
Prior audit reports
Created By Manish Mathur

12. Understanding IS controls
Identify entity wide(component level) controls and determine that they are effectively designed and implemented.
Identify business process level controls for key application and check for effectiveness.
Any internal audit or third party reviews performed during last year.
Management’s plan for corrective action for the IS weakness and IS control weakness.
Status of the prior year’s findings.
Created By Manish Mathur

13. Created By Manish Mathur
Review any significant security incident for the last year.
Review entity’s security plan.
Review risk assessment for the system.
Certification and accreditation document for system.
Review BCP and DRP.
Review description of outsourced activities.
Relevant laws and regulation and their relation with audit.
Procedure to consider risk of fraud that could affect audit objectives.
Created By Manish Mathur

14. Created By Manish Mathur
Plan audit resources.
Review current multi year testing plan.
Communication with entity’s management.
Audit procedure for service organization audit.
Decision to use the work of others.
Develop audit plan : objectives, scope, methodology etc.
Decision to reduce testing of IS controls.
Auditor should document all these information as their Preliminary Investigation Documentation.
Created By Manish Mathur

15. Created By Manish Mathur
IS Control Audit Test
Auditor uses information obtained in the investigation phase to test the effectiveness of IS controls.
While performing audit, auditor should assess evidence to identify any revision needed in audit plan. For example –
If significant weakness is found the auditor may decide to perform less testing in remaining areas.
Created By Manish Mathur

16. Auditor determine effectiveness of controls at following levels –
Created By Manish Mathur

17. Testing Critical Control Point
Critical control point is that component of system which is of significant importance.
Auditor tests controls related to the component, its operating system and its applications.
For e.g. – Router. Auditor tests the control related to the router itself, its operating system and applications.
Created By Manish Mathur

18. Effectiveness of IS controls
Auditor should conduct test of those control technique that are effective in operation.
To do so the best way is to test control in tired basis. Starting with –
Entity wide controls
System level controls
Business process application level controls
Data management controls
Ineffective IS controls at each tier generally prevent effective control at the subsequent tier.
Created By Manish Mathur

19. Created By Manish Mathur
General controls
General controls at entity wide and system level can be tested using techniques such as Inquiry, Observation, Inspection and re-performance thru test software.
After reaching favorable conclusion on general controls at these level auditor test general control at business process application level.
If general control are not effectively operating then auditor should-
Determine nature and extent of risk resulting from ineffectiveness.
Identify and test any manual controls as compensating control.
Created By Manish Mathur

20. Created By Manish Mathur
Application controls
Auditor tests those application controls that achieve control objectives when other general controls are ineffective.
If application controls are not likely to be effective auditor –
Understand the risk in terms of impact on audit objectives.
Identify any manual controls that achieve the control objectives.
If in the previous year controls were ineffective and management have not significantly improved it, the auditor need not to test them.
Created By Manish Mathur

21. Appropriateness of Control test
To keep appropriateness of control tests the auditor should perform appropriate mix of audit procedure that includes the following –
Inquiries of IT and management personnel
Questionnaires
Review documentation of control procedures
Inspection of approvals(authorisation)
Analysis of system information(configuration)
Analysis of output (accuracy of processing)
Review of data file
Re-performance of the control (use of test data)
Created By Manish Mathur

22. Multi year testing plan
Where auditor regularly performs control audit of the entity, the auditor may develop a multi year plan for control audit.
These plan should cover not more then 3 years and include schedule and scope of assessment.
Under multi year plan each control is tested at least once during the multi year period.
Created By Manish Mathur

23. Created By Manish Mathur
This concept allow auditor to test controls on risk basis rather then testing every control every year.
For example a multi year plan for an entity with 7 applications might include comprehensive test of 2-3 application annually.
Multi year plans are not appropriate in all situations. For example –
They are appropriate for first time audit.
They are not appropriate where audit has not been tested within a recent period.
For entity that do not have strong entity wide controls.
Created By Manish Mathur

24. Documentation of control testing phase
Information gathered during testing phase should be documented. This include –
Understanding of IS.
IS control objectives and activities.
Description of control techniques used by entity.
Specific test performed.
Description of nature, extent and timing of test.
Evidence of effectiveness of controls.
If ineffective then compensating controls.
Auditor’s conclusion about effectiveness of controls.
For each weakness; material weakness, significant deficiency or just deficiency.
Created By Manish Mathur

25. Created By Manish Mathur
Audit Reporting
After completing testing auditor summarizes the audit result, draw conclusion on the control weakness.
Auditor prepares this report on entitywide, system and BPA level collectively.
Such documentation may be developed as the audit progresses, allowing auditor to demo. that the weakness exist and can be exploited.
Auditor should also document the potential impact of weakness on completeness, accuracy, validity, confidentiality of system.
Created By Manish Mathur

26. Created By Manish Mathur
Some audit terms
Substantive testing
Substantive testing is used to determine the accuracy of information generated by a process.
Auditor generate and process test data to verify the processing steps.
Where controls are evaluated as ineffective, substantive testing may be required.
Auditor uses CAAT to generate test pack and conduct the test.
Created By Manish Mathur

27. Created By Manish Mathur
Some audit terms..
Analysis
Interviews and tests provide the raw facts for drafting a audit report but does not guarantee to produce a quality audit report.
Analysis is important to convert this raw material into finished product.
Timely analysis gives the auditor time to conduct further test and allow more time for corrective actions.
Thorough analysis includes the following 4 steps –
Created By Manish Mathur

28. Created By Manish Mathur
Steps
Created By Manish Mathur

29. Created By Manish Mathur
Step 1 : Re-examination
The two factors to be re-examined are : Standard and Facts.
Standard are the rules, procedures and practices that defines how an operation under audit should function.
The standards must be clearly understood by the auditors, because wrong understanding leads to incorrect findings.
Facts are evaluated after standards are reviewed. For accuracy the sample should be
Large enough to reflect behavior of population.
Representative of current control activity
Created By Manish Mathur

30. Created By Manish Mathur
Step 2 : Cause of Deviation
After understanding standards and facts, auditor identify the causes of the deviation.
Determining the cause is like answering the following questions –
Who (responsible)
What (initiating event)
Where (system component)
Why (contributing factor)
When (timing)
Cause determination helps to identify exposure and formulating recommendations.
Created By Manish Mathur

31. Created By Manish Mathur
Step 3 : Exposure and Materiality
These are consequences of deviation.
Exposure is the potential loss, harm, damage, theft or inefficient use and
Materiality is a qualitative judgment about whether a deviation’s frequency of occurrence and degree of exposure are significant enough for the deviation to be corrected.
Degree of exposure is related to Proximity and Severity of risk.
Proximity refers to the extent of asset availability to the users or environment. Limited access – less proximity.
Created By Manish Mathur

32. Created By Manish Mathur
Severity refers to the amount of loss. The greater the value of asset and higher the proximity, higher will be severity.
Frequency refers to how often the deviation will occur.
With understanding of Materiality and Exposure auditor can identify why corrections should take place.
Created By Manish Mathur

33. Created By Manish Mathur
Step 4 : Conclusion
Conclusions are auditor’s opinion on, whether the audit subject area meets the audit objectives.
Conclusions must be supported by factual evidences.
Created By Manish Mathur

34. Concurrent audit techniques
There are two categories of CAT : Embedded modules and Special audit records.
Some of the CAT are –
Created By Manish Mathur

35. Created By Manish Mathur
Snapshot
Special audit module built into the system where transaction processing occurs.
It takes images of the transactions of audit significance and stores them in auditor’s file.
Main issues to decide are –
Location of snapshot
Condition to capture the image
Reporting system
Created By Manish Mathur

36. Integrated Test Facility
It involves creation of dummy entity in the client system and processing special audit data against that.
Methods of creation of test pack –
An embed audit module, recognize transaction having certain characteristic. These tagged tr. can be used as test pack.
Auditor may use test data specially prepared for audit.
Created By Manish Mathur

37. Created By Manish Mathur
Method of removing effect of ITF Tr. –
Application system may be programmed to recognize ITF Tr. and ignore them in reporting.
Auditor may submitting additional inputs that reverses the effect of ITF Tr.
Created By Manish Mathur

38. System Control Audit Review File
It involves use of special audit module within system under audit.
It provide continuous monitoring of system’s transactions.
Collected information is stored in special audit file : SCARF.
Application system errors
Policy and procedure variance
System exceptions
Statistical sample
Snapshot and extended records
Profiling data
Performance measurement
Created By Manish Mathur

39. Created By Manish Mathur
Advantages of CAT
Created By Manish Mathur

40. Created By Manish Mathur
Disadvantages of CAT
Created By Manish Mathur

41. Created By Manish Mathur
Hardware Testing
H/w testing is done against FRS and SRS.
Types –
Created By Manish Mathur

42. Auditor’s review of hardware
Review of capacity management and performance evaluation procedure to determine –
Ensure continuous review of performance and capacity.
Whether historical data obtained from : system trouble log, processing schedule, system report, preventive maintenance are used on performance monitoring.
Decision of buy and sell h/w is based on capacity planning and workload forecast.
Created By Manish Mathur

43. Created By Manish Mathur
Review of Hardware Acq. plan to determine –
Mgmt issued a written policy regarding h/w acq.
Criteria for acquisition is laid out.
Procedure is established for acq. approval process.
There is awareness of budget constraint.
Request for acq. Is supported by C/B analysis.
All h/w are purchased thru IS purchase deptt.
Envi. is conducive & space is adequate for new h/w
Acq. Plan considers technology obsolescence.
plan considers lease expiration.
Document for h/w i.e manual, warranty card etc is properly maintained.
Created By Manish Mathur

44. Created By Manish Mathur
Review of change mgmt control to determine –
Changes in h/w are planned and scheduled.
Time for adequate installation and testing.
Operator’s manual is properly updated.
Cross reference between changes and cause.
Programmers and IS staff has been informed of all h/w changes.
Created By Manish Mathur

45. Created By Manish Mathur
Review of preventive maintenance practice–
Understand frequency of PM and compare it with contract.
Vendor compliance with agreement.
Ascertain PM does not have adverse effect in production scheduling.
Check that PM log is maintained.
Ensure PM contract commences when warranty expires.
Verify PM contract has call response time defined.
Created By Manish Mathur

46. Operating system review
Interview IS manager, system programming manager and others regarding –
Process of option selection.
Test procedure for system software
Review and approval procedure fro test results.
Implementation procedure
Documentation requirement
Review the feasibility study
Same selection criteria are applied to all proposals.
Created By Manish Mathur

47. Created By Manish Mathur
Review cost benefit analysis –
Direct financial cost of the product.
Cost of product maintenance.
Hardware capacity requirement.
Training and support requirement.
Impact of the product on the processing.
Impact on data security.
Financial stability of the vendor.
Review control over installation of changed System software –
All updates are implemented.
Installation of changes SS is scheduled when they least impact processing
Created By Manish Mathur

48. Created By Manish Mathur
There is a written plan for testing.
Problems encountered during testing were resolved and changes were re-tested.
Test procedures ensure that changes do not create new problems.
Restoration procedure are in place.
Software must be properly authorised prior moving from test to production environment.
Access to libraries is limited to individual’s need.
Review system software’s maintenance activities –
Changes made to the SS are documented.
Vendor support current version of software.
Created By Manish Mathur

49. Created By Manish Mathur
Review SS documentation –
Installation control statement.
Parameter tables.
Exit definition.
Activity log.
Review SS for adequacy of controls, such as –
Change procedure controls
Authorisation controls
Access privileges controls
Documentation controls
Testing controls
Audit trails
Created By Manish Mathur

50. Created By Manish Mathur
Review authorization document to determine –
Addition, deletion or change to access authorisation is documented.
Attempted violation reporting and response is documented.
Review SS security, to determine –
Procedure have been established to prevent bypass of access control.
Procedure have been established to limit access to system interrupt capability.
Physical and logical access controls are adequate.
Vendor supplied passwords are changed.
Created By Manish Mathur

51. Created By Manish Mathur
Review database supported controls –
Access to shared data is appropriate.
Data organization is appropriate.
Change procedures are established to ensure integrity of DBMS.
Integrity of data dictionary is maintained.
Data redundancy is minimised.
Created By Manish Mathur

52. Created By Manish Mathur
Network Review
Review the LAN, to understand –
LAN Architecture
Cost benefit analysis
LAN topology
LAN components
Internetworking
LAN uses
LAN administrator
LAN users
Created By Manish Mathur

53. Created By Manish Mathur
Review LAN to make an assessment of –
Threat Impact Controls
Review physical access controls –
Ensure that LAN h/w, file server and documentation are located in secured area.
Verify that LAN wiring is physically secured.
Observe LAN file server and verify that it is secure.
Keys to file server facility is controlled.
Obtain copy of key log for the file server room and determine that keys are assigned to appropriate persons.
Select keys held by people and determine that these keys do not permit to access LAN facilities.
Created By Manish Mathur

54. Created By Manish Mathur
Review Environment controls to –
Ensure that LAN file server is protected from electric surges.
Ensure that AC and humidity control system are adequate to maintain temperature.
Ensure that LAN server is equipped with UPS.
LAN file server is free of dust, smoke and pollutants.
Backup disks are protected from environmental damage.
Fire extinguishers are nearby.
Food and beverages are prohibited.
Created By Manish Mathur

55. Created By Manish Mathur
Review Logical access controls to ensure –
Users have unique password, password are change periodically and does not appears on screen while entry.
LAN access should be based on written authorization.
Remote access to the system supervisor should be prohibited.
All log-on attempts should be logged.
LAN supervisor should maintain up-to-date information of all outside communication.
Evaluate LAN server access profile.
Attempt to gain access using unauthorised ID/PWD.
If LAN is connected to an outside source through a modem attempt to gain access to the LAN thru correct and incorrect means.
Created By Manish Mathur