Presentation is loading. Please wait.

Presentation is loading. Please wait.

Chapter [4] Created By Manish Mathur. Testing : Definition Testing is the process of assessing Correctness, Completeness and Quality of system. Testing.

Similar presentations


Presentation on theme: "Chapter [4] Created By Manish Mathur. Testing : Definition Testing is the process of assessing Correctness, Completeness and Quality of system. Testing."— Presentation transcript:

1 Chapter [4] Created By Manish Mathur

2 Testing : Definition Testing is the process of assessing Correctness, Completeness and Quality of system. Testing is a process to determining whether the controls are adequately protect the system. Types – – SUBSTANTIVE : To prove the integrity of the actual processing and To ensure that processes work to produce reliable results. – COMPLIANCE : To ensure that system controls adhere to management directives. Created By Manish Mathur

3 Phases – – PLANNING : Here auditor determines the way to collect the evidence to achieve objectives IS audit. – Testing : Here auditor tests the effectiveness of IS controls. – Reporting : Here auditor concludes and reports the result of audit to the management. Created By Manish Mathur

4 Audit Planning Planning occurs throughout the audit and includes the following activities – Obtain an understanding of the entity and its operations. Obtain an understanding of internal controls. Assess the risk. Design the nature, extent and timing of audit procedure. Created By Manish Mathur

5 Auditor uses the concept of Materiality and Significance. According to these concepts – Auditor is not required to spend resources on item that are not material and significance i.e. those that would not affect the judgment of users of audit report. Created By Manish Mathur

6 Testing Methodology - – Auditor must find testing methods to determine that controls are effective. This may include reviewing documentary evidence, conducting personnel interview and personal observation. File interrogation - – Auditor must browse directories of PC to investigate use developed application files. Some important decisions before testing begins ~ Created By Manish Mathur

7 Test pack - – Auditor uses valid and invalid data to test the ability to prevent, detect, and correct errors. – The intensity and extent of testing depends upon importance of the application. Automated tools – – Audit team can use GAS (Generalised Audit Software) to do sampling, data extraction, summarizing and reporting. Created By Manish Mathur

8 Tasks – Understanding the entity and key business process. Understanding entitys network structure. Identify key area of audit interest. Assessing IT risks. Identify critical control points Understanding of IS controls. Performing other audit procedures. Created By Manish Mathur

9 Types of Control Audit Test Financial Audit – If IS control audit is performed as a part of financial audit, the auditor understand the controls over financial reporting to assess the risk of misrepresentation. Performance Audit – If IS control audit is performed as a part of performance audit, the auditor should evaluate the design and operating effectiveness of all the controls. Created By Manish Mathur

10 The following factors assist the auditor to determine which audit procedure to use to collect audit evidences – The extent to which internal controls are to be tested. The availability of evidences outside the system. The relationship of system controls and data reliability. Audit Objectives. Created By Manish Mathur

11 Key areas are those applications and files that are critical in achieving audit objectives. – For financial audit : key financial applications – For performance audit : all key system applications For each key area, auditor should document – – Operational location – Significant components (h/w, s/w) – Other support systems/resources – Prior audit reports Created By Manish Mathur

12 Identify entity wide(component level) controls and determine that they are effectively designed and implemented. Identify business process level controls for key application and check for effectiveness. Any internal audit or third party reviews performed during last year. Managements plan for corrective action for the IS weakness and IS control weakness. Status of the prior years findings. Created By Manish Mathur

13 Review any significant security incident for the last year. Review entitys security plan. Review risk assessment for the system. Certification and accreditation document for system. Review BCP and DRP. Review description of outsourced activities. Relevant laws and regulation and their relation with audit. Procedure to consider risk of fraud that could affect audit objectives. Created By Manish Mathur

14 Plan audit resources. Review current multi year testing plan. Communication with entitys management. Audit procedure for service organization audit. Decision to use the work of others. Develop audit plan : objectives, scope, methodology etc. Decision to reduce testing of IS controls. Auditor should document all these information as their Preliminary Investigation Documentation. Created By Manish Mathur

15 Auditor uses information obtained in the investigation phase to test the effectiveness of IS controls. While performing audit, auditor should assess evidence to identify any revision needed in audit plan. For example – – If significant weakness is found the auditor may decide to perform less testing in remaining areas. Created By Manish Mathur

16 Auditor determine effectiveness of controls at following levels – Processes designed to achieve control activities. E.g. Configuration mgmt., database updation, authorisation process etc. Entity/ Component Processes designed to control resources related to general support system E.g. Network, Operating system, Infrastructure application ( , browser, utilities i.e. not directly related to business process. System Policies and procedures for controlling specific business processes. E.g. General controls i.e. security guard, CCTV, door access locks etc. Business process application Created By Manish Mathur

17 Critical control point is that component of system which is of significant importance. Auditor tests controls related to the component, its operating system and its applications. For e.g. – Router. Auditor tests the control related to the router itself, its operating system and applications. Created By Manish Mathur

18 Auditor should conduct test of those control technique that are effective in operation. To do so the best way is to test control in tired basis. Starting with – – Entity wide controls – System level controls – Business process application level controls – Data management controls Ineffective IS controls at each tier generally prevent effective control at the subsequent tier. Created By Manish Mathur

19 General controls General controls at entity wide and system level can be tested using techniques such as Inquiry, Observation, Inspection and re-performance thru test software. After reaching favorable conclusion on general controls at these level auditor test general control at business process application level. If general control are not effectively operating then auditor should- – Determine nature and extent of risk resulting from ineffectiveness. – Identify and test any manual controls as compensating control. Created By Manish Mathur

20 Auditor tests those application controls that achieve control objectives when other general controls are ineffective. If application controls are not likely to be effective auditor – – Understand the risk in terms of impact on audit objectives. – Identify any manual controls that achieve the control objectives. If in the previous year controls were ineffective and management have not significantly improved it, the auditor need not to test them. Created By Manish Mathur

21 To keep appropriateness of control tests the auditor should perform appropriate mix of audit procedure that includes the following – Inquiries of IT and management personnel Questionnaires Review documentation of control procedures Inspection of approvals(authorisation) Analysis of system information(configuration) Analysis of output (accuracy of processing) Review of data file Re-performance of the control (use of test data) Created By Manish Mathur

22 Where auditor regularly performs control audit of the entity, the auditor may develop a multi year plan for control audit. These plan should cover not more then 3 years and include schedule and scope of assessment. Under multi year plan each control is tested at least once during the multi year period. Created By Manish Mathur

23 This concept allow auditor to test controls on risk basis rather then testing every control every year. For example a multi year plan for an entity with 7 applications might include comprehensive test of 2-3 application annually. Multi year plans are not appropriate in all situations. For example – – They are appropriate for first time audit. – They are not appropriate where audit has not been tested within a recent period. – For entity that do not have strong entity wide controls. Created By Manish Mathur

24 Documentation of control testing phase Information gathered during testing phase should be documented. This include – Understanding of IS. IS control objectives and activities. Description of control techniques used by entity. Specific test performed. Description of nature, extent and timing of test. Evidence of effectiveness of controls. If ineffective then compensating controls. Auditors conclusion about effectiveness of controls. For each weakness; material weakness, significant deficiency or just deficiency. Created By Manish Mathur

25 Audit Reporting After completing testing auditor summarizes the audit result, draw conclusion on the control weakness. Auditor prepares this report on entitywide, system and BPA level collectively. Such documentation may be developed as the audit progresses, allowing auditor to demo. that the weakness exist and can be exploited. Auditor should also document the potential impact of weakness on completeness, accuracy, validity, confidentiality of system. Created By Manish Mathur

26 Some audit terms Substantive testing Substantive testing is used to determine the accuracy of information generated by a process. Auditor generate and process test data to verify the processing steps. Where controls are evaluated as ineffective, substantive testing may be required. Auditor uses CAAT to generate test pack and conduct the test. Created By Manish Mathur

27 Some audit terms.. Analysis Interviews and tests provide the raw facts for drafting a audit report but does not guarantee to produce a quality audit report. Analysis is important to convert this raw material into finished product. Timely analysis gives the auditor time to conduct further test and allow more time for corrective actions. Thorough analysis includes the following 4 steps – Created By Manish Mathur

28 Steps Re- examination Cause of deviation Materiality Exposure conclusion Created By Manish Mathur

29 Step 1 : Re-examination – The two factors to be re-examined are : Standard and Facts. – Standard are the rules, procedures and practices that defines how an operation under audit should function. – The standards must be clearly understood by the auditors, because wrong understanding leads to incorrect findings. – Facts are evaluated after standards are reviewed. For accuracy the sample should be Large enough to reflect behavior of population. Representative of current control activity Created By Manish Mathur

30 Step 2 : Cause of Deviation – After understanding standards and facts, auditor identify the causes of the deviation. – Determining the cause is like answering the following questions – Who (responsible) What (initiating event) Where (system component) Why (contributing factor) When (timing) – Cause determination helps to identify exposure and formulating recommendations. Created By Manish Mathur

31 Step 3 : Exposure and Materiality These are consequences of deviation. Exposure is the potential loss, harm, damage, theft or inefficient use and Materiality is a qualitative judgment about whether a deviations frequency of occurrence and degree of exposure are significant enough for the deviation to be corrected. Degree of exposure is related to Proximity and Severity of risk. Proximity refers to the extent of asset availability to the users or environment. Limited access – less proximity. Created By Manish Mathur

32 Severity refers to the amount of loss. The greater the value of asset and higher the proximity, higher will be severity. Frequency refers to how often the deviation will occur. With understanding of Materiality and Exposure auditor can identify why corrections should take place. Created By Manish Mathur

33 Step 4 : Conclusion Conclusions are auditors opinion on, whether the audit subject area meets the audit objectives. Conclusions must be supported by factual evidences. Created By Manish Mathur

34 Concurrent audit techniques There are two categories of CAT : Embedded modules and Special audit records. Some of the CAT are – Created By Manish Mathur

35 Special audit module built into the system where transaction processing occurs. It takes images of the transactions of audit significance and stores them in auditors file. Main issues to decide are – – Location of snapshot – Condition to capture the image – Reporting system Created By Manish Mathur

36 It involves creation of dummy entity in the client system and processing special audit data against that. Methods of creation of test pack – – An embed audit module, recognize transaction having certain characteristic. These tagged tr. can be used as test pack. – Auditor may use test data specially prepared for audit. Created By Manish Mathur

37 Method of removing effect of ITF Tr. – – Application system may be programmed to recognize ITF Tr. and ignore them in reporting. – Auditor may submitting additional inputs that reverses the effect of ITF Tr. Created By Manish Mathur

38 System Control Audit Review File It involves use of special audit module within system under audit. It provide continuous monitoring of systems transactions. Collected information is stored in special audit file : SCARF. – Application system errors – Policy and procedure variance – System exceptions – Statistical sample – Snapshot and extended records – Profiling data – Performance measurement Created By Manish Mathur

39 Advantages of CAT 1 Reduction in the cost of audit. 2 Reduction in the time of audit. 3 Improvement in the quality of audit. 4 Comprehensive and detailed audit. 5 Surprise test capability 6 Information to system staff on meeting objectives 7 Training to new users Created By Manish Mathur

40 1 Auditor should be able to obtain resources. 2 Useful where audit is involved in development. 3 Audit need to have computer background. 4 Useful where audit trail is less visible. 5 Useful where cost of error is very high. 6 Effective when system is stable. Created By Manish Mathur

41 Hardware Testing H/w testing is done against FRS and SRS. Types – Function testing User interface testing Usability testing Compatibility testing Model based testing Error exist testing User help testing Security testing Capacity testing Performance testing Reliability testing Installation testing Maintenance testing Accessibility testing Created By Manish Mathur

42 Auditors review of hardware Review of capacity management and performance evaluation procedure to determine – – Ensure continuous review of performance and capacity. – Whether historical data obtained from : system trouble log, processing schedule, system report, preventive maintenance are used on performance monitoring. – Decision of buy and sell h/w is based on capacity planning and workload forecast. Created By Manish Mathur

43 Review of Hardware Acq. plan to determine – – Mgmt issued a written policy regarding h/w acq. – Criteria for acquisition is laid out. – Procedure is established for acq. approval process. – There is awareness of budget constraint. – Request for acq. Is supported by C/B analysis. – All h/w are purchased thru IS purchase deptt. – Envi. is conducive & space is adequate for new h/w – Acq. Plan considers technology obsolescence. – plan considers lease expiration. – Document for h/w i.e manual, warranty card etc is properly maintained. Created By Manish Mathur

44 Review of change mgmt control to determine – – Changes in h/w are planned and scheduled. – Time for adequate installation and testing. – Operators manual is properly updated. – Cross reference between changes and cause. – Programmers and IS staff has been informed of all h/w changes. Created By Manish Mathur

45 Review of preventive maintenance practice – – Understand frequency of PM and compare it with contract. – Vendor compliance with agreement. – Ascertain PM does not have adverse effect in production scheduling. – Check that PM log is maintained. – Ensure PM contract commences when warranty expires. – Verify PM contract has call response time defined. Created By Manish Mathur

46 Interview IS manager, system programming manager and others regarding – – Process of option selection. – Test procedure for system software – Review and approval procedure fro test results. – Implementation procedure – Documentation requirement Review the feasibility study – Same selection criteria are applied to all proposals. Created By Manish Mathur

47 Review cost benefit analysis – – Direct financial cost of the product. – Cost of product maintenance. – Hardware capacity requirement. – Training and support requirement. – Impact of the product on the processing. – Impact on data security. – Financial stability of the vendor. Review control over installation of changed System software – – All updates are implemented. – Installation of changes SS is scheduled when they least impact processing Created By Manish Mathur

48 – There is a written plan for testing. – Problems encountered during testing were resolved and changes were re-tested. – Test procedures ensure that changes do not create new problems. – Restoration procedure are in place. – Software must be properly authorised prior moving from test to production environment. – Access to libraries is limited to individuals need. Review system softwares maintenance activities – – Changes made to the SS are documented. – Vendor support current version of software. Created By Manish Mathur

49 Review SS documentation – – Installation control statement. – Parameter tables. – Exit definition. – Activity log. Review SS for adequacy of controls, such as – – Change procedure controls – Authorisation controls – Access privileges controls – Documentation controls – Testing controls – Audit trails Created By Manish Mathur

50 Review authorization document to determine – – Addition, deletion or change to access authorisation is documented. – Attempted violation reporting and response is documented. Review SS security, to determine – – Procedure have been established to prevent bypass of access control. – Procedure have been established to limit access to system interrupt capability. – Physical and logical access controls are adequate. – Vendor supplied passwords are changed. Created By Manish Mathur

51 Review database supported controls – – Access to shared data is appropriate. – Data organization is appropriate. – Change procedures are established to ensure integrity of DBMS. – Integrity of data dictionary is maintained. – Data redundancy is minimised. Created By Manish Mathur

52 Network Review Review the LAN, to understand – – LAN Architecture – Cost benefit analysis – LAN topology – LAN components – Internetworking – LAN uses – LAN administrator – LAN users Created By Manish Mathur

53 Review LAN to make an assessment of – ThreatImpactControls Review physical access controls – – Ensure that LAN h/w, file server and documentation are located in secured area. – Verify that LAN wiring is physically secured. – Observe LAN file server and verify that it is secure. – Keys to file server facility is controlled. – Obtain copy of key log for the file server room and determine that keys are assigned to appropriate persons. – Select keys held by people and determine that these keys do not permit to access LAN facilities. Created By Manish Mathur

54 Review Environment controls to – – Ensure that LAN file server is protected from electric surges. – Ensure that AC and humidity control system are adequate to maintain temperature. – Ensure that LAN server is equipped with UPS. – LAN file server is free of dust, smoke and pollutants. – Backup disks are protected from environmental damage. – Fire extinguishers are nearby. – Food and beverages are prohibited. Created By Manish Mathur

55 Review Logical access controls to ensure – – Users have unique password, password are change periodically and does not appears on screen while entry. – LAN access should be based on written authorization. – Remote access to the system supervisor should be prohibited. – All log-on attempts should be logged. – LAN supervisor should maintain up-to-date information of all outside communication. – Evaluate LAN server access profile. – Attempt to gain access using unauthorised ID/PWD. – If LAN is connected to an outside source through a modem attempt to gain access to the LAN thru correct and incorrect means. Created By Manish Mathur


Download ppt "Chapter [4] Created By Manish Mathur. Testing : Definition Testing is the process of assessing Correctness, Completeness and Quality of system. Testing."

Similar presentations


Ads by Google