We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byDerick Crutchley
Modified over 2 years ago
© 2010 Deloitte Touche Tohmatsu A statistician’s view on the EU sampling guidelines CB Conference, June 1st, 2010, Budapest Paul van Batenburg
© 2010 Deloitte Touche Tohmatsu A statistician’s view on the EU sampling guidelines 2 The situation: 1.PA executes on-the-spot checks on samples from claims. These risk based and random samples are performed for different support schemes 2.CB investigates whether the PA sample checks can be relied upon by: a)Performing a MUS sample on the population of items that were randomly selected by the PA b)Choosing a sample size using sheet A1 to compare the PA’s audit values with the CB audit values c)Evaluating by using sheet A2 confronting the most likely absolute error rate with a 2% materiality threshold Questions I would like discuss (in reverse order): 1.Is this the right sample from the right population? 2.Is this the right sample size? 3.Is this the right evaluation? If one or more answers turn out to be negative, a new question comes up: ‒ Is there an alternative method to recommend? I believe so, and will briefly introduce: ‒ AOQL ‒ Regression estimates
© 2010 Deloitte Touche Tohmatsu Presenter Paul van Batenburg Statistician, not an auditor Deloitte Netherlands “Effective and efficient audit by using statistical methods and techniques” Relevant assignments: NL Ministry of Economic Affairs, NL Ministry of Agriculture (PA’s and CB), EU DG Budget, Regione Puglia (goto: CB Conference for Excel sheet and this presentation)www.steekproeven.eu 3
© 2010 Deloitte Touche Tohmatsu Question 3: evaluation Do the percentages that make up (j) have the same denominator? MUS samples are not equipped to evaluate understatements; taking their absolute values does not solve that The sample was designed for a 95% maximum error to meet 2% materiality, so why confront the most likely error with this threshold? 4
© 2010 Deloitte Touche Tohmatsu Question 2: sample size The audit risk model was designed to economize on substantive testing in situations where controls were in place to prevent errors. A low required confidence from substantive testing combined with a high expected error rate is inconsistent with the rationale behind the ARM The form of the ARM used is additive: total confidence level = confidence level from control testing plus confidence level from substantive testing The AICPA audit risk model (as used by audit firms globally) is multiplicative: audit risk = control risk * detection risk, or total confidence level = 1-(1-confidence level from control testing)*(1- confidence level from substantive testing) I personally believe both models are flawed, but the AICPA model is better since it cannot yield a required confidence level of 0%, conform ISA 330 par.20 and A 42 Error rates mentioned are percentages of materiality and not percentages of the population Sample sizes should have been rounded up, see for an Excel sheet 5
© 2010 Deloitte Touche Tohmatsu Question 1: population and sample (in bold: what I read from the guidance) If the CB’s sample is meant to confirm the accuracy of the PA’s statistics The sample should be selected from all risk based and random samples by the PA The sample should be a line item sample The PA’s audit value should be compared to the CB’s audit value A materiality level should be chosen instead of the 2% that is meant for financial audits If the CB’s sample is meant to confirm with 95% assurance that the error rate in the payments is below 2% The sample may be selected from the PA random sample items (not MUS, not the risk based items) The CB sample should be MUS The PA’s recorded value should be compared to the CB’s audit values The (95% reliable maximum) error rate should be compared to 2% materiality In my view, 2 approaches are mixed: conclusion and comparison do not match population, sample selection and materiality definition 6
© 2010 Deloitte Touche Tohmatsu On understatements A MUS sample of 1 from this population of 300 in 3 transactions based on recorded values gives probabilities of selection of 1/3 for each item If no errors had been made, the 2nd transaction would have had a smaller probability: MUS samples are an excellent tool to audit on overstatements as the error increases the chance of being caught And the 3rd would have had a larger probability: MUS samples are not equipped to audit on understatements Either use MUS from the difference between a maximum value and the recorded value (completeness of VAT invoiced) or use a Line Item Sample Taking absolute error amounts yields an incorrect population audit value: understated amounts are no part of the population so it does not contain understatement errors 7
© 2010 Deloitte Touche Tohmatsu Alternative: relying on CB sample The horizontal axis shows the PA’s audit values and the vertical axis shows the CB’s audit values in the CB’s sample from all PA sample items The statistical technique of regression yields a line that best expresses the relation to predict CB audit values from PA audit values for all other PA sample items (Cochran, 1977) If only small differences (+ and - !) are found, the correlation will be high and the CB can rely on the PA’s sample checks When large differences occur, the PA may not have followed the audit protocol and the CB should consider rejecting the PA sample The decision to accept or reject the PA sample needs not to be taken for the sample as a whole but can be applied in batches of, say, 10% of the total PA sample based on AOQL (Dodge and Romig, 1933, 1957) References: Vera Raats, Monotone missing data and repeated controls of fallible auditors, Ph.D. thesis, Tilburg University,
© 2010 Deloitte Touche Tohmatsu Questions? 9
McGraw-Hill/Irwin © The McGraw-Hill Companies 2010 Audit Sampling: An Overview and Application to Tests of Controls Chapter Eight.
PLANNING THE AUDIT Individual audits must be properly planned to ensure: Appropriate and sufficient evidence is obtained to support the auditors opinion;
Copyright © 2010 Pearson Education, Inc. Slide
21-1 Copyright 2010 McGraw-Hill Australia Pty Ltd PowerPoint slides to accompany Croucher, Introductory Mathematics and Statistics, 5e Chapter 21 Hypothesis.
1 Audit Risk Week Risk Assessment in Planning AR = IR x CR x DR To meet desired level of Audit Risk Need to assess each component IR & CR can be.
1 AU 350 SAS 111 Audit Sampling C Delano Gray June 14, 2008.
Audit Sampling Chapter 09 McGraw-Hill/Irwin Copyright © 2012 by The McGraw-Hill Companies, Inc. All rights reserved.
Chapter 4 Evaluating Classification and Predictive Performance 1.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlikeLicense. Your use of this material constitutes acceptance of that license.
Sample size calculations for cross-classified models William Browne, Mousa Golalizadeh and Richard Parker University of Bristol.
“Students” t-test. Recall: The z-test for means The Test Statistic.
Direct Time study: Selecting and timing the job First step in time study is to select the job to be studied. There is always a reason why a particular.
Introduction to Hypothesis Testing. What is a Hypothesis Test? A hypothesis test is a statistical method that uses sample data to evaluate a hypothesis.
1 Audit Evidence Week Audit evidence What evidence is required? Auditors should obtain sufficient appropriate audit evidence to be able to draw.
Sampling and monitoring the environment Marian Scott Sept 2006.
Quantitative Methods Lecture 3 Populations and Samples Statistics books often assume we already know the true mean or the true variance of the whole population.
CS1512 Foundations of Computing Science 2 Lecture 5 Inferential statistics.
Sampling Design & Procedure. Population The aggregate of the all the elements, sharing some common set of characteristics that comprises the universe.
Chapter 10 Substantive tests of transactions and balances 10-1 Copyright 2010 McGraw-Hill Australia Pty Ltd PPTs t/a Auditing and Assurance Services in.
Effect Sizes and Power Review. Statistical Power Statistical power refers to the probability of finding a particular sized effect Specifically, it is.
Innovation Surveys: Advice from the Oslo Manual National training workshop Amman, Jordan October 2010.
SECTION 13.2 Comparing Two Proportions. In this scenario, we desire to compare two populations or the responses to two treatments based on two independent.
FINANCIAL AUDIT METHODOLOGY PETER CARLILL UK NATIONAL AUDIT OFFICE.
Internal Assessment Your overall IB mark (the one sent to universities after the IB test) in any IB science course is based upon two kinds of assessments.
Chapter 10: Introduction to Inference If you believe in miracles, head for the Keno lounge. Jimmy the Greek.
1 Assessing Evidence Reliability In Performance Audits NSAA April 14, 2008.
Sample Size calculations in multilevel modelling William Browne University of Nottingham (With thanks to Mousa Golalizadeh and Lynda Leese)
Z Test & T Test PRESENTED BY : GROUP 1 MD SHAHIDUR RAHMANROLL# 003 MD AMINUL ISLAMROLL# 007 MRS ROZINA KHANAMROLL# 038.
Sample Size calculations for multilevel models (Part II) William Browne and Mousa Golalizadeh Department of Clinical Veterinary Sciences University of.
© 2016 SlidePlayer.com Inc. All rights reserved.