1. Statistical Sampling in Testing Internal Controls
Donald K. McConnell Jr. CPA, CFE, Ph D
The University of Texas at Arlington
9/18/2018

2. Audit Sampling
Application of an audit procedure to < 100% of the items in an account balance or class of transactions (AU )
To form a conclusion about a population by examining only part of the data
Should not be used for balances or transactions likely to contain misstatements (AU )
9/18/2018

3. Circumstances in Which Sampling Does Not Apply (AU 350.32)
Tests of controls that depend primarily on appropriate segregation of duties
Tests that provide no documentary evidence of performance
Small populations: why sample?*
We can’t read every tenth line of board meeting minutes
Testing footings
*if there are three items in a population, why draw up a sampling plan? Just audit everything
9/18/2018

4. Nonstatistical Vs. Statistical Sampling
Both are acceptable under GAAS
Both require professional judgment in planning, performing, and evaluating a sample
Statistical sampling has one distinguishing feature:
Allows us to measure mathematically the uncertainty resulting from examining only part of the data (AU )
i.e., Allows us to quantify sampling risk
9/18/2018

5. What Does This Mean: Quantifying Sampling Risk?
Assume the following:
A population to be tested consists of 100,000 purchase transactions
The population contains only 2 fraudulent entries
However, Both fraudulent entries are randomly selected in our sample of 100 transactions
This would be an example of sampling error arising from sampling risk
Sample result was not representative of the population characteristic: occurrence rate 00.05%
9/18/2018

6. Prior to Sarbanes-Oxley, Use of Statistical Sampling Had Diminished!
June 2002 Accounting Horizons:
223 usable responses from 600 survey instruments mailed, of which:
50% were government auditors
36% were public accountants
Findings:
Only 15% using statistical sampling
12% used DUS (PPSS)
2% used simple random sampling
74% use haphazard selection
3% used block selection
9/18/2018

7. Sarbanes-Oxley Has Rekindled Interest in Use of Statistical Sampling!
A mathematically defensible test result, e.g.:
We can state that we’re 90% or 95% certain our sample result was representative of the population
As of this date, at least 3 of the Big 4 Public Accounting Firms are again using statistical sampling!
9/18/2018

8. Let’s Look at Some Basic Statistical Sampling Concepts
9/18/2018

9. Sampling Risk Vs. Non- Sampling Risk
Sampling risk: the risk that the sample result is not representative of the population
Sampling risk reduced by increasing sample sizes **
Nonsampling risk: everything else that can go wrong in a test, e.g.:
Inappropriate procedures
Failure to recognize misstatements
Reduced by adequate planning and supervision
**If you examined a population 100 percent, there can be no sampling risk.
9/18/2018

10. Sampling Risk: Alpha Risk Concepts (AU 350.12)
Risk of incorrect rejection [of an account balance presenting fairly]
Risk of assessing control risk too high [in a test of controls]
9/18/2018

11. Sampling Risk: Beta Risk Concepts
Risk of incorrect acceptance [of an account balance presenting fairly]
Risk of assessing control risk too low [in a test of controls]
9/18/2018

12. Why Is Alpha Risk Generally of Less Concern Than Beta Risk?
Ordinarily the auditor would expand testing, thus arriving at the correct conclusion
The audit may be less efficient, but nevertheless effective (AU )
In practice, a bad result in testing controls usually causes an external auditor to “drop back 10 punt”
that is, expand year end substantive testing
9/18/2018

13. Always Evaluate Qualitative Aspects of Misstatements
Consider these issues:
Does the item appear to be due to error or fraud?
Does it appear to be isolated or systemic?
Fraud/systemic requires much broader consideration than error/isolated (AU )
9/18/2018

14. Projected Deviation Rates or Misstatements
Where projected sample misstatement or deviations less than tolerable...
Consider risk that such result might be obtained…
Even though true misstatement or deviation rate exceeds tolerable in the population (AU and .41)
That is, there is a 5% or 10% risk your sample result was not representative!
9/18/2018

15. Nonstatistical Sampling
Some auditors select sample items randomly, but evaluate nonstatistically
Forms of nonstatistical selection:
Haphazard selection*
Block selection**
Nonstatistical sampling sizes must approximate what would be obtained from selecting a statistical sample size using reasonable parameters [AICPA Audit Sampling Industry Audit Guide (IAG)]
*selecting sample items without injecting any conscious bias: always selecting from the top of a page would be a problem
** blocks “crossing months” example
9/18/2018

16. Forms of Probabilistic Selection
Must be used to draw statistical inferences when using a statistical sampling approach
Random numbers software
Systematic selection:
Selecting every “n th” item after a random start.
Some auditors avoid using: what if population not randomly arranged?*
A solution: two or three random starts (e.g. three starts selecting 20 items, rather than one start selecting 60 items)
*inventory price test: expensive items always In 0. If you start with random start of 4, then select every 50th item, you would never audit the more important items. Certain transactions always at in the month
9/18/2018

17. Attributes Sampling Plans
Used typically to test compliance with internal controls
Results are always in terms of a percentage projection of rate of occurrence in a population
Do not test dollar amounts
9/18/2018

18. Types of Attributes Sampling Plans
Acceptance sampling (obsolete)
Fixed sample size attributes plans (probably most commonly used in practice)
Stop or go sampling
Discovery sampling (for fraud examination)
9/18/2018

19. Attributes Sampling Typically Used in Tests of Controls for:
Voucher processing in A/P
Cash disbursements
Billing systems
Payroll and related personnel policy systems
Inventory pricing
Fixed asset additions
Depreciation computations
9/18/2018

20. How Do I Use fixed Sample Size Attributes Sampling to Test Controls?
9/18/2018

21. Assume the following:
Auditor wants to test controls in the acquisitions and payments cycle for the 9 month period 1/2/xx-9/30/xx
What is an attribute?
Auditor must define attributes of interest
N=100,000 checks issued in relevant 9 month period
9/18/2018

22. The Auditor Must Specify Parameters
Risk of assessing control risk too low [risk of over-reliance on internal controls (ARO)]
Tolerable error rate (TER)
Expected population error rate (EPER)
Also first check number issued 1/2/xx and last check number issued 9/30/xx
9/18/2018

23. Common Parameters in Practice
95-5-0, which is interpreted as:
ARO= 5% [Complement of confidence level is risk of overreliance on IC’s]
TER= 5%
EPER= 0% for good internal controls environment: a common assumption
9/18/2018

24. Issues Concerning Tolerable Error Rates Selected
Low TER’s (e.g 2-3%) selected:
Where IC’s have highly significant effect on account balances, i.e.:
Auditor wants more precise estimate
Tight precision: bigger sample sizes
High TER’s (e.g 5-10%) selected:
Where IC’s have less significant effect on account balances
Provides less precise estimate
Looser precision: smaller sample sizes
9/18/2018

25. Issues Concerning Risk of Overreliance
5% would be relatively high IC’s reliance
10% would be moderate IC’s reliance
20% would be low IC’s reliance
9/18/2018

26. Concerning These Parameters:
How does the auditor determine EPER?
Examples:
Use last year’s actual sample result
First time audit: pilot sample of 50 items, randomly selected (AICPA IAG)
Even a “WAG” is acceptable for attributes plans
Look for example at : n=114
9/18/2018

27. What Attributes Would We Want to Test? Examples:
Is purchase entry supported by a vendor’s invoice in that amount?
Is there an authorized P.O. signed by purchasing agent?
Is there a receiving report from the dock?
Was account classification correct?
Was vendor an authorized vendor?
Was final approval for payment authorized?
9/18/2018

28. Let’s do the Test Laptop computer and audit software would be used
What if you are doing a branch audit in West Texas and your hard drive crashed!
You don’t even need a computer!
Recall test parameters were
T 14-8 in H.O.: Sample size determination
n= 59 checks randomly selected, which we will round to 60 for T14-9 purposes
Tables on page 429 in 9th ed.
9/18/2018

29. Some Issues concerning the Sample Size
Population size has little effect on attributes plan sample sizes
i.e., yields a sample size of 59 (or 60) regardless of whether the population is 1,000 items or 100 million items!
If a pilot sample was used to determine EPER, those items can be the first 50 items of your plan sample size
(We need just need another 10 items randomly selected, in this case)
9/18/2018

30. Evaluating Sample Results
Assume no compliance deviations for 9 of 10 tested attributes (T14-9 in H.O.) in the sample of 60 transactions
4.9% is “upper error bound” or CUER
What does this mean?
We are 95% certain true rate of controls deviations doesn’t exceed 4.9%
Is this acceptable?
Yes, tolerable error (TER) was 5%
And, can’t do better than no deviations from IC’s!
Really a two tailed test, but not concerned with how low error rate might be!
9/18/2018

31. Evaluating Sample Results (Con.)
Assume for one item in sample no P.O. could be found
Can we select another check number randomly as substitute?
NO!!!! It’s a compliance deviation
What is projected error rate upper bound?
7.7%
Is this acceptable?
No, tolerable error (TER) was 5%
9/18/2018

32. What Options Do We Have? Expand sample size?
Only works if bad result was likely caused by sampling error!
Unless sampling error, very likely to find at least one more purchase transaction with no supporting P.O.
9/18/2018

33. What If We Expanded Sample Size?
What would sample size need to be?
100 (i.e., 40 additional sample items)
What if we found one more “no P.O. item” in expanded sample?
Projected error rate is 6.2%: still unacceptable!
Need to examine population more extensively for that control [public co.]
Do not rely on that control; instead do more extensive substantive testing in that area [non-public co.]
9/18/2018

34. Does a 5% TER Bother You?
That’s what we typically used at KPMG years ago: when testing controls; however:
We were not opining on controls comprehensively under SOX
The external auditor supplements tests of controls with substantive tests of balances
You’ll probably want to use lower TER’s, e.g. 2-3%, especially for SOX purposes
Sample sizes will be larger due to tighter precision!
9/18/2018

35. Some Final Words on Selecting Attributes
Don’t define too many attributes: the process gets unwieldy
Don’t define too few attributes: you’ll be evaluating disparate circumstances
9/18/2018

36. Be Sure You’re Sampling from the Right Population in Testing Assertions!
Completeness: sample items are receiving reports, traced to system entry [source document to recorded entry
Existence: sample from evidence of system entry; trace to receiving reports [recorded entry to source document]
9/18/2018

37. Benefits of Using Software
No need to round initial sample sizes (n= 59 rounded to 60)
No need to interpolate sample results, if we had used n= 59, vs 60
Expanded sample size (in our example) would have been smaller
Recall we used 100 from T14-9
Software would’ve calculated an actual sample of about 95
Smaller sample sizes if initial sample size > 10% of population size
9/18/2018

38. Special Considerations: Tips and How to Avoid Invalidating Your Tests
Randomly select additional sample items
Dealing with voided transactions
Sample items for which the test is inapplicable
Excessive controls deviations found early in your testing
Sample items which cannot be located
[Above per IAG]
9/18/2018

39. Randomly Select Additional Sample Items
It’s a good idea to randomly select more sample items than test parameters dictate
Why? Voided transactions can be in your sample
IMPORTANT: additional items used as replacements must be used in order in which the random numbers were generated!
9/18/2018

40. An Example:
Assume test parameters indicate your initial sample size should be 100 items
You might randomly select items [5 -10 extras]
Assuming two voided sample items, replacement sample items should be the 101st and 102nd items in random selection order
9/18/2018

41. Dealing with Voided Transactions
Examine to insure properly voided, and not a controls deviation
If properly voided, use an additional replacement item
Replacement item must be in order in which the random numbers were generated
9/18/2018

42. Sample Items for Which a Test Is Inapplicable
Assume the attribute being tested is “does transaction have supporting receiving report”
Assume a voucher for telephone expense has been selected in the sample
There would be no receiving report
Replace the item with another replacement random number
9/18/2018

43. Excessive Deviations Found Early in Evaluating A Sample
Even if no additional deviations from control were found, the results would exceed TER, and would not support planned reliance
You wouldn’t want to continue examining sample items for that control
Perform an in-depth analysis for problems with that control
9/18/2018

44. How to Deal with Sample Items Which Cannot Be Located
The item should be considered a controls deviation in evaluating sample result
Do not substitute with a replacement random number for the item!
9/18/2018

45. Other Useful Attributes Sampling Plans
Stop or Go Sampling
Discovery Sampling
9/18/2018

46. Stop or Go Sampling Requires estimates of only ARO and TER
No need to estimate EPER!
Highly effective when zero or low rates of compliance deviations are expected (very good controls)
Typically results in smaller sample sizes than with fixed sample size attributes plans
9/18/2018

47. Stop or Go Sampling (con.)
Sample is taken in steps, with each step conditional on the results of the previous step (IAG, p. 35)
Some simple calculations required to construct steps
Where deviations found, projected error rates more conservative (higher) than with fixed sample size plans
See Guy, et. al. for more
9/18/2018

48. Discovery Sampling For fraud investigations
e.g., to discover at least one fraudulent disbursement from a population when the rate of fraud is at an extremely low rate
Can result in very large sample sizes (300 sample items would not be unusual) due to stringent TER’s (0.2% or less)
9/18/2018

49. Evaluating the Discovery Sampling Plan
Having specified TER and ARO, auditor draws required sample size
Sample transactions are examined until a single instance of fraud is identified
If no fraudulent transactions found, auditor can conclude that if fraud exists, it is it a rate < TER
9/18/2018

50. Evaluating the Discovery Sampling Plan (con.)
Once first incidence of fraud is found in sample, auditor can cease auditing sample items, if sole objective is fraud discovery*
Hypothesis of fraud has been confirmed
Need to examine entire population extensively
See Dan Guy, et al. for more
*The auditor may continue with sample selection, even though an instance of fraud has been found. Auditor could then use attributes tables [ e.g stop or go] to project a fraud rate in the population
9/18/2018

51. Useful References
American Institute Of Certified Public Accountants (AICPA) Audit Sampling. New York, N. Y.
Arens, A.A., R.J. Elder and M.S. Beasley Auditing and Assurance Services: an Integrated Approach, 9th edition. Prentice-Hall. Upper Saddle River, N.J.
Guy, Dan M., D. R. Carmichael and R. Whittington Audit Sampling: an Introduction, 5th edition. John Wiley and Sons. New York, N. Y.
9/18/2018