1. 2008 FRAUD FROUM MAY 14-16, 2008 / HYATT REGENCY COCONUT POINT RESORT & SPA / BONITA SPRINGS, FL, USA Detection ofFraud Detection of Fraud Patterns Using May 16, 2008 Digital Analysis

2. Slide 2 Session objectives 1.Understand why and how 2.Understand statistical basis for quantifying differences 3.Identify ten general tools and techniques 4.Understand use of Excel 5.How pattern detection fits in

3. Slide 3 Overview Fraud patterns detectable with digital analysis Basis for digital analysis approach Usage examples Continuous monitoring Business analytics Using Excel

4. Slide 4 The Why and How Three brief examples IIA Guidance Paper Auditors “Top 10” Process Overview Who, What, Why, When & Where Objective 1

5. Slide 5 Example 1 Wake County Transportation Fraud Supplier Kickback – School Bus parts $5 million Jail sentences Period of years Objective 1a

6. Slide 6 Too little too late Understaffed internal audit Software not used Data on multiple platforms Transaction volumes large Objective 1a

7. Slide 7 Preventable Need structured, objective approach Let the data “talk to you” Need efficient and effective approach Objective 1a

8. Slide 8 Cost of six types of AIDS drugs Example 2 Objective 1a

9. Slide 9 Typical Prescription Patterns Example 2 Objective 1a

10. Slide 10 Prescriptions by Dr. X Example 2 Objective 1a

11. Slide 11 Off-label use Serostim –Treat wasting syndrome, side effect of AIDS, OR –Used by body builders for recreational purposes –One physician prescribed $11.5 million worth (12% of the entire state) Example 2 Objective 1a

12. Slide 12 Revenue trends Example 3 Objective 1a

13. Slide 13 Dental Billings Example 3 Objective 1a

14. Slide 14 Guidance Paper A proposed implementation approach “Managing the Business Risk of Fraud: A Practical Guide” http://tinyurl.com/3ldfza http://tinyurl.com/3ldfza Five Principles Fraud Detection Coordinated Investigation Approach Objective 1b

15. Slide 15 Managing the Business Risk of Fraud: A Practical Guide Exposure draft of IIA, AICPA and ACFE Exposure draft issued 11/2007 Section 5 – Fraud Detection Objective 1b

16. Slide 16 Section 5 – Fraud Detection Detective Controls Process Controls Anonymous Reporting Internal Auditing Proactive Fraud Detection Objective 1b

17. Slide 17 Proactive Fraud Detection Data Analysis to identify: –Anomalies –Trends –Risk indicators Objective 1b

18. Slide 18 Specific Examples Cited Journal entries – suspicious transactions Identification of relationships Benford’s Law Continuous monitoring Objective 1b

19. Slide 19 Data Analysis enhances ability to detect fraud Identify hidden relationships Identify suspicious transactions Assess effectiveness of internal controls Monitor fraud threats Analyze millions of transactions Objective 1b

20. Slide 20 Peeling the Onion Fraud Items Possible Error Conditions Population as Whole Objective 1c

21. Slide 21 Fraud Pattern Detection Target Group Round Numbers Benford’s Law GapsUnivariateDuplicatesDay of WeekHolidayTrend LineStratification Market Basket Objective 1d

22. Slide 22 Digital Analysis (5W) Who What Why Where When Objective 1e

23. Slide 23 Who Uses Digital Analysis Traditionally, IT specialists With appropriate tools, audit generalists (CAATs) Growing trend of business analytics Essential component of continuous monitoring Objective 1e

24. Slide 24 What - Digital Analysis Using software to: –Classify –Quantify –Compare Both numeric and non- numeric data Objective 1e

25. Slide 25 How - Assessing fraud risk Basis is quantification Software can do the “leg work” Statistical measures of difference –C–Chi square –K–Kolmogorov-Smirnov –D–D-statistic Specific approaches Objective 1e

26. Slide 26 Why - Advantages Automated process Handle large data populations Objective, quantifiable metrics Can be part of continuous monitoring Can produce useful business analytics 100% testing is possible Quantify risk Repeatable process Objective 1e

27. Slide 27 Why - Disadvantages Costly (time and software costs) Learning curve Requires specialized knowledge Objective 1e

28. Slide 28 When to Use Digital Analysis Traditional – intermittent (one off) Trend is to use it as often as possible Continuous monitoring Scheduled processing Objective 1e

29. Slide 29 Where Is It Applicable? Any organization with data in digital format, and especially if: –Volumes are large –Data structures are complex –Potential for fraud exists Objective 1e

30. Slide 30 Objective 1 Summarized Three brief examples IIA Guidance Paper “Top 10” Metrics Process Overview Who, What, Why, When & Where Objective 1

31. Slide 31 Objective 1 - Summarized 1.Understand why and how 2.Understand statistical basis for quantifying differences 3.Identify ten general tools and techniques 4.Understand use of Excel 5.How pattern detection fits in Next is the basis …

32. Slide 32 Basis for Pattern Detection Analytical review Isolate the “significant few” Detection of errors Quantified approach Objective 2

33. Slide 33 Understanding the Basis Quantified Approach Population vs. Groups Measuring the Difference Stat 101 – Counts, Totals, Chi Square and K-S The metrics used Objective 2

34. Slide 34 Quantified Approach Based on measureable differences Population vs. Group “Shotgun” technique Objective 2a

35. Slide 35 Detection of Fraud Characteristics Something is different than expected Objective 2a

36. Slide 36 Fraud patterns Common theme – “something is different” Groups Group pattern is different than overall population Objective 2b

37. Slide 37 Measurement Basis Transaction counts Transaction amounts Objective 2c

38. Slide 38 A few words about statistics Detailed knowledge of statistics not necessary Software packages do the “number-crunching” Statistics used only to highlight potential errors/frauds Not used for quantification Objective 2d

39. Slide 39 How is digital analysis done? Comparison of group with population as a whole Can be based on either counts or amounts Difference is measured Groups can then be ranked using a selected measure High difference = possible error/fraud Objective 2d

40. Slide 40 Histograms Attributes tallied and categorized into “bins” Counts or sums of amounts Objective 2d

41. Slide 41 Two histograms obtained Population and group Objective 2d

42. Slide 42 Compute Cumulative Amount for each Objective 2d

43. Slide 43 Are the histograms different? Two statistical measures of difference Chi Squared (counts) K-S (distribution) Both yield a difference metric Objective 2d

44. Slide 44 Chi Squared Classic test on data in a table Answers the question – are the rows/columns different Some limitations on when it can be applied Objective 2d

45. Slide 45 Chi Squared Table of Counts Degrees of Freedom Chi Squared Value P-statistic Computationally intensive Objective 2d

46. Slide 46 Kolmogorov-Smirnov Two Russian mathematicians Comparison of distributions Metric is the “d-statistic” Objective 2d

47. Slide 47 How is K-S test done? Four step process 1.For each cluster element determine percentage 2.Then calculate cumulative percentage 3.Compare the differences in cumulative percentages 4.Identify the largest difference Objective 2d

48. Slide 48 Kolmogorov-Smirnov Objective 2d - KS

49. Slide 49 Classification by metrics Stratification Day of week Happens on holiday Round numbers Variability Benford’s Law Trend lines Relationships (market basket) Gaps Duplicates Objective 2e

50. Slide 50 Auditor’s “Top 10” Metrics 1.Outliers / Variability 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates Objective e

51. Slide 51 Understanding the Basis Quantified Approach Population vs. Groups Measuring the Difference Stat 101 – Counts, Totals, Chi Square and K-S The metrics used Objective 2

52. Slide 52 Objective 2 - Summarized 1.Understand why and how 2.Understand statistical basis for quantifying differences 3.Identify ten general tools and techniques 4.Understand examples done using Excel 5.How pattern detection fits in Next are the metrics …

53. Slide 53 The “Top 10” Metrics Overview Explain Each Metric Examples of what it can detect How to assess results Objective 3

54. Slide 54 Trapping anomalies Objective 3

55. Slide 55 Fraud Pattern Detection Target Group Round Numbers Benford’s Law GapsUnivariateDuplicatesDay of WeekHolidayTrend LineStratification Market Basket Objective 3

56. Slide 56 Outliers / Variability Outliers are amounts which are significantly different from the rest of the population 1 - Outliers

57. Slide 57 Outliers / Variability Charting (visual) Software to analyze “z- scores” Top and Bottom 10, 20 etc. High and low variability (coefficient of variation) 1 - Outliers

58. Slide 58 Drill down to the group level Basic statistics –Minimum, maximum and average –Variability Sort by statistic of interest –Variability (coefficient of variation) –Maximum, etc. 1 - Outliers

59. Slide 59 Example Results ProviderNCoeff Var 34784213,243342.23 23567214,53687.23 35467893,42123.25 54631222,31118.54 Two providers (3478421 and 2356721) had significantly more variability in the amounts of their claims than all the rest. 1 - Outliers

60. Slide 60 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

61. Slide 61 Unusual stratification patterns Do you know how your data looks? 2 - Stratification

62. Slide 62 Stratification - How Charting (visual) Chi Squared Kolmogorov-Smirnov By groups 2 - Stratification

63. Slide 63 Purpose / types of errors Transactions out of the ordinary “Up-coding” insurance claims “Skewed” groupings Based on either count or amount 2 – Stratification

64. Slide 64 The process? 1.Stratify the entire population into “bins” specified by auditor 2.Same stratification on each group (e.g. vendor) 3.Compare the group tested to the population 4.Obtain measure of difference for each group 5.Sort descending on difference measure 2 – Stratification

65. Slide 65 Units of Service Stratified - Example Results Two providers (2735211 and 4562134) are shown to be much different from the overall population (as measured by Chi Square). ProviderNChi SqD-stat 27352116,0117,4530.8453 45621348,9135,2340.7453 43210893,4103420.5231 42378692,5032980.4632 2 – Stratification

66. Slide 66 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

67. Slide 67 Day of Week Activity on weekdays Activity on weekends Peak activity mid to late week 3 – Day of Week

68. Slide 68 Purpose / Type of Errors Identify unusually high/low activity on one or more days of week Dentist who only handled Medicaid on Tuesday Office is empty on Friday 3 – Day of Week

69. Slide 69 How it is done? Programmatically check entire population Obtain counts and sums by day of week (1-7) Prepare histogram For each group do the same procedure Compare the two histograms Sort descending by metric (chi square/d-stat)

70. Slide 70 Day of Week - Example Results Provider 2735211 only provided service for Medicaid on Tuesdays. Provider 4562134 was closed on Thursdays and Fridays. ProviderNChi SqD-stat 27352115,40412,4350.9802 45621345,1827,7460.8472 43210895,162870.321 42378697,905560.2189 3 – Day of Week

71. Slide 71 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

72. Slide 72 Round Numbers It’s about…. Estimates! 4 – Round Numbers

73. Slide 73 Purpose / Type of Errors Isolate estimates Highlight account numbers in journal entries with round numbers Split purchases (“under the radar”) Which groups have the most estimates 4 – Round Numbers

74. Slide 74 Round numbers Classify population amounts –$–$1,375.23 is not round –$–$5,000 is a round number – type 3 (3 zeros) –$–$10,200 is a round number type 2 (2 zeros) Quantify expected vs. actual (d- statistic) Generally represents an estimate Journal entries 4 – Round Numbers

75. Slide 75 Round Numbers in Journal Entries - Example Results Two accounts, 2735211 and 4562134 have significantly more round number postings than any other posting account in the journal entries. AccountNChi SqD-stat 27352114,13654,6370.9802 456213483335,3240.97023 43210898,3187680.321 42378699,5495460.2189 4 – Round Numbers

76. Slide 76 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

77. Slide 77 Made up Numbers Curb stoning Imaginary numbers Benford’s Law 5 – Made up numbers

78. Slide 78 What can be detected Made up numbers – e.g. falsified inventory counts, tax return schedules 5 – Made Up Numbers

79. Slide 79 Benford’s Law using Excel Basic formula is “=log(1+(1/N))” Workbook with formulae available at http://tinyurl.com/4vmcfs http://tinyurl.com/4vmcfs Obtain leading digits using “Left” function, e.g. left(Cell,1) 5 – Made Up Numbers

80. Slide 80 Made up numbers Benford’s Law Check Chi Square and d-statistic First 1,2,3 digits Last 1,2 digits Second digit Sources for more info 5 – Made Up Numbers

81. Slide 81 How is it done? Decide type of test – (first 1-3 digits, last 1-2 digit etc) For each group, count number of observations for each digit pattern Prepare histogram Based on total count, compute expected values For the group, compute Chi Square and d-stat Sort descending by metric (chi square/d-stat) 5 – Made Up Numbers

82. Slide 82 Invoice Amounts tested with Benford’s law - Example Results During tests of invoices by store, two stores, 324 and 563 have significantly more differences than any other store as measured by Benford’s Law. StoreHi DigitChi SqD-stat 324795,2340.9802 563894,7350.97023 432234760.321 217743120.2189 5 – Made Up Numbers

83. Slide 83 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

84. Slide 84 Market Basket Medical “Ping ponging” Pattern associations Apriori program References at end of slides Apriori – Latin a (from) priori (former) Deduction from the known 6 – Market Basket

85. Slide 85 Purpose / Type of Errors Unexpected patterns and associations Based on “market basket” concept Unusual combinations of diagnosis code on medical insurance claim 6 – Market basket

86. Slide 86 Market Basket JE Accounts JE Approvals Credit card fraud in Japan – taxi and ATM 6 – Market basket

87. Slide 87 How is it done? First, identify groups, e.g. all medical providers for a patient Next, for each provider, assign a unique integer value Create a text file containing the values Run “apriori” analysis 6 – Market basket

88. Slide 88 Apriori outputs For each unique value, probability of other values If you see Dr. Jones, you will also see Dr. Smith (80% probability) If you see a JE to account ABC, there will also an entry to account XYZ (30%) 6 – Market basket

89. Slide 89 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

90. Slide 90 Trend Busters Does the pattern make sense? 7 - Trends

91. Slide 91 Trend Busters Linear regression Sales are up, but cost of goods sold is down “Spikes” 7 – Trends

92. Slide 92 Purpose / Type of Errors Identify trend lines, slopes, etc. Correlate trends Identify anomalies Key punch errors where amount is order of magnitude 7 – Trends

93. Slide 93 Linear Regression Test relationships (e.g. invoice amount and sales tax) Perform multi-variable analysis 7 – Trends

94. Slide 94 How is it done? Estimate linear trends using “best fit” Measure variability (standard errors) Measure slope Sort descending by slope, variability, etc. 7 – Trends

95. Slide 95 Trend Lines by Account - Example Results Generally the trend is gently sloping up, but two accounts (43870 and 54630) are different. AccountNSlopeStd Err 32451181.2300.87 43517171.0704.3 32451271.0230.85 43517321.0100.36 43870230.3402.36 5463056-0.5601.89 7 – Trends

96. Slide 96 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

97. Slide 97 Numeric Sequence Gaps What’s there is interesting, what’s not there is critical … 8 - Gaps

98. Slide 98 Purpose / Type of Errors Missing documents (sales, cash, etc.) Inventory losses (missing receiving reports) Items that “walked off” 8 – Gaps

99. Slide 99 How is it done? Check any sequence of numbers supposed to be complete, e.g. Cash receipts Sales slips Purchase orders 8 – Gaps

100. Slide 100 Gaps Using Excel Excel – sort and check Excel formula Sequential numbers and dates 8 – Gaps

101. Slide 101 Gap Testing - Example Results Four check numbers are missing. StartEndMissing 10789107911 12523125262 17546175481 8 – Gaps

102. Slide 102 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

103. Slide 103 Duplicates Why is there more than one? Same, Same, Same, and Same, Same, Different 9 - Duplicates

104. Slide 104 Two types of (related) tests Same items – same vendor, same invoice number, same invoice date, same amount Different items – same employee name, same city, different social security number 9 – Duplicates

105. Slide 105 Duplicate Payments High payback area “Fuzzy” logic Overriding software controls 9 - Duplicates

106. Slide 106 Fuzzy matching with software Levenshtein distance Soundex “Like” clause in SQL Regular expression testing in SQL Vendor/employee situations Russian physicist 9 - Duplicates

107. Slide 107 How is it done? First, sort file in sequence for testing Compare items in consecutive rows Extract exceptions for follow-up 9 - Duplicates

108. Slide 108 Possible Duplicates - Example Results Five invoices may be duplicates. Vendor Invoice Date Invoice AmountCount 102456/15/20073,544.784 102458/31/20072,010.372 175462/12/20071,500.002 9 - Duplicates

109. Slide 109 Next Metric 1.Outliers 2.Stratification 3.Day of Week 4.Round Numbers 5.Made Up Numbers 6.Market basket 7.Trends 8.Gaps 9.Duplicates 10.Dates

110. Slide 110 Date Checking If we’re closed, why is there … Adjusting journal entry Receiving report 10 - Dates

111. Slide 111 Holiday Date Testing Red Flag indicator 10 – Dates

112. Slide 112 Date Testing challenges Difficult to determine Floating holidays – Friday, Saturday, Sunday, Monday 10 – Dates

113. Slide 113 Typical audit areas Journal entries Employee expense reports Business telephone calls Invoices Receiving reports Purchase orders 10 – Dates

114. Slide 114 Determination of Dates Transactions when business is closed Federal Office of Budget Management An excellent fraud indicator in some cases 10 – Dates

115. Slide 115 Holiday Date Testing Identifying holiday dates: –Error prone –Tedious U.S. only 10 – Dates

116. Slide 116 Federal Holidays Established by Law Ten dates Specific date (unless weekend), OR Floating holiday 10 – Dates

117. Slide 117 Federal Holiday Schedule Office of Personnel Management Example of specific date – Independence Day, July 4 th (unless weekend) Example of floating date – Martin Luther King’s birthday (3 rd Monday in January) Floating – Thanksgiving – 4 th Thursday in November 10 – Dates

118. Slide 118 How it is done? Programmatically count holidays for entire population For each group, count holidays Compare the two histograms (group and population) Sort descending by metric (chi square/d-stat) 10 – Dates

119. Slide 119 Holiday Counts - Example Results Two employees (10245 and 32325) were “off the chart” in terms of expense amounts incurred on a Federal Holiday. Employee NumberNChi SqD-stat 10245375,2340.9802 32325234,7350.97023 17546184760.321 24135343120.2189 10 – Dates

120. Slide 120 The “Top 10” Metrics Overview Explain Each Metric Examples of what it can detect How to assess results Objective 3

121. Slide 121 Objective 3 - Summarized 1.Understand why and how 2.Understand statistical basis for quantifying differences 3.Identify ten general tools and techniques 4.Understand examples done using Excel 5.How pattern detection fits in Next – using Excel …

122. Slide 122 Use of Excel Built-in functions Add-ins Macros Database access Objective 4

123. Slide 123 Excel templates Variety of tests –R–Round numbers –B–Benford’s Law –O–Outliers –E–Etc. Objective 4

124. Slide 124 Excel – Univariate statistics Work with Ranges =sum, =average, =stdevp =largest(Range,1), =smallest(Range,1) =min, =max, =count Tools | Data Analysis | Descriptive Statistics Objective 4

125. Slide 125 Excel Histograms Tools | Data Analysis | Histogram Bin Range Data Range Objective 4

126. Slide 126 Excel Gaps testing Sort by sequential value =if(thiscell-lastcell <> 1,thiscell-lastcell,0) Copy/paste special Sort Objective 4

127. Slide 127 Detecting duplicates with Excel Sort by sort values =if testing =if(=and(thiscell=lastcell, etc.)) Objective 4

128. Slide 128 Performing audit tests with macros Repeatable process Audit standardization Learning curve Streamlining of tests Examples - http://tinyurl.com/576tp8 http://tinyurl.com/576tp8 Objective 4

129. Slide 129 Using database audit software Many “built-in” functions right off the shelf with SQL Control totals Exception identification “Drill down” Quantification June 2008 article in the EDP Audit & Control Journal (EDPACS) “SQL as an audit tool” Objective 4

130. Slide 130 Use of Excel Built-in functions Add-ins Macros Database access Objective 4

131. Slide 131 Objective 4 - Summarized 1.Understand why and how 2.Understand statistical basis for quantifying differences 3.Identify ten general tools and techniques 4.Understand examples done using Excel 5.How Pattern Detection fits in Next – Fit …

132. Slide 132 How Pattern Detection Fits In Business Analytics Fraud Pattern Detection Continuous monitoring Objective 5

133. Slide 133 Where does Fraud Pattern Detection fit in? Business Analytics Fraud Pattern Detection Continuous fraud pattern detection Continuous Monitoring Right in the middle Objective 5

134. Slide 134 Business Analytics Fraud analytics -> business analytics Business analytics -> fraud analytics Objective 5

135. Slide 135 Role in Continuous Monitoring (CM) Fraud analytics can feed (CM) Continuous fraud pattern detection Use output from CM to tune fraud pattern detection Objective 5

136. Slide 136 Objective 5 - Summarized 1.Understand why and how 2.Understand statistical basis for quantifying differences 3.Identify ten general tools and techniques 4.Understand use of Excel 5.How pattern detection fits in Next: Links …

137. Slide 137 Links for more information Kolmogorov-Smirnov http://tinyurl.com/y49sec Benford’s Law http://tinyurl.com/3qapzu http://tinyurl.com/3qapzu Chi Square tests http://tinyurl.com/43nkdh http://tinyurl.com/43nkdh Continuous monitoring http://tinyurl.com/3pltdl http://tinyurl.com/3pltdl

138. Slide 138 Market Basket Apriori testing for “ping ponging” Temple University http://tinyurl.com/5vax7r http://tinyurl.com/5vax7r Apriori program (“open source”) http://tinyurl.com/5qehd5 http://tinyurl.com/5qehd5 Article – “Medical ping ponging” http://tinyurl.com/5pzbh4 http://tinyurl.com/5pzbh4

139. Slide 139 Excel macros used in auditing Excel as an audit software http://tinyurl.com/6h3ye7 http://tinyurl.com/6h3ye7 Selected macros - http://tinyurl.com/576tp8 http://tinyurl.com/576tp8 Spreadsheets forever - http://tinyurl.com/5ppl7t http://tinyurl.com/5ppl7t

140. Slide 140 Questions?

141. Slide 141 Contact info E-mail: Mike.Blakley@ezrstats.comMike.Blakley@ezrstats.com Web: http://ezrstats.com