Unemployment Insurance Integrity Performance Measures A Discussion of Proposed Integrity Measures ETA Office of Unemployment Insurance

RHONDA COWIE Team Lead US DOL – ETA Division of Performance Management National Office

DANIEL SOMMERS ANDY SPISAK Statistician Statistician US DOL – ETA Division of Performance Management National Office ANDY SPISAK Statistician US DOL – ETA Division of Performance Management National Office

Outline Background/Motivation Measure 1 Measure 2 Summary & Feedback Combined Benefit Accuracy Measurement (BAM)- Benefit Payment Control (BPC) Fraud Measure Measure 2 BPC Fraud and Non-Fraud Measure Summary & Feedback

Background/Motivation In 2011 ETA established the Benefit Year Earnings (BYE) Performance Measure (UIPL No. 34-11). This measure supported states as they addressed the largest reason for overpayments both on a state and national level. For example, in Performance Year 2014 – 2015, there were an estimated $1.08 billion in BYE overpayments, representing 30 percent of overpayments. For the CY 2014 performance period the acceptable level of performance (ALP) was a 25% reduction from a state’s CY 2010 to 2012 baseline BYE rate . UIPL 34-11, change 1 communicated ETA’s intent to replace this measure.

Background/Motivation In 2014, Mathematica Policy Research (MPR) conducted a Methodology Study to assess various revisions to the sampling and rate estimation methodology, and operation of the BAM survey. Based on consultations with States that were part of the study, MPR recommended the Department explore the establishment of a new measure to emphasize the importance of detecting and reducing fraud overpayments. To begin this process, ETA developed the proposed measures, which are outlined below for discussion.

Measure 1: Combined BAM, BPC Fraud Measure Definition and Calculation Ratio of Fraud Overpayments Established by BPC to BAM-estimated Fraud Overpayments, expressed as percentage: 𝐹 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑 = 𝐹 𝑏𝑝𝑐 𝐹 𝑏𝑎𝑚 ×100 Similar construction as the current Overpayment Detection Measure (BPC overpayments / BAM operational rate X 100) Data Sources: ETA 227 and BAM

Measure 1: Combined BAM, BPC Fraud Measure This measure utilizes the data collected through the BAM and BPC programs. The following are two criteria for this measure: A single performance target of 50% Based on historical data trends that suggest 50% would be the most appropriate target. A three-year performance period Variability of the BPC/BAM detection ratio is significantly lower for a three year performance period, compared to a one year period Reduces sampling errors for the BAM component Reduces variation due to operational issues such as the reprogramming of state staff

Measure 1: Combined BAM, BPC Fraud Measure Pros: Uses BAM-estimated fraud as a benchmark for BPC fraud established BPC fraud overpayments established are based on administrative data and is not subject to sampling error Encourages state BPC units to use all available tools such as cross-matches, NDNH/SDNH, wage record, and SIDES in order to improve the efficiency of their operations and increase detection ratio.

Measure 1: Combined BAM, BPC Fraud Measure Cons: Considerable variation in the data, even with a three-year performance period, which may increase difficulty of meeting and maintaining performance targets States vary with regards to definitions of fraud May be more advantageous to states having a narrower fraud definition The BAM data are subject to sampling error, which raises an operational issue in evaluating state performance. Potential for non-sampling error, such as BPC’s application of filters to cross-matches, or miscoding in BAM.

Measure 2: BPC Average Overpayment Measure Definition and Calculation: This measure utilizes the data collected through the BPC program, derived from the ETA 227 report. It measures an average overpayment (OP) for both fraud and nonfraud cases across root causes and is represented in the following formula: 𝐴𝑣𝑔 𝑂𝑃 𝑏𝑝𝑐 = 𝐹𝑟𝑎𝑢𝑑+𝑁𝑜𝑛𝑓𝑟𝑎𝑢𝑑 𝑂𝑃 𝐹𝑟𝑎𝑢𝑑+𝑁𝑜𝑛𝑓𝑟𝑎𝑢𝑑 𝐶𝑎𝑠𝑒𝑠

Operational Issues 1. Should a single measure combining fraud and non-fraud overpayments be established or should there be separate fraud and non-fraud measures? Based on historical data, a combination fraud and non-fraud measure is advantageous to most states because the impact of varying fraud definitions among states is greatly reduced. States with large differences between their fraud and non-fraud overpayment rates might have difficulty meeting separate fraud and non-fraud measures. A combined measure balances one rate against the other, possibly leading to more favorable performance for this measure.  Therefore, establishing a combined fraud and non-fraud measure appears to be the better alternative.

Operational Issues 2. How should changes in the average amount overpaid be adjusted in order to take into account increases in the WBA? Option 1: Adjust the average amount overpaid by an adjustment factor that represents the inflation (increase) of the WBA. Option 2: Adjust the average amount overpaid by expressing it as the average number of UI weeks overpaid: (Avg. $ OP / Avg. $ Paid).

Operational Issues Option 1: Adjust the average amount overpaid by an adjustment factor that represents the inflation (increase) of the WBA. Example: If the average amount paid increases from $300 in 2014 to $315 in 2015 - a 5% increase - the adjustment factor is 1.05 ($315 / $300). If the average amount overpaid increases from $800 in 2014 to $875 in 2015, the adjusted average overpayment in 2015 is $833.33 ($875 / 1.05).

Operational Issues Option 2: Adjust the average amount overpaid by expressing it as the average number of UI weeks overpaid: (Avg. $ OP / Avg. $ Paid). From the preceding example: The average number of UI weeks overpaid in 2014 is: $800 / $300 = 2.67. The average number of UI weeks overpaid in 2015 is: $875 / $315 = 2.78.

Measure 2: BPC Average Overpayment Measure Pros: Provides incentive for states to pursue overpayments more aggressively (through increased use of cross- matches such as NDNH, wage record, and SIDES) to detect overpayments sooner and therefore reduce average overpayments. States strive to detect and pursue all overpayments, versus examining only fraud. Uses average overpayment established as opposed to BPC fraud rate (BPC fraud rate is consistently low and difficult to demonstrate improvement) Single percentage reduction target from each state’s baseline average can be set.

Measure 2: BPC Average Overpayment Measure Cons: A measure that includes all overpayments represents a shift in the Department’s focus from targeting BYE overpayments. Potential for non-sampling error, such as erroneous reporting on the 227 report and other state operational issues, such as integrity staff reprogramming. If “UI Weeks overpaid” is used, there is a potential challenge linking a fractional reduction in the number of weeks paid to the improper payment rate since WBA’s vary greatly across states.

SUMMARY Measure comparison: Measure 1 (BPC/BAM measure) Pulling strength from both programs for a composite measure Non-sampling error is controllable Similar to the current Overpayment Detection Measure Measure 2 (BPC only) Gives full picture of overpayments Non-sampling error controllable

Feedback: Do these measures address the concerns you raised during the Study? What, if anything would you want to change in these new measures and why?

Subri Raman Rhonda Cowie Team Lead Division Chief Division of Performance Management Cowie.rhonda.m@dol.gov 202-693-3821 Division Chief Division of Performance Management Raman.subri@dol.gov 202-693-3058