1. Encounter Data Validation: Review and Project Update
Presenters:
Thomas Miller, MA
Executive Director, Research and Analysis Team
Amy Kearney, BA
Associate Director, Research and Analysis Team 1

2. Welcome About the presenters Rules for engagement
Presentation overview
The importance of encounter data
CMS protocols
Florida EDV study, including best practices for medical record procurement 2

3. CMS Pre-Rulemaking Process and NQF Consensus Development Process
4/16/2017 5:15 AM
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Review status of Florida Medicaid’s Year One encounter data validation study. Discuss best practices for medical record procurement. 3
CMS Informational Series

4. Importance of Encounter Data
Accurate and complete data are critical to success of managed care programs
Essential for overall management and oversight of Florida’s Medicaid program
Ability to monitor and improve quality of care
Establish performance measures
Generate accurate and reliable reports
Obtain utilization and cost information
Highlight increased importance and weight being given by CMS on the State’s effort to ensure quality encounter data being collected.
1- For years, CMS’ Medicare program has used data collected through its some 26 programs to monitor, evaluate, (and in come cases reward/punish), guide quality improvement activities.
2- Introduction of Children’s and Adult Core Measures
3- Health care in this country is undergoing unprecedented change, and at a quick pace... in order to direct and ensure that change is successful, complete and accurate data is becoming increasingly relied upon. 4

5. Importance of Encounter Data
Used by MCOs and the State for many purposes
Performance measure development and calculation
Performance improvement measurement
Focused studies/quality activities
Rate-setting
Compliance monitoring
Provider practice patterns 5

6. Key Trends Importance of Federal and State monitoring
Development of core measurement sets
Medicare versus Medicaid
Health care reform
Holding health care accountable
Data, not anecdotes
Review of key encounter data activities
1- ACA – unprecedented focus on the development of monitoring metrics
2- Historical focus was on Medicare performance measurement programs; new focus on Medicaid metrics with the introduction of the adult and child core measure sets.
… anecdotes make a weak foundation for public policy. Instead, “evidence-based knowledge” is underpinning all kinds of public policy reform, whether the topic is health care, transportation, criminal justice, education or election administration.
… finding “evidence” is tricky. Every state, and frequently every jurisdiction, conducts elections differently, making comparisons difficult. Data is not gathered uniformly nationwide as it is in many other government arenas. 6

7. CMS Pre-Rulemaking Process and NQF Consensus Development Process
4/16/2017 5:15 AM
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Review status of Florida Medicaid’s Year One encounter data validation study. Discuss best practices for medical record procurement.
Quick review of where we’re at with our objectives… any questions… moving on to the next objective… identifying the core evaluation components outlined in CMS’ EQR protocols. 7 7
CMS Informational Series

8. 8 It’s picture time again…
… Public review and examination of the quality of encounter data critical to successful improvement of not only data, but eventually outcomes.
So let’s examine the “how”… 8

9. CMS Pre-Rulemaking Process and NQF Consensus Development Process
4/16/2017 5:15 AM
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Review status of Florida Medicaid’s Year One encounter data validation study. Discuss best practices for medical record procurement.
Quick review of where we’re at with our objectives… any questions… moving on to the next objective… identifying the core evaluation components outlined in CMS’ EQR protocols. 9
CMS Informational Series

10. EQR Protocol
Developed and refined with the evolution of the External Quality Review program 10

11. EQR Protocol
Guidelines for External Quality Review Organizations (EQRO) to use when assessing completeness and accuracy of encounter data.
Data submitted by Managed Care Organizations (MCO) to the State
Note: the protocols really are a guideline and must be adapted to the individual needs of each state, as well as the state and structure Medicaid data; which I can honestly tell you is quite varied across the county
Moving beyond simply encounter data submission, but looking at complete data pathway...
1- Use example of newly initiated Pharmacy Rx EDV project. 11

12. EQR Protocol State establishes standards for encounter data
State must establish the following standards:
Definition of “encounter”
Types of encounters
Data accuracy and completeness
Objective standards for data comparison 12

13. EQR Protocol Five key activities Review state requirements
Review MCO’s capability
Analyze electronic encounter data
Review of medical records
Submission of findings and recommendations
Data submitted by Managed Care Organizations (MCO) to the State 13

14. EQR Protocol
Attachment A: Encounter Data Tables Table 2: Data Element Validity Requirements 14

15. EQR Protocol Five key activities Review state requirements
Develop understanding of State-specific policies and procedures for collecting and submitting encounter data
Identify data exchange protocols and layouts
Evaluate encounter data system interchange flows, including system edits and submission timelines
Review existing encounter data quality activities, requirements, and performance standards
Often customized to identify specific areas of concern local to specific States... Identify examples 15

16. EQR Protocol Five key activities, continued Review MCO’s capability
Develop, conduct, and review MCO’s Information System Capabilities Assessment
Identification of IS vulnerabilities
Key informant interviews
Key findings address:
Data processing and procedures
Claims/encounter processing and system demonstration
Enrollment 16

17. EQR Protocol Five key activities, continued
Analyze electronic encounter data
STEP 1 - Develop data quality test plan to determine:
Magnitude and type of missing encounter data
Overall data quality issues
MCO data submission issues
Data submitted by Managed Care Organizations (MCO) to the State 17

18. EQR Protocol Five key activities, continued
Analyze electronic encounter data
STEP 2 - Verify integrity of encounter data
Macro-level analysis
Encounter file completeness and reasonableness
Volume and utilization by encounter type and service setting
Internal field consistency
General field completeness and validity
Step 2 and Step 3 go hand in hand…. 18

19. EQR Protocol Five key activities, continued
Analyze electronic encounter data
STEP 3 – Generate and Review Analytic Reports
Micro-level analysis
Encounter record completeness and reasonableness
Step 2 and Step 3 go hand in hand….
Follows similar analysis as outlined in Step 2
1- Analyzing volume/consistency by time, provider, service type 19

20. EQR Protocol Five key activities, continued
Analyze electronic encounter data
STEP 4 – Compare findings to state-identified standards
Identification of appropriate benchmark population 20

21. EQR Protocol Five key activities, continued Review of medical records
Verification of the accuracy of coding
Protocol assumptions
STEP 1 – Determine sampling for medical record review
Identify valid sample size
Encounter- vs. recipient-based samples
Won’t go into too much detail as we’ll spend some time on this a little bit later with Amy. 21

22. EQR Protocol Five key activities, continued Review of medical records
STEP 2 – Obtain and review medical records and document findings
Procurement efficiencies
Abstraction staff and training
Categorization of errors by level, type, and source
Procurement tracking and abstraction tools 22

23. EQR Protocol Five key activities, continued Submission of findings
Narrative report summarizing findings from Activities 1-4
Actionable recommendations for overall encounter data quality improvement 23

24. CMS Pre-Rulemaking Process and NQF Consensus Development Process
4/16/2017 5:15 AM
Questions?
Whatcha talkin’ about? 24
CMS Informational Series

25. CMS Pre-Rulemaking Process and NQF Consensus Development Process
4/16/2017 5:15 AM
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Review status of Florida Medicaid’s Year One encounter data validation study. Discuss best practices for medical record procurement.
Quick review of where we’re at with our objectives… any questions… moving on to the next objective… identifying the core evaluation components outlined in CMS’ EQR protocols. 25
CMS Informational Series

26. 26

27. CMS Pre-Rulemaking Process and NQF Consensus Development Process
4/16/2017 5:15 AM
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Review status of Florida Medicaid’s Year One encounter data validation study. Discuss best practices for medical record procurement.
Quick review of where we’re at with our objectives… any questions… moving on to the next objective… review status of Florida Medicaid’s Year One encounter data validation study. 27
CMS Informational Series

28. SFY 2013-2014 Encounter Data Validation (EDV) Study
Agency for Health Care Administration
Validation of Encounter Data 28

29. Year One Encounter Data Validation (EDV) Study
Four key steps for conducting successful evaluations
Project implementation
Study design
Data collection & analysis
Reporting & recommendations 29

30. Year One Encounter Data Validation (EDV) Study
Study design
Prepared and finalized methodology which included:
Study objectives and research questions
Data source and collection procedures
Measurement methodology
Analytic methods
Timeline 30

31. Year One Encounter Data Validation (EDV) Study
Data collection and analysis
Information systems review
Questionnaire for AHCA
Assessment of AHCA’s policies and procedures for data exchange, its capacity and ability to acquire and process data, and its staff responsible for executing data processing
Questionnaire for MCOs
Assessment of MCOs’ claims processing systems and processes, and its capability to submit encounter data 31

32. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Information systems review
MCO questionnaire divided into five sections:
Submitting Encounter Data to AHCA
Handling Submission Information from AHCA
Encounter Data Submission from Capitated Providers
Processing and Submission of Medicare Crossover and other Third Party Claims
Policies and Procedures in Processing Payment Information
Will evaluate MCOs’ capabilities for processing and submitting accurate and complete data 32

33. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Information systems review
AHCA questionnaire divided into three sections:
Data Exchange Policies and Procedures
Data Submission Processing Procedures and Personnel
Encounter Data Processing within the Florida MMIS 33

34. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Information systems review
Documentation will be used to assess encounter data quality
Questions target how data moves through AHCA’s data systems and how the MCOs prepare data files for submission
HSAG developed questions to investigate MCOs’ processes and procedures in preparing encounter data files
Provides information on strengths and limitations of AHCA’s and MCOs’ information systems 34

35. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Information systems review
What has been completed?
Questionnaires were approved by AHCA and distributed to AHCA and the MCOs
Received completed questionnaires from AHCA and MCOs
What needs to be completed?
Currently reviewing responses from AHCA and MCOs
May conduct additional follow-up for clarification 35

36. Questions?

37. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Comparative data analysis of AHCA and MCO encounter data
Evaluates the extent to which encounters submitted by MCOs to AHCA are accurate, complete, and reasonable
Included all claim/service types—i.e., inpatient/outpatient, physician visits, dental, and pharmaceutical
Final status encounters from the Florida Medicaid Management Information System and Decision Support System (FMMIS/DSS)
Final status claims/encounters from MCO adjudication systems 37

38. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Comparative data analysis of State and MCO encounter data
Indicators to measure degree of completeness and accuracy for each encounter type
Overall record matching—percentage of state encounters present in MCO files
Field-level matching—percentage of state encounters with exact value match in MCO file for each select data element 38

39. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Comparative data analysis
What has been completed?
Distributed data submission requirements documents to AHCA and MCOs
Conducted technical assistance sessions with MCOs on 9/16 & 9/17/2013
Received, processed, and loaded encounter data
Reviewed companion guides before developing data submission requirements documents
Reviewed expectations for MCO involvement during TA sessions
Conducted technical assistance sessions with MCOs on 9/16 & 9/17/2013
Discussed study methodology
Reviewed data submission requirements
Requested encounter files with dates of service from 1/1/2012 to 12/31/2012 39

40. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Comparative data analysis of State and MCO encounter data
What needs to be completed?
Conducting preliminary file review
Ensuring files are sufficient for processing
Completing basic checks
Generate comparative analysis tables and figures for final report
Completing basic checks
Percentage present
Percentage valid
Percentage valid values 40

41. Phew… Questions? 41

42. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review
Represents the “gold standard”
Evaluation of service level accuracy and completeness
Methodology developed:
Only includes MCOs operational as of January 2013
Year One – SFY 2016: review one-third of plans each year as selected by AHCA
Minimum 50 cases reviewed per plan
Target professional, dental, and inpatient/outpatient encounters
Exclude pharmacy and certain ancillary outpatient services (lab, radiology, and transportation)
SFY 2017 – SFY 2018: review one-half of plans each year as selected by AHCA 42

43. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review
Sample selection methodology
To generate list of randomly selected encounters for medical review, HSAG will use AHCA’s data files from comparative analyses
Two-stage stratified sampling design used to ensure:
Member’s record is selected only once
Number of encounters included in final sample covers all encounter types and proportional to total distribution of encounters
Sample selection methodology
Identify all users by encounter type per MCO
Determine required sample size of each encounter type based on total distribution of users
For MCOs that contract for Reform and Non-Reform services, identify a 50 percent sample for Reform and 50 percent sample for Non-Reform
Randomly select users from each encounter type based on required sample size
Identify all encounters associated with applicable encounter types for the selected users
Final sample will consist of 50 cases randomly selected from three encounter types per MCO per year 43

44. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review
Sample selection methodology
HSAG will evaluate the key data elements below:
Key Data Elements for Medical Records Review
Key Data Fields
Dental
Inpatient/
Outpatient
Physician
Date of Service √
Diagnosis Code
CPT/CDT/HCPCS Code/ Surgical Procedure Code
To be eligible for medical record review, member must be enrolled in a MCO as of 12/31/12 and must have had at least one visit during study period
Member must be continuously enrolled in the same MCO during study period with no gaps. 44

45. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review
Procurement and abstraction process
Based on established policies and procedures
Continually monitored to ensure validity and accuracy
Inter-rater reliability testing & Rater-to-standard testing
All reviewers must achieve 95% accuracy rate
Variety of reports will be generated, i.e., medical record compliance rates
Clinical nurses with:
Bachelor’s degrees in nursing
Three or more years of clinical experience and
Minimum of two years of medical record review experience 45

46. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review – analysis of cases
Verify the service(s) provided on selected data of service and one additional date of service
Each enrollee listed on sample has corresponding selected date of service
Validate services conducted by provider on date of service as compared with encounter data
Reviewers to validate services for additional date of service.
Compare electronic encounter data to medical record data
Types of documents providers need to submit
A copy of the completed enrollee sample list must be submitted along with the medical records.
If a medical record is not submitted, provider to select a non-submission reason.
Documentation must include the enrollee’s name, date of birth, and date of service.
All records must be legible.
Reviewers to validate services for additional date of service.
Providers to submit additional date of service occurring closest to selected date of service or inpatient stay, if available
Visit must occur in review period (1/1/12 – 12/31/12) 46

47. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review – analysis of cases
Analyze record completeness and the accuracy of coding
Four primary indicators for data completeness and accuracy
Medical Record Agreement
Medical Record Omission (surplus)
Encounter Record Omission (missing)
Erroneous 47

48. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review
What has been completed?
Introductory letter sent to MCOs on 10/1/13
Conducted technical assistance calls with all participating MCOs on 10/16 & 10/18/13 48

49. Year One Encounter Data Validation (EDV) Study
Data collection and analysis, continued
Medical record review
What needs to be completed?
Pull samples and send lists of study cases
Provide letter to send to its providers with sample
MCOs will procure records from provider and accommodate various submission methods
MCOs to submit identified medical records to HSAG for review
Extracting data from AHCA’s systems has taken more time than we had originally anticipated. Once we complete the review, we will pull the samples.
MCOs should accommodate the following methods of submissions from provider:
Mail via traceable carrier
Fax to MCO
Hand deliver to MCO office
Direct upload to MCO FTP (Not HSAG’s FTP)
MCO may go on-site to copy record for providers with multiple sampled members or for providers that refuse to submit. 49

50. Year One Encounter Data Validation (EDV) Study
Reporting and recommendations
Prepare aggregate EDV report of findings from:
Information system review
Comparative Analysis
Medical Record Review
Preparation of supplemental findings for future evaluation by MCOs
Present statewide and MCO-specific results
Actionable recommendations for improvement 50

51. CMS Pre-Rulemaking Process and NQF Consensus Development Process
4/16/2017 5:15 AM
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Review status of Florida Medicaid’s Year One encounter data validation study. Discuss best practices for medical record procurement.
Quick review of where we’re at with our objectives… any questions… moving on to the next objective… identifying the core evaluation components outlined in CMS’ EQR protocols. 51
CMS Informational Series

52. Questions