introduction to encounter data validation
DESCRIPTION
Introduction to Encounter Data Validation. Presenter : Thomas Miller, MA Executive Director, Research and Analysis Team. 1. Welcome. About me Rules for engagement Presentation overview The importance of encounter data Trends in Federal policy CMS protocols Florida EDV study. 2. - PowerPoint PPT PresentationTRANSCRIPT
Introduction to Encounter Data Validation
Presenter: Thomas Miller, MA Executive Director, Research and Analysis Team
1
Welcome
About me Rules for engagement Presentation overview
• The importance of encounter data • Trends in Federal policy• CMS protocols• Florida EDV study
2
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study.
3
4
Importance of Encounter Data
Accurate and complete data are critical to success of managed care programs
Essential for overall management and oversight of Florida’s Medicaid program– Ability to monitor and improve quality
of care– Establish performance measures– Generate accurate and reliable reports– Obtain utilization and cost information
5
Importance of Encounter Data
6
Importance of Encounter Data
Used by MCOs and the State for many purposes– Performance measure development and calculation– Performance improvement measurement– Focused studies/quality activities– Rate-setting– Compliance monitoring– Provider practice patterns
7
Key Trends
Importance of Federal and State monitoring– Development of core measurement sets
• Medicare versus Medicaid• Health care reform• Holding health care accountable
Data, not anecdotes
8
Key Trends in the News
9
Key Trends
Findings from a recent article in Medicare and Medicaid Research Review, Assessing the Usability of MAX 2008 Encounter Data for Comprehensive Managed Care– Objective: Assess availability, completeness,
quality, and usability of encounter data– Results: High rates for reporting by key encounter
data types– Conclusions: Completeness and quality of
encounter data were high
10
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study.
1111
12
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study.
13
EQR Protocol Developed and refined with the maturation of the
External Quality Review program
14
EQR Protocol
Specific guideline for External Quality Review Organizations (EQRO) to use when assessing completeness and accuracy of encounter data.
Data submitted by Managed Care Organizations (MCO) to the State
15
EQR Protocol
State establishes standards for encounter data
State must establish the following standards:– Definition of “encounter”– Types of encounters – Data accuracy and
completeness– Objective standards for data
comparison
16
EQR Protocol
Five core activities1. Review state
requirements2. Review MCO’s
capability3. Analyze electronic
encounter data4. Review of medical
records5. Submission of findings
and recommendations
17
EQR Protocol Attachment A: Encounter Data Tables
Table 2: Data Element Validity Requirements
18
EQR Protocol
Five core activities1. Review state requirements
• Develop understanding of State-specific policies and procedures for collecting and submitting encounter data
• Identify data exchange protocols and layouts• Evaluate encounter data system interchange
flows, including system edits and submission timelines
• Review existing encounter data quality activities, requirements, and performance standards
19
EQR Protocol Five key activities, continued
2. Review MCO’s capability• Develop, conduct, and review MCO’s
Information System Capabilities Assessment– Identification of IS vulnerabilities– Key findings address:
» Data processing and procedures» Claims/encounter processing and system
demonstration» Enrollment
• Key informant interviews
20
EQR Protocol
Five key activities, continued3. Analyze electronic encounter data
• STEP 1 - Develop data quality test plan to determine:– Magnitude and type of
missing encounter data– Overall data quality issues– MCO data submission issues
21
EQR Protocol
Five key activities, continued3. Analyze electronic encounter data
• STEP 2 - Verify integrity of encounter data– Macro-level analysis– Encounter file completeness and
reasonableness» Volume and utilization by encounter type and
service setting» Internal field consistency» General field completeness and validity
22
EQR Protocol
Five key activities, continued3. Analyze electronic encounter data
• STEP 3 – Generate and Review Analytic Reports– Micro-level analysis– Encounter record completeness and
reasonableness» Follows similar analysis as outlined
in Step 2» Analyzing volume/consistency by
time, provider, service type
23
EQR Protocol
Five key activities, continued3. Analyze electronic encounter data
• STEP 4 – Compare findings to state-identified standards– Identification of appropriate benchmark
population
24
EQR Protocol
Five key activities, continued4. Review of medical records
• Verification of the accuracy of coding• Protocol assumptions• STEP 1 – Determine sampling for medical record
review– Identify valid sample size– Encounter- vs. recipient-based samples
25
EQR Protocol
Five key activities, continued4. Review of medical records
• STEP 2 – Obtain and review medical records and document findings– Procurement efficiencies– Abstraction staff and training– Categorization of errors by level, type, and
source– Procurement tracking and abstraction tools
26
EQR Protocol
Five key activities, continued5. Submission of findings
• Narrative report summarizing findings from Activities 1-4
• Actionable recommendations for overall encounter data quality improvement
27
Proto what?
Questions?
28
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study.
29
30
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study.
31
SFY 2013-2014 Encounter Data Validation (EDV) Study
Agency for Health Care Administration
32
VALIDATION OF ENCOUNTER DATA
SFY 2013-2014 Encounter Data Validation (EDV) Study
Review proposed encounter data validation process– Submitted as part of EQR RFP response– Will be conducted in alignment with CMS’ EQR
Protocol 4– Evaluates the accuracy and completeness of encounter
data submitted to AHCA by capitated health plans
33
SFY 2013-2014 Encounter Data Validation (EDV) Study
Background– Experience– Core competency evaluating data
• Information system reviews• Comparative analyses of MCO and State Medicaid data• Medical/clinical record review
– Methodology is constructed to provide an effective validation of the quality of data maintained by State agencies within resource requirements
34
SFY 2013-2014 Encounter Data Validation (EDV) Study
35
SFY 2013-2014 Encounter Data Validation (EDV) Study
Four key steps for conducting successful evaluations
– Project implementation– Study design– Data collection &
analysis– Reporting &
recommendations
36
SFY 2013-2014 Encounter Data Validation (EDV) Study
Project Implementation– Kick-off meeting with AHCA
• Initiated during contract implementation period• Review and define overall scope of project• Discuss anticipated timelines• Define evaluation parameters
– Number of MCOs included– Data requirements and limitations– Implementation procedures to validate AHCA’s encounter data
– Kick-off meeting with participating MCOs• Description of project and finalized study methodology• Expectations for MCO involvement
37
SFY 2013-2014 Encounter Data Validation (EDV) Study
Study design– Prepare draft methodology including:
• Study objectives and research questions• Data source and collection procedures• Measurement methodology • Analytic methods• Timeline
– Review and approval of methodology by AHCA – Develop of detailed analysis plan or technical companion
document methodology
38
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis– Information systems review
• Scope to be defined in collaboration with AHCA
• Identify key encounter data policies and procedures
– Selection of key evaluation fields, service groups, and encounter types
– Identification of existing/proposed standards– Review of processes affecting data quality
• Expected to be limited in scope– Focused on building contextual knowledge of
systems to facilitate development of effective and actionable recommendations
39
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis– Information systems review, continued
• Request for supplemental documents– Encounter data submission process– Previous studies conducted by AHCA
• Documentation will be used to assess encounter data quality• Used of NCQA® Roadmap where appropriate
40
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Encounter data source files
• Review of State encounter data file layouts• Prepare data requirements documents • Receive, process, and load encounter data
– Final status encounters from the Florida Medicaid Management Information System and Decision Support System (FMMIS/DSS)
– Final status claims/encounters from MCO adjudication systems– Includes all claim/service types—i.e., inpatient/outpatient, physician
visits, dental, and pharmaceutical
41
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis– Comparative data analysis of State and MCO
encounter data• Evaluates the extent to which encounters submitted by MCOs to
AHCA are accurate, complete, and reasonable• Preliminary file review
– Ensures files are sufficient for processing– Involves the basic checks
» Percentage present» Percentage valid» Percentage valid values
42
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Comparison: State data to MCO data
• Indicators to measure degree of completeness and accuracy for each encounter type
– Overall record matching—percentage of state encounters present in MCO files
– Field-level matching—percentage of state encounters with exact value match in MCO file for each select data element
» Standard fields include: date of service, recipient ID, provider ID, primary diagnosis, procedure code(s), and payment fields
43
44
SFY 2013-2014 Encounter Data Validation (EDV) Study
Table X—Diagnosis Code Matching Rates for Institutional Claims
Plan
Total Number of Matched
Claims
Encounter-Level Matching Field-Level Matching: % Correctly Matched
% With All Diagnoses Correctly Matched in
Both FilesIn First Diagnosis
FieldIn Second
Diagnosis FieldIn Third
Diagnosis FieldIn Fourth
Diagnosis Field
In Fifth Diagnosis
Field
Statewide 4,655,817 92.1% 99.1% 82.1% 88.2% 93.0% 94.9%
Plan A 144,090 96.3% 97.8% 99.0% 99.6% 99.8% 99.9%
Plan B 500,980 99.5% >99.9% 99.9% 99.8% 99.8% 99.8%
Plan C 2,429,624 89.1% 100.0% 75.4% 85.0% 91.5% 94.9%
Plan D 737,587 92.3% >99.9% 68.2% 75.3% 84.0% 89.8%
Plan E 224,193 >99.9% >99.9% >99.9% >99.9% >99.9% >99.9%
Plan F 367,800 89.8% 89.8% >99.9% >99.9% >99.9% 89.9%
Plan G 251,543 >99.9% >99.9% >99.9% >99.9% >99.9% >99.9%
44
45
SFY 2013-2014 Encounter Data Validation (EDV) Study
Table Y—Second Diagnosis Field Code Matching Rates for Institutional Claims
MCPTotal Number of Matched Claims
% Correctly Matched in Both Files
% Mismatch Due to:
Diagnosis Omitted in State File
Diagnosis Omitted in Plan File
True Diagnosis Mismatch
Statewide 4,655,817 82.1% 0.9% 12.1% 4.9%
Plan A 144,090 99.0% <0.1% 0.0% 1.0%
Plan B 500,980 99.9% 0.1% 0.0% 0.1%
Plan C 2,429,624 75.4% 0.0% 23.2% 1.4%
Plan D 737,587 68.2% 5.7% <0.1% 26.1%
Plan E 224,193 >99.9% <0.10% <0.1% <0.1%
Plan F 367,800 >99.9% 0.0% <0.1% 0.0%
Plan G 251,543 >99.9% 0.0% <0.1% <0.1%
45
Phew… Questions?
46
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Medical record review
• Represents the “gold standard” • Evaluation of service level accuracy
and completeness• Proposed methodology
– Only include MCOs operational as of January 2013– EQRO Contract Years 1, 2, and 3 (7/1/2013-6/30/2016): review one-
third of selected plans each year– EQRO Contract Years 4 and 5 (7/1/2016-6/30/2018): review one-half of
selected plans each year– Procure and abstraction 25 percent of all sampled records each quarter– Minimum 50 cases reviewed per plan– Target professional, dental, and pharmacy encounters
47
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Medical record review
• Sample selection methodology1. To generate list of randomly
selected encounters for medical review, HSAG proposes using data files from comparative analyses
2. Two-stage stratified sampling design used to ensure:» Member’s record is selected only
once» Number of encounters included in
final sample covers all encounter types and proportional to total distribution of encounters
48
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Medical record review
• Sample selection methodology– Identify all users by encounter type per MCO– Determine required sample size of each encounter type based on total
distribution of users– Randomly select users form each encounter type based on required
sample size– Identify all encounters associated with applicable encounter types for the
selected users– Final sample will consist of 50 cases randomly selected from applicable
encounter types per MCO per year, OR1,200 cases for 1/3 of all MCOs being reviewed per year
– For each encounter type, HSAG will define specific data elements for validation
49
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Medical record review
• Procurement of selected sample records– General Process
» Once sample is selected, each MCO to receive list of its study cases
» HSAG will match selected date of service for each sampled member with rendering provider
» MCOs will procure and submit identified medical records to HSAG for review
50
SFY 2013-2014 Encounter Data Validation (EDV) Study Data collection and analysis, continued
– Medical record review• Procurement of selected sample records
– Two-hour technical assistance call with all participating MCOs
– HSAG to review project and procurement protocols– Able to accommodate a variety of procurement
methods:» Faxing» Hardcopy submissions» Electronic submission via secure file transfer protocol
– Note: HSAG applies strict protocols to ensure security and confidentiality of members’ medical records
51
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Medical record review
• HSAG procurement and abstraction tool– Data collection, management, and reporting system
• HSAG reviewers are experienced:– Clinical nurses– Nurse coders
• Procurement and abstraction process– Based on established policies and procedures– Continually monitored to ensure validity and accuracy
» Inter-rater reliability testing & Rater-to-standard testing» All reviewers must achieve 95% accuracy rate» Variety of reports will be generated, i.e., medical record compliance
rates
52
SFY 2013-2014 Encounter Data Validation (EDV) Study
Data collection and analysis, continued– Medical record review – analysis of cases
• Compare electronic encounter data to medical record data• Analyze record completeness and the accuracy of coding• Four primary indicators for data completeness and accuracy
1. Medical Record Agreement2. Medical Record Omission (surplus)3. Encounter Record Omission (missing)4. Erroneous
53
SFY 2013-2014 Encounter Data Validation (EDV) Study
Reporting and recommendations– Prepare draft report of findings including:
• Indicator results• Sub-analysis findings• Preparation of supplemental findings
for future evaluation by MCOs– Presented for statewide and MCO-specific results– Actionable recommendations for improvement
54
Objectives
1. Learn why Encounter Data Validation studies are important.
2. Identify the core evaluation components outlined in CMS’ protocols for validating the quality of encounter data.
3. Understand the proposed scope of work for Florida Medicaid’s SFY 2013-2014 encounter data validation study.
55
Questions