January 2004/Issue 9Translating Research to Policy

Do National Surveys Overestimate the Numberof Uninsured? Findings from the MedicaidUndercount Experiment in Minnesota

OVERVIEW

General population surveys of healthinsurance coverage provide timely estimatesof uninsurance. These estimates informresource allocation and policy decisionsmade by Federal and state lawmakers tryingto make health insurance more accessibleand affordable. Policymakers and analystsuse survey estimates of coverage and lack ofcoverage to monitor the dynamics of healthinsurance markets, evaluate the success ofcurrent programs in reaching targetpopulations, and assess the costs andbenefits of program changes, outreachactivities, and other coverage initiatives.These survey estimates are also used infederal formulas that allocate billions ofdollars annually to states for the StateChildren’s Health Insurance Program(SCHIP). With these factors in mind, theimportance of obtaining accurate estimatesof the number of people lacking insurancebecomes clear.

One area of ongoing concern to researchersis that general population surveys like theCurrent Population Survey (CPS)systematically underestimate the number ofindividuals known through administrativerecords to be enrolled in Medicaidprograms. This discrepancy between surveyand administrative counts of Medicaidenrollment—or the “Medicaidundercount”—is problematic, not in andof itself, but to the extent that it is thought

to cause upward bias in survey estimatesof the number of uninsured. When surveyestimates of Medicaid enrollment do notmatch administrative data counts, thediscrepancy raises concerns about otherestimates produced by the survey.

General population surveys are the onlysource of estimates on the number of peoplecovered by private insurance, those who areuninsured, and those who are uninsured buteligible for public programs. This SHADACissue brief summarizes our recent study ofthe Medicaid undercount in Minnesota.

EXPLAINING THE MEDICAID

UNDERCOUNT

Comparisons of survey estimates ofMedicaid participation to Medicaidadministrative data indicate that anywherefrom 15 to 50 percent of Medicaid cases aremissed by national population surveys suchas the Current Population Survey (CPS),the Survey of Income and ProgramParticipation, and the Community TrackingStudy.1 One might infer from these resultsthat some portion of Medicaid recipientsdo not report their Medicaid coverage insurveys asking about health insurancecoverage.

Medicaid enrollees might provide inaccurateresponses to survey questions addressinginsurance coverage for a number of reasons.Some Medicaid recipients may be confused

1See Lewis, K., M. Ellwood, and J.L. Czajka. 1998. Counting the Uninsured: A Review of the Literature.Washington, D.C.: The Urban Institute.

about what program they are in, either because theyhaven’t accessed health care services in some time, orbecause their enrollment status changes frequently.Others may provide misleading information becausethey are embarrassed to be associated with a welfare-like public program. Still others may report a source ofcoverage other than Medicaid if they: are confused bythe similarity of the program names (e.g., Medicareand Medicaid); have multiple sources of coverage (e.g.,Medicare, private third-party coverage); associateMedicaid coverage with a commercial product becausethey are enrolled in a Medicaid managed care plan; orthink they are covered by a different state-subsidizedhealth care program altogether.

While the research community has established that thenumber of people reporting Medicaid coverage isconsistently lower than the number enrolled in theprogram according to administrative records, thequestion remains whether some portion of Medicaidrecipients report having no insurance or some othersource of insurance in surveys asking about healthinsurance coverage.

MINNESOTA’S MEDICAID UNDERCOUNT

EXPERIMENT

To examine the accuracy of Medicaid enrollees’responses to health insurance surveys, SHADACresearchers conducted the Medicaid UndercountExperiment (MUE). By asking a random sample ofknown Minnesota Health Care Program enrollees (i.e.,Medicaid, MinnesotaCare and General AssistanceMedical Care) about their health insurance coverage inconjunction with a statewide general population survey,researchers were able to determine: (1) the frequencywith which Medicaid recipients accurately reportedtheir public coverage, and (2) the impact of inaccuratereports on survey estimates of coverage derived fromthe statewide survey.

As shown in Figure 1, only 37% of known Medicaidenrollees responded accurately to survey questionsabout their health insurance. The remaining 63% werelabeled “missed Medicaid cases” due to the following:communication barriers or refusals (13.8%), lack oftelephone (18.0%), or inaccurate responses to questions

about insurance coverage (31.4%). The inaccurateresponses consisted of 2.8% reporting no coverageat all, 7.8% reporting private coverage, and 20.8%reporting the wrong type of public coverage (e.g.,Medicare, MinnesotaCare, or General AssistanceMedical Care). About half of the later group (10.2%)were eligible for both Medicare and Medicaid,and reported having Medicare, but not Medicaid,coverage.

The policy implications of these findings areimportant: the Medicaid undercount—at least asmeasured by the MUE in Minnesota—introduced onlya negligible upward bias to estimates of the uninsuredproduced by the state survey. Specifically, we calculatedthe bias introduced by inaccurate survey responsesamong all public program enrollees in the MUEwhich reduced Minnesota’s uninsurance estimate byonly 0.26 percentage points, from 5.29 to 5.03percent.2 This difference is not significant; thereforeinaccurate reports of coverage among Medicaidrecipients were found not to bias the estimate ofuninsurance.

IMPLICATIONS FOR POLICY AND FURTHER

RESEARCH

SHADAC’s findings imply that while generalpopulation surveys like the CPS systematicallyunderestimate participation in the Medicaid program,the effect on estimates of uninsurance may beextremely modest. This seemingly technical result hasreal policy implications at state and national levels, andis good news for analysts concerned about the validityof survey estimates of those lacking health insurancecoverage. Our research suggests that, at least withrespect to the survey implemented in the state ofMinnesota, health policy and resource allocationdecisions have not been misinformed.

We recognize the importance of replicating our resultsin other states, as these findings have implicationsbeyond Minnesota’s borders. We also acknowledgethat differences in survey instruments, public

programs, the populations they serve, and in the healthcare delivery systems that serve them may affect theoutcome of this research. Future work by SHADACresearchers will therefore repeat the MUE in additionalstates to determine the magnitude of the Medicaidundercount and examine sources of the undercount.This will allow us to assess the extent to which theresults can be generalized to other states, and thefeasibility of developing a method for adjusting surveyestimates to account for the Medicaid undercount.

2State estimate of uninsurance reported in 1999 Minnesota Health Access Survey (MNHA).

State Health Access Data Assistance Center (SHADAC) | University of Minnesota School of Public Health612-624-4802 | fax: 612-624-1493 | www.shadac.org

IB-09-104

The State Health Access Data Assistance Center at the University of Minnesota promotesthe effective use of available data to inform the debate on health coverage and access. For acomplete account of this study, please see:

Call, Kathleen Thiede, Gestur Davidson, Anna Stauber Sommers, Roger Feldman, PaulFarseth, and Todd Rockwood, 2002. “Uncovering the Missing Medicaid Cases andAssessing their Bias for Estimates of the Uninsured.” Inquiry. 38(4): 396-408.