results from the 2010 census race and hispanic origin alternative questionnaire experiment
DESCRIPTION
Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment. Nicholas Jones Roberto Ramirez U.S. Census Bureau. Presentation for the 2012 FCSM Statistical Policy Seminar (December 5, 2012 Washington , DC). Overview of Presentation Topics. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/1.jpg)
Results from the 2010 Census Race and Hispanic Origin
Alternative Questionnaire ExperimentNicholas Jones Roberto Ramirez
U.S. Census Bureau
Presentation for the 2012 FCSMStatistical Policy Seminar
(December 5, 2012 Washington, DC)
![Page 2: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/2.jpg)
Overview of Presentation Topics
• Goals and Research Strategies
• Methodology
• Experimental Designs
• Major Findings
• Recommendations
2
![Page 3: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/3.jpg)
What is the AQE?
The 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment (AQE) focused on improving the race and Hispanic origin questions by testing a number of different questionnaire design strategies.
3
![Page 4: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/4.jpg)
What is the AQE?
4
![Page 5: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/5.jpg)
Goals and Research Strategies
5
![Page 6: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/6.jpg)
Goals and Research Strategies • Increase reporting in the standard U.S.
Office of Management and Budget (OMB) race and ethnic categories
6
RACE CATEGORIESWhite; Black; American Indian and Alaska Native; Asian; Native Hawaiian and Other Pacific Islander
ETHNIC CATEGORIESHispanic or Latino;
Not Hispanic or Latino
![Page 7: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/7.jpg)
Goals and Research Strategies • Lower item nonresponse to the race and
Hispanic origin questions
7
![Page 8: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/8.jpg)
Goals and Research Strategies • Improve the accuracy and reliability of
race and ethnic data
8
![Page 9: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/9.jpg)
Goals and Research Strategies • Elicit the reporting of detailed race and
ethnic groups
9
WHITE: German; Irish; Lebanese
BLACK: African American; Haitian; Nigerian
AIAN: Navajo; Aleut; MayanASIAN: Chinese; Korean; Vietnamese
NHPI: Fijian; Native Hawaiian; Samoan
HISPANIC: Mexican; Puerto Rican; Cuban
![Page 10: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/10.jpg)
Methodology
10
![Page 11: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/11.jpg)
2010 AQE Methodology
• Three-part design:
- mail survey
- reinterview study
- focus groups
11
![Page 12: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/12.jpg)
AQE Mail Survey• 15 experimental questionnaire panels
• 2 control questionnaire panels
• Mailed to national sample of housing units- about 30,000 per panel- about 500,000 total
• Design focuses on oversampling of various race and ethnic groups in mail survey
12
![Page 13: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/13.jpg)
AQE Reinterview Study• One-in-five housing units from the 2010
AQE mail survey were selected for a reinterview study
• Assess the relative accuracy of the 2010 AQE race and Hispanic origin questionnaires
• Probe more extensively into the racial and ethnic background of respondents
13
![Page 14: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/14.jpg)
AQE Focus Groups• Supplement 2010 AQE quantitative research
• Understand self-identification of race and Hispanic origin and fit of responses within OMB categories
• Identify issues respondents have with experimental questionnaires; reasons behind issues
• Help refine questionnaires for future testing
• Understand how and why people identify their race and ethnicity in different ways and contexts
14
![Page 15: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/15.jpg)
Group Community
White Middle Eastern/North African; Iranian;European/North American
Black or African American
African American; African; Afro-Caribbean
American Indian and Alaska Native
American Indian (on and off reservation);Alaska Native; Indigenous Central and South American
Asian Detailed Asian groups
Native Hawaiian and Other Pacific Islander
Native Hawaiian; Detailed Pacific Islander groups
Hispanic English speaking; Spanish speaking;Detailed Hispanic groups
Multiracial Parents of multiracial children; multiracial adults
Ancestry differs from place of birth
Examples: Asian Indians from Trinidad;Chinese from Jamaica
AQE Focus Groups
15
![Page 16: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/16.jpg)
ExperimentalQuestionnaire Panels
16
![Page 17: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/17.jpg)
2010 Census - Control Panel
17
![Page 18: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/18.jpg)
18
• Modify examples
• Remove the term “Negro”
• Allow reporting of multiple responses to the Hispanic origin question
Design Strategy 1
![Page 19: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/19.jpg)
Design Strategy 2
19
• Clarify the Asian checkbox
categories
• Clarify the Pacific Islander checkbox categories
• Limit the use of the term “race”
![Page 20: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/20.jpg)
Design Strategy 3Combine the race and Hispanic origin questions:
• Detailed approach
• Streamlined approach
• Very Streamlined approach
• Alternative Control
20
![Page 21: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/21.jpg)
• Serves as an alternative control for the combined race and Hispanic origin design approach
• Maintains 2010 Census race and Hispanic origin questions’ write-in areas and checkboxes, but in a combined, single question format
Alternative Control
Approach
21
![Page 22: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/22.jpg)
Detailed Approach
22
![Page 23: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/23.jpg)
• Includes examples and write-ins for all OMB race and Hispanic origin categories
• Maintains all original race and Hispanic origin checkboxes
Detailed Approach
23
![Page 24: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/24.jpg)
Streamlined Approach
24
![Page 25: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/25.jpg)
Streamlined Approach
• Includes examples and write-ins for all OMB race and Hispanic origin categories
• Removes specific national origin checkboxes; presented as example groups
• Streamlined presentation of OMB race and Hispanic origin categories
25
![Page 26: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/26.jpg)
Very Streamlined Approach
26
![Page 27: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/27.jpg)
Very Streamlined Approach
• Part 1 - Very streamlined presentation of OMB race and Hispanic origin categories
• Part 2 - Examples for all OMB race and Hispanic origin categories
• Write-in areas for specific race(s), origin(s), tribe(s)
27
![Page 28: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/28.jpg)
Major Findings
28
![Page 29: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/29.jpg)
Item Nonresponse• Separate question format
RACE 3.5 percent to 5.7 percent
HISPANIC 4.1 percent to 5.4 percent
2929
• Combined question format
COMBINED 0.6 percent to 1.2 percent
![Page 30: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/30.jpg)
Race and Hispanic FindingsLargely due to Hispanics choosing their identity (i.e., only “Hispanic”) in the combined question format
• “Some Other Race” substantially reduced
• “White” dropped to levels reflecting the “Non-Hispanic White” population
30
![Page 31: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/31.jpg)
Race and Hispanic Findings• Two or More Responses population was larger on combined question
• Distributions similar across panels for other groups (AIAN, Asian, NHPI)
• Removal of the term “Negro” did not reduce proportion of respondents reporting as “Black”
31
![Page 32: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/32.jpg)
Race and Hispanic Findings• Hispanic proportions were similar across
panels
• Combined question approach did not reduce Hispanic proportions
• Multiple Hispanic reporting consistent, regardless of instruction to “Mark one or more” responses
32
![Page 33: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/33.jpg)
Detailed Group Findings• No differences in detailed reporting between separate question forms
• Increased detailed reporting for White groups and Black groups when provided with dedicated write-in lines
33
![Page 34: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/34.jpg)
Detailed Group Findings
• Decrease in specific Hispanic origin reporting for the combined question, especially for those where detailed checkboxes are not present
• Decrease in specific Asian reporting where detailed checkboxes are not present in the combined question
34
![Page 35: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/35.jpg)
Reinterview Findings• High consistency for all panels, especially
the combined questions
• Measure estimated reliability and accuracy
35
![Page 36: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/36.jpg)
Reinterview Findings• Gross difference rate (GDR)
– proxy for reliability
• Combined question panels had lower GDRs for White responses and Black responses
36
![Page 37: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/37.jpg)
• Net difference rates (NDR) – measures difference between number reported and actual number in group (measure of accuracy)
• Combined question panels had lower NDRs for White responses
• Combined questions appear to yield a closer representation of self-identification
37
Reinterview Findings
![Page 38: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/38.jpg)
Focus Group Findings• Prefer fair and equitable treatment of all
groups
• Many Hispanics saw the race question instructions as preventing self-identification
• Separate category for “Middle Eastern, North African, Arab”
38
![Page 39: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/39.jpg)
Focus Group Findings• Use of the term “Negro” offensive and
outdated
• Lack of agreement on definitions of “race” and “origin”
• Combined question approach presents equity
39
![Page 40: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/40.jpg)
Recommendations
40
![Page 41: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/41.jpg)
Recommendations• Further test combined race and Hispanic origin question refinements, paying special attention to research in improving detailed Asian and detailed Hispanic reporting
41
![Page 42: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/42.jpg)
Recommendations• Continue researching the optimal use of
examples for each race and origin response category
42
HISPANIC OR LATINO (for example, Mexican, Puerto Rican, Cuban, Dominican, Guatemalan, Salvadoran, and so on)
ASIAN (for example, Asian Indian, Chinese, Filipino, Japanese, Korean, Vietnamese, and so on)
![Page 43: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/43.jpg)
Recommendations• If Hispanic origin question is kept separate,
allow multiple responses by explicitly including “Mark one or more” instruction
43
![Page 44: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/44.jpg)
Recommendations• Remove term “Negro” from the “Black,
African Am., or Negro” response category
44
![Page 45: Results from the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816138550346895dd0963b/html5/thumbnails/45.jpg)
QUESTIONS?
45
For More Informationwww.2010.census.gov/news/press-kits/aqe/aqe.html