developing survey handbooks as educational tools for data users presented at the european conference...
TRANSCRIPT
Developing Survey Handbooks as Educational Tools for Data Users
Presented at the European Conference on Quality in Official Statistics
May 2010
Deborah GriffinU.S. Census Bureau
2
What I’ll Cover
Background on the American Community Survey
Producing and Disseminating the Handbooks
Innovative Approaches Used
Quantitative and Qualitative Assessments
Challenges and Lessons Learned
3
The American Community Survey
Since 1790, decennial censuses in the United States have counted the population to support apportionment
The decennial census also served as the data collection vehicle for detailed demographic, social, economic, and housing data
4
The American Community Survey
The 2010 Census is only collecting critical data required of the full population
The detailed demographic, social, economic, and housing data are now collected and disseminated using the American Community Survey
5
The American Community Survey
While ACS content is very similar to the census, the design and methodology differ
ACS data are collected continuously throughout the year and estimates are released once-a-year in the form of 1-year, 3-year, and 5-year aggregates
6
The American Community Survey
The ACS has a diverse set of data users including federal, state, and local governments, academia, media, and the private sector
Uses range from a single estimate for a grant application or homework assignment to complex research using microdata files
7
Census Bureau Strategy
The diversity of ACS data users and user applications lead to the decision to produce several different types of educational materials, including a series of ACS handbooks – each written for a different audience including audience-specific issues and case studies
8
Handbook Audiences
• General Data Users• Federal Agencies• Business Community• Researchers • Congress• High School Teachers• Public Use Microdata Sample Users• State and Local Governments
9
Handbook Audiences
• Media• Users of Data for Rural
Areas• Users of Data about
American Indians and Alaska Natives • Puerto Rico Community Survey Data Users
10
Producing the Handbooks
Experienced data users from each audience were identified to develop draft handbooks
Required topics, reference materials, and several core tables were provided
A common set of technical appendices and a glossary were developed
11
Generic Table
12
Technical Appendices
13
Technical Appendices
14
Innovative Approaches
We encouraged each content developer to use a style and case studies that would work best with their audience
Challenge was to maintain common look-and-feel and consistent messaging across all approaches
15
Media
16
State and Local Governments
17
General Users
18
Researchers
19
High School Teachers
20
Rural Area Data Users
21
Producing the Handbooks
Each handbook was peer reviewed by a set of individuals who were a part of the target audience
Census Bureau staff conducted a detailed review of all handbooks to ensure accuracy and consistency of messaging
Final stage of graphics design and review
22
Disseminating the Handbooks
Handbooks were released online beginning in October 2008
Webpage created for users to access and download handbooks
Printed copies also produced and disseminated
23
Quantitative AssessmentAudience PDF views each month
General Data Users 269
Congress 129
High School Teachers 583
Puerto Rico Community Survey Users 58
Business Community 111
Public Use Microdata Sample Users 150
Researchers 327
Users of Data for Rural Areas 185
Users of Data for American Indians & Alaska Natives 109
Federal Agencies 100
Media 260
State and Local Governments 157
24
Quantitative AssessmentAudience Hard copies distributed
General Data Users 1,200
Congress 400
High School Teachers 400
Business Community 1,000
Public Use Microdata Sample Users 500
Researchers 800
Users of Data for Rural Areas 700
Users of Data for American Indians & Alaska Natives 800
Federal Agencies 400
Media 500
State and Local Governments 1,500
TOTAL 8,200
25
Lessons Learned
Striking a balance between technical accuracy and simple language that will be understood by non-technical audiences
Gaining agreement within Census on detailed guidance
Gaining Census Bureau acceptance of a non-traditional approach to user materials
26
Lessons Learned
Use of external content developers was a huge benefit to producing user-friendly handbooks
Use of peer reviewers was an important way to gauge the ability of each handbook to reach key audiences
27
Conclusion
Production required a great investment of time and resources
Audience-specific was the right decision
Dissemination and assessment of utility are a challenge
Users want more
