Fully Leverage External Data Sources: A Census Bureau Change Principle

Amy O’Hara, U.S. Census BureauWashington Statistical Society Seminar on

Administrative Records for Best Possible EstimatesSeptember 18, 2014

Vision and Planning to Effect a Paradigm Change

Pipedream about making changes in income estimates in surveys

Suggestions for moving forward Need for vision and long-term planning “Requires leadership buy-in at all levels of a

statistical agency”

Change Principle: Fully Leverage External Data Sources

We will fully leverage administrative records and other external data sources, including big data, to supplement and possibly supplant frame development and direct data collection, enhance self-response capabilities, support new data products or expand existing data products. Using external data will help the Census Bureau reduce data collection costs, redundancies, and respondent burden and improve data quality. The Census Bureau also has the opportunity to serve as a data integrator for the federal statistical system.

- Future of Census Bureau Operations, April 25, 2013https://www.census.gov/aboutus/pdf/FutureofCensusBureauOperations_2013-04-25_v1.0_10ChangePrinciples.pdf

What are we doing to Fully Leverage External Data?

Research and testing with administrative records for use in 2020 Census

Demographic surveys using administrative records for frame, contact, modeling, and imputation

Big Data explorations in Economic Directorate

Strategic Plan:Research and Implement Records

Strategic Plan Tactic 1.2.4: Integrate data from existing sources, including administrative records, to produce new information products that provide deeper insights into our people and economy by combining data sets that had not previously been linked.

Strategic Plan Tactic 2.2.4: Explore and research uses of Big Data.

- U.S. Census Bureau Strategic Plan, FY2013-2017https://intranet.ecm.census.gov/sites/v2/futureon/Documents/2013-2017%20Cenus%20Bureau%20Strategic%20Plan_2013-04-25_v1.0.pdf

More from the Strategic Plan 1.1.3 – Make the 2020 Census more cost effective while

achieving quality targets. 1.2.7 - Perform world-class research. 1.6.2 - Use the most cost effective technologies available

to update the MAF and assess its quality. 2.2.1 - Promote the use of Census Bureau products. 2.3.1 - Support reimbursable surveys and continue to

improve the Interagency Agreement process. 3.1.8 - Increase survey and census efficiency by using

empirical data to facilitate intelligent business decisions prior to and during data collection.

What we are working on

Integrate new data sources – acquiring state data, negotiating to obtain new variables, school frames, housing data

Big Data – Big Data lab to test concurrent analysis and estimation, exploring options to acquire new data sources, Big Data class

Business Plan for Change: Identify and Acquire Data

Activity 1.1.2 Introduce new business process for integrating data sets and producing new data products.

Activity 2.2.1Work with the Department of Commerce, the Office of Management and Budget, and key record suppliers to establish statistical policies, procedures, and timelines for maximizing the use of administrative records.

Activity 2.2.3 Enter into Memoranda of Agreement with key suppliers.

- U.S. Census Bureau Business Plan for Changehttps://www.census.gov/aboutus/pdf/BusinessPlanforChange_2013-04-25_v1.0.pdf

More from the Business Plan for Change

1.1.1 –Identify data gaps and evaluate internal and external data sets for relevant analyses, data products, and services.

2.4.3: Explore options to reduce burden. 4.3.1 - Establish processes to engage individual and

network partners in Census Bureau knowledge areas.

What we are working on Business process to integrate data and create new

products – documentation on data ingestion and processing, portfolio management

Policies, processes, and timelines to integrate data – policies in place, improving communication and knowledge management to standardize processes and introduce external data into operations and products

Enter MOA with suppliers – new federal, state, and third party negotiations underway

Summary

Seeking and testing more external data from federal, state, third party sources.

Need to: Understand data quality Design systems to accommodate "big" data Collaborate with program and statistical agencies