jsm slides--are state-level estimates for the ahs feasible
TRANSCRIPT
![Page 1: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/1.jpg)
Are State-Level Estimates for the American Housing
Survey Feasible?Ernest Lawley, Stephen Ash, Brian Shaffer,
Kathy ZhaDemographic Statistical Methods Division
Sample Design and EstimationUS Census Bureau
August 2015
![Page 2: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/2.jpg)
Disclaimer
This presentation is released to inform interested parties of ongoing research and to encourage discussion of work in progress. The views expressed on statistical, methodological, technical, or operational issues are those of the authors and not necessarily those of the U.S. Census Bureau.
2
![Page 3: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/3.jpg)
Today’s discussion
1. Background2. State-Level Estimation & Results3. Conclusion
3
![Page 4: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/4.jpg)
Background
4
![Page 5: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/5.jpg)
AHS-N Background
• The American Housing Survey – National sample (AHS-N) is a 30-year longitudinal survey of housing units (occupied and vacant)
• Produces estimates at the national level; for each census division (9) and census region (4) New England, Middle Atlantic, East North Central,
West North Central, South Atlantic, East South Central, West South Central, Mountain, Pacific
Northeast, Midwest, South, West
5
![Page 6: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/6.jpg)
AHS-N Background
• Data available from prior design: 1985 - 2013• Redesigned in 2015
• Data Collection currently going on• Sample selection is a two-stage process
= First stage Primary Sampling Unit (PSU) selection= Second stage Housing Unit (HU) selection
6
![Page 7: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/7.jpg)
AHS-N Sample Design
1. Primary Sampling Unit (PSU) selectionFirst Stage
• A PSU consists of a county or group of counties.• If 100,000+ housing units, Self-Representing (SR).
All SR PSUs were selected (170)
• If <100,000 HUs, Non-Self-Representing (NSR) NSR PSUs grouped into strata, 1 PSU per stratum was
selected (224), with probability proportional to size.
• The “prior design” (ie design being used for this study) selected PSUs by REGION
7
![Page 8: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/8.jpg)
AHS-N Sample Design (cont.)
8
Census Regions
![Page 9: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/9.jpg)
AHS-N Sample Design (cont.)
9
![Page 10: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/10.jpg)
AHS-N Sample Design (cont.)
10
Chelan, Douglas, Kittitas PSU
NevadaCalifornia
OregonIdaho
Montana
Wyoming
UtahColorado
ArizonaNew Mexico
Washington
Alaska
Hawaii
Selected this PSU!
![Page 11: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/11.jpg)
AHS-N Sample Design (cont.)
2. Housing Unit (HU) selectionSecond Stage
• Systematic Sampling• Within-PSU rate determined so that the overall
probability of selection was the same
11
![Page 12: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/12.jpg)
AHS-N Weighting Procedures
1. Nonresponse adjustment– Cell-based method to ensure the adjusted sum of
weights for interviewed units = total eligible weight.
2. First stage (PSU) adjustment− Aligns NSR PSU weights to the stratum housing
unit totals from the census3. Adjust Estimate to Independent Housing Unit
Estimate.4. Adjust Estimate to Population Controls
12
![Page 13: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/13.jpg)
AHS-N Variance EstimationWe want to evaluate variances (standard errors) to determine the feasibility of State-Level estimates.Replication Method (Fay and Train 1995)•Successive Differences Replication (SDR) used for Self-Representing (SR) PSUs•Balanced Repeated Replication (BRR) used for Non Self-Representing (NSR) PSUs
− BRR accounts for PSU selection (ie first-stage selection)
•160 Replicates created for each Interviewed Housing Unit
13
![Page 14: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/14.jpg)
AHS-N Variance Estimation (cont.)
14
![Page 15: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/15.jpg)
AHS-N Variance Estimation (cont.)
15
![Page 16: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/16.jpg)
State-Level Estimation & Results
16
![Page 17: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/17.jpg)
SR vs. NSR PercentagesPopulation Division 1980 Housing Unit counts aggregated by SR and NSR
County
17
![Page 18: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/18.jpg)
Methods for State-Level Estimates
• Method 1: “Brute Force”—Sum all SR and NSR cases in each state
• Method 2: Synthetic Method—Distribute NSR cases to each state within region, then sum
• Method 3: Adjust to Individual State & Population Control Totals & Raked
18
![Page 19: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/19.jpg)
Methods for State-Level EstimatesCHECKS:•State Estimate small CVs (less than 10-15%)•Compare to Population Division Housing Unit State Level Estimate to ensure each state’s estimate is in the “same ballpark”•“Good” states will be used to produce state-level estimates of Total HUs as well as other smaller domains (ie estimates of owners, renters, occupied, vacant, family composition, income, housing cost, housing quality, neighborhood quality, fuel type, etc)
19
![Page 20: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/20.jpg)
METHOD 1—”BRUTE FORCE”Sum all SR and NSR cases in each state
NOTE: %Diff = 100 * (Method 1 – Control)/Control
20
![Page 21: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/21.jpg)
METHOD 2—SYNTHETIC METHOD
• Each NSR sample (and interviewed) case receives a final weight
• This weight is distributed accordingly to each state within Region
- This occurs because we don’t know what other PSUs in the Region (which likely are in other states) were selected in the sample case’s Strata
21
![Page 22: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/22.jpg)
METHOD 2—SYNTHETIC METHODEXAMPLE
Suppose we sample and successfully interview a house in Lassen County, CA. This house receives a final weight of 2,250. We do not know the other PSUs in the West Region that the Lassen County PSU is representing. Because of this unknown, we have to “spread out” this weight of 2,250 in all states in the West Region.
22
![Page 23: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/23.jpg)
METHOD 2—SYNTHETIC METHODEXAMPLE (cont.)
Weight=2,250Percent of Housing Units in each of the West Region 13 states, with distribution received of the Lassen County housing weight:
23
![Page 24: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/24.jpg)
METHOD 2—SYNTHETIC METHODProportionally distribute NSR Weights to each State throughout region, then sum each state’s SR and (synthetically adjusted) NSR weights
NOTE: %Diff = 100 * (Method 2 – Control)/Control
24
![Page 25: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/25.jpg)
METHOD 3—Use State-Level Control Totals• Sum all SR and NSR cases in each state (ie “brute force
method” then make adjustments based on State-Level and Population control totals
• Control totals obtained from Census Bureau’s Population Division
- Pop Division control totals by County, aggregated to State total by SR, NSR
- Totals were raked using Black and Hispanic totalsMethod currently used; this method will change based on other
ongoing AHS research (Yesterday’s session 137 @8:50am “Results of Calibration Research for the 2015 American Housing Survey”)
25
![Page 26: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/26.jpg)
METHOD 3—Use State-Level & Population Control Totals
26NOTE: %Diff = 100 * (Method 3 – Control)/Control
![Page 27: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/27.jpg)
METHOD 3—Use State-Level & Population Control Totals
27
A Few Sub-Domains of the AHS
•Total Occupied•Total Vacant•Seasonal•New Construction•Mobile Homes
QUESTION OF THE DAY: What’s a good sample size within a sub-domain to yield a good estimate?
![Page 28: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/28.jpg)
METHOD 3—Use State-Level & Population Control Totals
28
Estimated Values of Sub-Domains
![Page 29: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/29.jpg)
Ongoing Research•Determination of sufficient sample size for sub-domains•Sample sizes must produce estimates with CVs < 15%•Some sub-domains may be suppressed due to high CVs
29
METHOD 3—Use State-Level & Population Control Totals
![Page 30: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/30.jpg)
METHOD 3—Use State-Level & Population Control Totals
30
Are these sample sizes sufficient to calculate feasible sub-domain estimates?
![Page 31: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/31.jpg)
METHOD 3—Use State-Level & Population Control Totals
31
State-Level CVs for Each Sub-Domain
![Page 32: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/32.jpg)
Conclusion
Future Research•Data from 2015 (New Design)•Improvements to Synthetic Method
– More known information to NSR weights
•Small Area Estimation Techniques?•Use of Calibration with HU/Pop Controls in new design
– Raking used with prior design
•Possible inclusion of more states
32
![Page 33: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/33.jpg)
Conclusion
“Just when you think you have all the answers, I CHANGE THE QUESTIONS!”– Rowdy Roddy Piper
33
![Page 34: JSM Slides--Are State-Level Estimates for the AHS Feasible](https://reader034.vdocuments.us/reader034/viewer/2022052705/58d17a6b1a28abed798b5ebf/html5/thumbnails/34.jpg)
QUESTIONS?
34
Ernie [email protected]
Steve [email protected]
Brian [email protected]
Kathy [email protected]