Best Practices Met Council Household Travel Survey (HTS)2010-2012
May 8. 2013
Metropolitan Council Household Travel Survey: Design, Implementation, and Lessons Learned
14th TRB Planning Applications Conference 5-9 May 2013Columbus, OH
Authors:Jason Minser, Abt SRBILaurie Wargelin, Abt SRBIJonathan Ehrlich, Metropolitan CouncilAnurag Komanduri, Cambridge SystematicsKimon Proussaloglou, Cambridge Systematics
Cemal Ayvalik, Cambridge Systematics.
The HTS was Part of a Total Travel Behavior Inventory with Cambridge Systematics as Prime
Household Travel Survey (HTS) Overview
• 19-counties in Minnesota (16) and Wisconsin (3)
• Address-based sampling approach – three tiered stratification plan
• Comprehensive inventory of . . . 12,000 households’ 24-hour travel for persons 6 years
of age and older
Subsample of 250 GPS households
• Special subgroups MnPass users
University students
4
Why Address-Based Sampling (ABS)?• Includes cell-phone only households
• Supplemental cell phone data collection is expensive with low response rates – not geographically based
• With ABS geographically based sample can be pulled and weighted by census tract
• Duel frame sampling (ABS/RDD) is difficult to weight
Weighting is not the solution to unrepresentative samples – only builds in the bias.
5
ABS Allows Targeting by HH Type
• As random, proportional data collection proceeds to 60-75% of recruitments, results are monitored and examined on a weekly basis
• Since ABS is census tract/block based – at this point - oversamples can be ordered for tracts/blocks with high incidence rates for under-represented variables of interest—i.e. those with H2R household characteristics
• Examples (Zero-vehicle HHs, 4+ person HHS)
• Implementation can be targeted by geographic area
This approach is called Responsive Interviewing Design (Groves and Herringa )
Highly Stratified ABS Sampling Plan• ABS improves the ability to define geographic strata
7 distinct regions sampled at differential rates Stratified by household size and total number of vehicles
Sub-Region
Minneapolis St. PaulSuburban
MinneapolisSuburban
St.PaulMPO Core
Ring Counties, MN
Ring Counties, WI
Sampling Rate 1% 1% 1% 1% 1% 0.5% 0.5%
Zero-Vehicle HH 189 100 116 90
One-Vehicle HH 411 269 721 516 321
Two-Vehicle HH
Three-Vehicle HHFour or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 147 95 200 189 94
Two-Vehicle HH 237 168 716 632 400 184 74
Three-Vehicle HHFour or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 84 72
Two-Vehicle HH 226 195 147
Three-Vehicle HH 153 137 105Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH
Two-Vehicle HH 105 100 411 358 279
Three-Vehicle HH 195 174 121Four or More Vehicle HH 116 147 74
158
72
105
95
126
147
84
72
174 116
74
111
7674
174
72
76
Sampling Cell Targets
190
95
Two-Person HH
95
105
72Three-Person HH
One-Person HH
116
76
Four or more Person HH
95
74
72
72
72
Imbedded Design for Hard – to – Reach Populations
• Clearly define populations
• But not every group can be hard to reach or treated as such
• Identify appropriate levers; not all levers work for each group
• Use post-collection techniques sparingly (e.g., weighting)
Address Based Sampling
Cell only
Geography
Multi-Methods
Phone
Web
DesignSponsor
Legitimacy Incentives Reminders
Design Qualities of 2010-11 Minnesota HTS
• Advance letters tailored to region and sponsorship emphasized
• Recruitment by phone (matched) and web (unmatched)
• Personalized activity-based travel diary • Reminder calls made evening before scheduled
travel day (weekdays only)• Contingent incentive offered in follow-up for
specific sub-groups ($20)• Follow-up by phone, web, and mail
GPS Subsample
• 250 Total Completed Households
• Data collection consistent with main study
• 7-day travel period One day diary Diary date randomly assigned
within 7-day period• Pilot study conducted
Incentive experiment conducted
• Full GPS/diary comparison conducted by PlanTrans, Inc.
Matched
ABS
Unmatched
Phone Recruit Web Recruit
Travel PacketDeployment
7-day Travel Period
Travel Packet Returned
Diary and GPS DeviceValidation
Completed HH Incomplete HH
Study Compliance and Participation
• Thirteen percent of households recruited by web• Parity between web and mail-back of retrievals
Phone becoming less of a retrieval method Method consistent with multi-method scenario
Recruitment
Phone Web TotalRetrieval Method N % N % N %
Phone 1,563 15.1 46 2.6 1,609 13.3
Web 4,292 41.5 1,211 68.1 5,503 45.5
Mail-back/Multiple Methods 4,469 43.3 522 29.3 4,991 41.2
Total Completed 10,324 100 1,779 100 12,103 100
Total Households Recruited 22,986 87.3 3,356 12.7 26,342 100
Design Strategies to Overcome Low Responding Markets• Tailored strategies are crucial throughout the process
to overcome low responding markets
Sampling
Recruitment
Retrieval
Customized LettersPhone and Web
Targeted Sampling
Unmatched/Cell only
4+ HH 0-vehicle
Low IncomeUniversity
Non-Responders
IncentivesPhone or Web
Reminder LettersTargeted Sample
Phone, Web, MailReminder CallsReassignment
Incentives
Sampling geographic strata requires close and continual monitoring
12
Recruitment at 60% Complete
Total Recruit 8,029
Percent Recruit 66.9%
Minneapolis St. PaulSuburban
Minneapolis
Suburban
St.Paul
MPO
Core
Ring Counties,
MN
Ring Counties,
WI
Zero-Vehicle HH 40% 44% 41% 49%
One-Vehicle HH 60% 55% 64% 68% 48%
Two-Vehicle HH
Three-Vehicle HH
Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 81% 51% 64% 71% 61%
Two-Vehicle HH 82% 86% 85% 78% 71% 78% 74%
Three-Vehicle HH
Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 32% 39%
Two-Vehicle HH 71% 56% 48%
Three-Vehicle HH 64% 76% 61%Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH
Two-Vehicle HH 88% 66% 76% 74% 61%
Three-Vehicle HH 69% 68% 55%Four or More Vehicle HH 69% 75% 72%
One-Person HH
PERCENT RECRUITED SAMPLE CELL
28%
Four or more Person HH
Three-Person HH
126%
65%
Two-Person HH
29%
94%
58%
71%
32%
43%
119%
67% 54%
52%
45%
Table 1. Target Adherence by Recruited Households - March 2011
33%
66% 63%79%
86%
51%
75%99%
22%
59%
33% 30%
57%
26%
85%
13
Retrieval at 30% Complete
Total Recruit 3,056 Percent Recruit 25.4%
Minneapolis St. PaulSuburban
Minneapolis
Suburban
St.Paul
MPO
Core
Ring Counties,
MN
Ring Counties,
WI
Zero-Vehicle HH 15% 16% 10% 11%
One-Vehicle HH 31% 29% 32% 29% 24%
Two-Vehicle HH
Three-Vehicle HH
Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 28% 24% 22% 24% 20%
Two-Vehicle HH 32% 32% 34% 32% 27% 29% 27%
Three-Vehicle HH
Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 6% 7%
Two-Vehicle HH 32% 18% 22%
Three-Vehicle HH 19% 28% 21%Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH
Two-Vehicle HH 30% 22% 23% 19% 19%
Three-Vehicle HH 22% 24% 18%Four or More Vehicle HH 19% 21% 28% 21%
One-Person HH
16%25%
31%33%
14%Two-Person HH
Four or more Person HH
3% 3%4% 6%
14%27% 24%
11% 4%
55% 49% 44% 30%
Three-Person HH
12% 11%28%
15% 26%35%
12%
29% 21%26%
Table 2. Target Adherence by Retrieved Households - March 2011
PERCENT RETRIEVED SAMPLE CELL
14
Strategies for Corrections at this Point
• Incentives for under-represented cells going forward: $20 for 0-Vehicle and 4+ Person Households. Added incentives for all hhs. with incomes <$25,000.
• Additional incentives: -- Reassignment: $20 offered to recruited hhs. in under- represented cells--that did not complete. Assigned a new travel day. Refusal Conversion-- $25 to University Oversample-- $10 to GPS households that had not returned units.
• Increased reminder calls to under-represented cells• Introduced targeted sample by census tract geography for
under-represented cells
15
Results of Corrective Actions on Final Retrieval
Minneapolis St. PaulSuburban
Minneapolis
Suburban
St.Paul
MPO
CoreRing Counties, MN
Ring Counties,
WI
Zero-Vehicle HH 100% 97% 53% 48%
One-Vehicle HH 137% 125% 111% 114% 97%
Two-Vehicle HH
Three-Vehicle HH
Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 134% 114% 89% 93% 88%
Two-Vehicle HH 134% 134% 128% 121% 104% 107% 166%
Three-Vehicle HH
Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH 31% 22%
Two-Vehicle HH 73% 72% 55%
Three-Vehicle HH 74% 101% 62%Four or More Vehicle HH
Zero-Vehicle HH
One-Vehicle HH
Two-Vehicle HH 128% 91% 88% 97% 81%
Three-Vehicle HH 81% 89% 80%Four or More Vehicle HH 67% 66% 96%
121% 104%98%
Table 3. Target Adherence by Retrieved Households - Final Delivered Sample
PERCENT RETRIEVED SAMPLE CELL
Three-Person HH
54% 33%120%
82% 107%
83%89%
43% 20%
145% 146% 151% 154%
3% 1%39% 25%
122%
76%
99%114%
One-Person HH
43%113%
88%123%
101%Two-Person HH
Four or more Person HH
Geographic RepresentativenessRegional Classifications Completed ACS 2006 - 2008 Diff
N % N % %Minneapolis 1,789 14.8 158,389 11.8 -3.0
St. Paul 1,122 9.3 108,708 8.1 +1.2
Core West – Suburban Minneapolis
3,313 27.4 345,262 25.7 +1.7
Core East – Suburban St. Paul 2,922 24.1 213,200 15.9 +8.2
Core South – MPO Core 1,638 13.5 278,352 20.7 -7.2
Ring Counties, Minnesota 841 6.9 176,191 13.1 -6.2
Ring Counties, Wisconsin 478 3.9 63,977 4.8 -0.9
Total 12,103 100 1,344,079 100 -6.2
Demographic RepresentativenessHH Characteristics Completed ACS 2006 - 2008 DiffHH Size N % N % %1 3,717 30.7 377,973 28.1 +1.4
2 4,804 39.7 445,443 39.7 0
3+ 3,582 29.5 520,663 38.8 -9.3
Total 12,103 100 1,344,079 100 -7.9
Number of Vehicles N % N % %
0 483 4.0 91,586 6.8 -2.8
1 3,806 31.4 413,357 30.8 +1.4
2+ 7,814 64.6 839,136 62.5 +2.1
Total 12,103 100 1,344,079 100 +0.7
Best Practices and Lessons Learned
19
Best Practices for a Representative HTS
• ABS allows inclusion of cell-only households and improves geographic targeting by key variables
• Establish legitimacy of sponsor – advance letters/postcards/website• Continual monitoring as random data collection proceeds as designed.• Continual interval data review/editing and reporting -- so that hhs. not
meeting quality standards can be replaced as data collection proceeds• Implementation of corrective actions at approximately the two-thirds
recruitment point
Results in a representative sample that can be weighted with bias minimized.