sample and design changes, the methodological results

Center for Survey Research in Public HealthUniversity of Minnesota

Human Services & Public Health DepartmentHennepin County

Sample and Design Changes, the Methodological Results from the

2006 SHAPE Survey.

Sheldon Swaney, BA Tim Zimmerman, BATodd Rockwood, Ph.D. Michael Davern, Ph.D.

12th Annual Minnesota Health Services Research ConferenceMarch 6, 2008



Symposium Agenda

Overview of the SHAPE projectAddress-based sampling frameCell phone-only households“Nearest neighbor” replacement methodologyMultiple mode



SHAPE Overview

Sheldon Swaney, BA




What is SHAPE

SHAPE, the Survey on the Health of All thePopulation, and the Environment, is apublic health surveillance and assessmentproject of the Hennepin County HumanServices and Public Health Department toperiodically survey the health of residents ofHennepin County.



SHAPE History

Three waves of the survey: SHAPE 1998SHAPE 2002SHAPE 2006

www.hennepin.us/SHAPEQuestionnairesDocumentationReports



Collaborative Contributions: Hennepin County and Community

Hennepin – SHAPE Project TeamSheldon SwaneyMelissa BarkerMei DingUrban LandremanTim ZimmermanAnn Kinney - MDH

CommunityFace Value ResearchersCommunity Organizations

Meg Hargreaves

Michelle Chiezah

Matthias Kirch



Collaborative Contributions: University of Minnesota

In-Kind Contributions

Michael Davern - Sampling and administrationKaren Virnig - Instrument development, translation and administrationTodd Rockwood - Instrument development, translation and administration

ExpertiseKaren Turner & Joe Hallgren – ProgrammingMary Sigrah – TranslationInterviewing Staff



Project Cycle 2006

18 monthsAnalysis/Reporting

4 months6 monthsData set preparation

8 months6 monthsSurvey Implementation

12 months9 monthsPlanning & DesignMethodology & Content

ActualEstimateAction



What’s New in SHAPE 2006

New partnersNew sampling frameNew geographyNew data collection methodsUpdated adult questionnaireNew child questionnaire



SHAPE Reports 2006



Response Rates 2006

44.6%52.8%27.6%Adult response rate

91.4%92.3%86.0%Child from adult response rate

51.9%

2,097

1,941

5,590

Phone

51.9%-Child only response rate

2,097-Child only

1,941-Child from adult

8,1912,601Adult

TotalMailQuestionnaire



Sample Frame

Tim Zimmerman, BA




Background

In 2002, the SHAPE survey relied mainly on a listed telephone number sample frame

We did this to make sure we could analyze by geography, and map the results

Sampling frame was a telephone list purchased from Marketing Systems GroupNot included in the sampling frame:

Households without telephonesHouseholds with unlisted telephone numbers

SHAPE 2002 Sampling Frame



SHAPE 2002 Sampling FrameListed telephone sample14 geographic strata

11 Minneapolis community areas3 suburban areas + 2 sub-areas

Over sample racial/ethnic groupsTelephone interviews

200 in-personResponse rate 67%9,500 adult respondents



In 2006 we were interested in examining alternative sample framesThe proportion of the population with land-line telephones was decreasing nationwide

Decreasing number listed land-line phonesIncreasing cell phone only households

Increasing call screeningDecreasing response rates




Those without land-line telephone service tended to have different health characteristics than those with land-lines

More likely to smoke, drink, and lack health insurance according to national numbers from Blumberg, et al.

This concerned us greatly




Sample Frame Assessment Conclusion

After careful consideration, the team decided to rely on the Delivery Sequence File (DSF) from the U.S. Postal Service (purchased through Marketing Systems Group).




DSF advantages

It provided fairly good coverage in most of the countyAllowed us to map each addressAllowed us to try some new methods, such as sampling nearest neighbors for non-responding households




Driven by planned reports6 sampling strata0.6% of the households will be sampled in each of the sampling strataIn each of the 6 sampling strata, there are specific census tracts in which a 5% over-sample will drawn

These census tracts to be over-sampled were selected based on high representation of 4 racial and ethnic populations

SHAPE 2006 Sample Design



SHAPE 2006 Geographic Stratification

6 sampling strata (based on political boundaries)

4 in Minneapolis2 in the suburbs

Maple Grove

MinnetonkaBeach

GreenwoodTonka Bay

Osseo

Robbinsdale

Fort SnellingChanhassen

Medicine Lake

Rockford

Loretto

SpringPark

Woodland

ExcelsiorSt. Bonifacius

Long Lake

Maple Plain

St.Anthony

Deephaven

Hanover

Wayzata

New Hope

Hopkins

Brooklyn Center

Rogers

Mound

Golden Valley

St. Louis Park

Richfield

Crystal

Champlin

Shorewood

Brooklyn Park

Greenfield

Minnetonka

Independence

Edina

Eden Prairie

Minnetrista

Bloomington

HassanDayton

Medina Plymouth

Corcoran

Orono

Minneapolis

06

05

04

0201

03

Stratification boundariesNear-North/CamdenNortheast/University/LongfellowCentral/Phillips/PowderhornCalhoun Isles/Southwest/Nokomis1st Ring2nd Ring/3rd Ring/Far West

Stratification Geography



SHAPE 2006 Geographic Over-sampling

SHAPE 2006:Oversampled Census TractsHennepin County City of Minneapolis

Maple Grove

MinnetonkaBeach

GreenwoodTonka Bay

Osseo

Robbinsdale

Fort SnellingChanhassen

Medicine Lake

Rockford

Loretto

SpringPark

Woodland

ExcelsiorSt. Bonifacius

Long Lake

Maple Plain

St.Anthony

Deephaven

Hanover

Wayzata

New Hope

Hopkins

Brooklyn Center

Rogers

Mound

Golden Valley

St. Louis Park

Richfield

Crystal

Champlin

Shorewood

Brooklyn Park

Greenfield

Minnetonka

Independence

Edina

Eden Prairie

Minnetrista

Bloomington

HassanDayton

Medina Plymouth

Corcoran

Orono

Minneapolis

Shingle Creek

Lind-Bohanon

Columbia Waite Park

Windom Park

Audubon Park

Victory

Webber-Camden

Cleveland

FolwellMcKinley

MarshallTerrace

HollandBottineau

Sheridan

LoganPark

Northeast Park

Mid-City Industrial

Como

BeltramiSt.

AnthonyEast

St. AnthonyW estNear North

HawthorneJordan

Willard-Hay

Sum

ner-G

lenwood

NicolletIsland

Marcy-Holmes

University Prospect Park

Harrison

Bryn-Mawr

Kenwood

Lowry Hill

North Loop

DowntownWest Downtown

East

Elliot ParkLoring Park

Stevens Square

Seward

CooperLongfellow

Howe

Hiawatha

MorrisParkWenonah

KeewaydinMinnehaha

Ericsson

Standish

Corcoran

PowderhornPark

Central

BancroftBryant

ReginaNorthrup

Hale

Field

Page

Diamond Lake

Windom

KennyArmatage

Fulton LynnhurstTangletown

KingField

East HarrietLinden Hills

West Calhoun Ecco Carag

Whittier

Lowry H

ill East

Cedar-Isles

Cam

den Industrial

East Isles

Cedar-Riverside

Lind-Bohanon

Lyndale

W estPhillips

MidtownPhillips East

Phillips

Ventura Village

Stratum 01 oversampleStratum 02 oversampleStratum 03 oversampleStratum 04 oversampleStratum 05 oversampleStratum 06 oversample

City/n'hood boundariesInterstate highways



Hennepin County 2006 Sample Frame

The sample frame assessment questions were:

What proportion of the housing units in Hennepin county were in the DSF? How did this vary by geography?What proportion of the DSF records were linkable to listed telephone numbers so they could be called? How did this vary by county geography?

Sample Frame Assessment



To answer these questions, we purchased all of the DSF addresses in the county, and all of the listed telephone numbers in the county, from Marketing Systems Group





Sampling frame is the U.S. Postal Service Delivery Sequence File (DSF)

Includes all housing units to which mail can be delivered (485,000 households)It is current (as of November 2005)Includes the delivery sequence code (effectively sequencing the file in the order in which mail is delivered)We geocoded the DSF file, and assigned census tract numbers to each address




Listed telephone file (265,000 households)Addresses from the telephone list were matched against the DSF file55% of the DSF household addresses were successfully matched to the listed telephone numbersSampled DSF records linked alternate “Nearest Neighbor” with a phone number (more on this later)




01 7,109 51.3% 1.0401A 15,727 36.1% 1.0502 33,787 43.2% 1.0202A 11,012 33.5% 0.9803 25,099 33.2% 1.1203A 23,553 28.0% 1.0504 48,499 57.8% 1.0004A 4,874 52.2% 1.0005 92,784 61.3% 1.0405A 19,886 47.9% 1.0206 186,698 63.3% 1.1206A 16,544 46.1% 1.10Total 485,572 54.7% 1.06

Percent of DSF Addresses with Listed Telephone

Ratio of DSF Addresses to Occupied Census 2000 Housing

DSF Addresses as of November 2005

Geographic Sample Stratum

Table 1: Delivery Sequence File Housing Units, and Delivery Sequence File Housing Units with a Listed Telephone Number Compared to US Census 2000 Total Occupied Housing Units by Survey Strata




Q: What proportion of the housing units in Hennepin county were in the DSF?Q: How did this vary by geography?





Q: What proportion of the DSF records were linkable to listed telephone numbers so they could be called?Q: How did this vary by county geography?




2002Listed telephones14 geographic strataOver sampling –racial/ethnic groupsTelephone interviews

9,500 adults

Response rate: 67%

2006Address frame6 geographic strataOver sampling –racial/ethnic groupsMixed mode – mail & telephone interviews8,000 adults / 4,000 childrenResponse rate: 45%

2002 & 2006 Survey Design Comparison




What is the effect of using the address-based sample frame vs.. the listed telephone-based frame?

How did estimates of health status, behaviors and other factors affecting the health of the population change by using the more complete address-based coverage?




Percentage who, when sick or needing care, have no regular place of care to which to go

20.3%

11.6%

0%

10%

20%

30%

40%

50%

Without listed phone With listed phone

Perc

en

t o

f ad

ult

s Among primary respondents



Percentage who had an unmet mental health care need

21.9%

10.4%

0%

10%

20%

30%

40%

50%


Perc

en

t o

f ad

ult




Percentage who have not had their blood cholesterol checked within the past five years

18.2%

12.0%

0%

10%

20%

30%

40%

50%


Perc

en

t o

f ad

ult




23.7%

14.5%

0%

10%

20%

30%

40%

50%


Perc

en

t o

f ad

ult

s

Percentage who had five or more drinks on an occasion at least once during the past 30 days

Among primary respondents



12.4%

6.0%

0%

10%

20%

30%

40%

50%


Perc

en

t o

f ad

ult

s

Percentage who missed a rent or mortgage payment during the past 12 months because they did not have enough money




33.5%

23.5%

0%

10%

20%

30%

40%

50%


Perc

en

t o

f ad

ult

s

Percentage who are never involved in any school, community, or neighborhood activities




14.3%

3.2%

0%

10%

20%

30%

40%

50%


Perc

en

t o

f ad

ult

s

Percentage who moved two or more times during the past two years




Cell Phone-only Households

Michael Davern, Ph.D.(Karen Soderberg, Mei Ding, Tim Zimmerman and Sheldon Swaney all helped me

put this presentation together)




Why the Concern About Cell Phones and Surveys?

Random Digit Dial Surveys traditionally only dial land line telephone numbers

Why?The proportion of the population with land-line telephones was decreasing nationwide

Mainly because cell phone only households were increasing in number

Those people living in cell phone only households tended to have different health characteristics than those with land-lines

More likely to smoke, drink, and lack health insurance according to national numbers from Blumberg et al

This greatly concerns many survey researchers



Rates of Cell Phone Only

1.6% 1.7% 1.8% 1.7% 1.6%

1.5% 1.8% 1.9%2.3%

1.7%

4.4%

5.4%

6.7%

7.7%

9.6%

11.8%

12.6%

3.7%

4.9%

5.8%

7.6%

8.6%

11.6% 11.9%

1.5% 1.8%

2.3%1.8%

0%

2%

4%

6%

8%

10%

12%

14%

Jan - Jun 2004 Jul - Dec 2004 Jan - Jun 2005 Jul - Dec 2005 Jan - Jun 2006 Jul - Dec 2006 Jan - Jun 2007

Adults with wireless service only

Children with wireless service only

Adults with no telephone service

Children with no telephone service

(Source: Blumberg and Luke 2007)



Our Research Questions

How many people in Hennepin County lived in cell phone only households and did they show similar health characteristics as the nationwide sample?How would our estimates have been different if we had used a listed telephone number sample frame in 2006 like we did in 2002?



SHAPE 2006 Survey Basics

The basic survey was of 4,304 adults 18 years of age and olderMixed mode data collection using a mail survey and phoneOversampled low-income and minority areasUnweighted response rate was roughly 41% (we are still awaiting the final AAPOR calculation)The survey was conducted between May, 2006 and March 2007



Demographics of Cell Phone Only Population Versus Others

Socio-demographic characteristicsAge 18-24 8.2 30.7 -22.5 ***Age 65 and older 16.9 0.8 16.1 ***Male 47.0 48.0 -0.9Race and ethnicityHispanic/Latino 3.1 3.0 0.1White (Non-Hispanic) 84.5 87.2 -2.7Black/African American (Non-Hispanic) 7.1 5.0 2.1American Indian (Hon-Hispanic) 0.3 0.7 -0.5Asian/Pacific Islander (Non-Hispanic) 3.8 3.4 0.3Having less than High School education 4.6 0.7 3.9 ***Household income below 100% FPL 6.8 11.6 -4.8Household income below 200% FPL 17.3 22.8 -5.5From Minneapolis 30.0 54.2 -24.2 *** Overall rate (weighted by cellphone weight) 89.5 10.5

*** p <.001

* p < .05** p<.01

Selected Characteristics by Status of Telephone Service at Home for Hennepin County Adults: SHAPE 2006

DifferenceLand Line Telephone

Cell only HH



Demographics of Cell Phone Only Population Versus Others

Community and neighborhood Never involved in school, community or neighborhood activities 21.9 31.4 -9.5 *People in this neighborhood are afraid to go out at night due to violence - Strongly agree or somewhat agree

18.4 24.9-6.5

Gangs are a serious issue in this neighborhood - Strongly agree or somewhat agree

12.8 18.1-5.3

Worry food would run out before had money to buy more- Often or Sometime

9.2 16.4-7.2 *

Moved at least 2 times in past 2 years 5.3 41.4 -36.1 ***Missed a rent or mortgage payment due to not have enough money 6.0 15.5 -9.5 **



Cell only HH



Key Health Characteristics of Cell Phone Only Population vs.. Others

Overall Health Status Selfrated health as poor or fair 9.0 8.3 0.7Recent frequent mental distress 9.7 13.9 -4.3Currently having asthma 8.0 7.7 0.2Ever been diagnosed with diabetes 6.7 0.3 6.3 ***Ever been diagnosed with Depression 22.6 26.3 -3.7Serious psychological distress 1.9 4.3 -2.5Obese (BMI>=30.0) 18.7 9.7 9.0 ***Overweight (BMI>=25.0, including Obese) 55.8 45.0 10.9 *



Cell only HH




Health access & utilization

Currently uninsured 6.1 8.4 -2.3Difficult in paying for premium, co-pay, deductible for you & your family 25.8 35.0

-9.3 *Difficult in paying prescription each month 10.8 9.5 1.4Having no regular source of care (personal Dr or health care provider 22.0 47.9

-26.0 ***Having no regular place of care (DrOffice/ clinic,VA clinic/hospital 14.6 29.2 -14.6 ***Speaks a language other than English at home 4.3 2.6 1.7Unmet medical care needs in past 12 m 12.7 22.4 -9.7 **Unmet mental health care needs in past 12 m 13.1 27.7 -14.6 ***Having no cholesterol check in 5 years or never had one 14.4 34.9 -20.5 ***



Cell only HH




Lifestyles, nutrition and exercise

Having 5 or more servings of veg/fruit yesterday 34.6 37.1 -2.5Currently smoking 16.5 24.1 -7.6 *No leisure time physical activity in past 30 days 15.0 7.8 7.2 **Binge drinking at least once in past 30 days 18.3 55.1 -36.8 ***



Cell only HH



Logistic Regression Analysis

It’s clear that people who live in cell phone only households are different from those who do notHowever, the key question for researchers is after controlling for variables used to make weighting adjustments (age, sex, race, strata, etc.) do the differences remain?




Overall Health Status Selfrated health as poor or fair (A01) 1.1Recent frequent mental distress (A03) 1.1Ever Been diagnosed with having asthma (A05) 0.8Ever been diagnosed with diabetes (P:A10a A10b, M:A11) 0.3 *Ever been diagnosed with Depression (A26) 1.2Serious psychological distress (k6, A32a to A32f) 1.1Obese (BMI>=30.0) (A22 A23) 0.8

Selected health Indicator

Cell Phone Only Odds Ratio

yRelative to Others Regressed on Selected

Characteristics Controlling for Variables Used in Weighting Procedures




Health access & utilization

Currently uninsured (B01_1 to B01_9) 1.3 *Difficult in paying for premium, co-pay, deductible for you & your family (B03)

1.2

Difficult in paying prescription each month (B04 B05) 0.9Having no regular source of care (personal Dr or health care provider (B07)

1.5*

Having no regular place of care (DrOffice/ clinic,VA clinic/hospital (B08) 1.4*

Unmet medical care needs in past 12 m (B13 B14) (population) 1.3

Unmet mental health care needs in past 12 m (B16 B17) (population) 1.6 **Having no cholesterol check in 5 years or never had one (B24) 1.2








Lifestyles, nutrition and exercise

Having 5 or more servings of veg/fruit yesterday(C01 C02) 1.0Currently smoking (C19 C20) 1.5 *No leisure time physical activity in past 30 days (C09) 0.8Binge drinking at least once in past 30 days (C05 C08) 2.4 **Community and neighborhood Never involved in school, community or neighborhood activities (D02) 1.4 *People in this neighborhood are afraid to go out at night due to violence - Strongly agree or somewhat agree (D05d)

0.9

Gangs are a serious issue in this neighborhood - Strongly agree or somewhat agree (D05e)

0.9

Worry food would run out before had money to buy more- Often or Sometime (D11)

1.1

Moved at least 2 times in past 2 years (D12) 3.7 **Missed a rent or mortgage payment due to not have enough money (D13) 2.0

**







Conclusions

The cell-phone only population Very different with respect to demographic and health characteristicsAfter controlling for demographic characteristics used in weighting most differences are no longer significant

Diabetes, Access to health care (no regular source of care, having an unmet need in the last 12 months), current smoking and binge drinking remainSome community variables also remain significant

Bottom line is that weighting adjustments are still working but not well enough in some areas

These domains are likely to grow



“Nearest neighbor”replacement methodology

Sheldon Swaney, BA




Survey error

Non-coverage errorSampling frame improvement

Total non-response errorReplacementWeighting

Item non-response errorImputation



Sample Replacement 1998 & 2002

Sampling frame – Listed telephonesSurvey method – Telephone interviews1998 – 19 sampling strata2002 – 14 sampling strata with 11 over-sample substrata = 25 strataResponse failures were replaced with another randomly selected household from the same sampling strata



Sample Replacement 2006

Sampling frame – Mailing addresses linked with listed telephone numbersSurvey method – Mail - Mail with Telephone follow-up 2006 – 6 sampling strata with 6 over-sampled substrata = 12 strata



Assumptions

Non-response rates are not equally distributed geographically

Response rates for mail surveys 10 – 30%

Neighbors are more likely to have similar demographic and health characteristics than people living farther away



Sample Replacement Plan 2006

“Nearest Neighbor”Alternative sampling units were defined individually for each primary sampling unit at the time of sample selectionAlternative unit was defined as being the nearest record (geographic proximity) in the sampling frame that matched the primary sampling unit selection strata and that also had an associated listed telephone number



Replacement Sampling 2006: “Nearest Neighbor"



Distance between Primary and Alternate (N = 16,948)

5813Distance inMeters

112# of Mail stops

MeanMedianMeasure



Respondents by Primary and Alternate

2,754138050.1%

1,37449.9%

Cycle 2(n=6,000)

3,29143.2%

1,91139.3%

Alternate

7,6204,32956.8%

Total

4,8662,95560.7%

Cycle 1(n=8,500)

TotalPrimaryCycle



Sampling strata

Oversampled areas

Alternate as % of all completesNo completes

0% - 20%

21% - 40%

41% - 60%

61% - 80%

81% - 100%

SHAPE 2006: Alternate Completesas a % of All Completes

Data Displayed by Census Block Group

Hennepin County

Source: SHAPE 2006



Demographic Factors for Primary vs. Alternate Respondents

3.7%5.0%2.7%Hispanic

3.7%3.8%3.6%Asian/Hawaiian/PI non-Hispanic

12.7%16.1%10.0%Black/African American non-Hispanic

1.1%1.5%0.8%American Indian non-Hispanic

1.3%1.7%1.2%Two or more non-Hispanic

61.6%60.3%62.6%Female

Race/Ethnicity –

77.5%72.1%81.7%White non-Hispanic

38.4%39.7%37.4%Gender - Male

20.9%20.4%21.2%65 + years

18.0%18.0%18.0%55-64 years

21.9%21.8%21.9%45-54 years

19.3%19.1%19.4%35-44 years

14.8%15.3%14.4%25-34 years

5.2%5.3%5.1%Age - 18-24 years

TotalAlternatePrimaryFactor



Sampling and Survey Design Factors for Primary vs. Alternate Respondents

15.5%8.8%20.5%06 Suburban 2nd 3rd Ring

4.4%3.1%5.4%04 Minneapolis South

3.9%3.4%4.2%04A Minneapolis South Over-sample

8.2%6.4%9.6%05 Suburban 1st Ring

14.7%14.6%14.8%05A Suburban 1st Ring Over-sample

2.1%2.3%1.9%03 Minneapolis Central

16.1%22.6%11.2%03A Minneapolis Central Over-sample

11.3%11.1%11.5%06A Suburban 2nd 3rd Ring Over-sample

8.9%10.4%7.8%02A Minneapolis East Over-sample

3.1%2.9%3.3%02 Minneapolis East

11.3%14.1%9.1%01A Minneapolis North Over-sample

0.5%0.3%0.6%01 Minneapolis North

Sample Strata

59.0%67.4%52.6%Over-Sample

32.2%10.1%49.0%Survey Mode (mail)

TotalAlternatePrimaryFactor



Overall Health Status for Primary vs. Alternate Respondents

25.1%

3.4%

24.9%

9.1%

8.9%

11.6%

16.2%

Total

1.091

1.573*

1.008

0.909

1.063

1.025

1.057

Odds Ratio

27.2%23.5%Obese (BMI >=30.0)

4.1%2.9%Serious psychological distress

24.6%25.1%Ever diagnosed with depression

9.5%8.8%Ever diagnosed with diabetes

9.2%8.7%Current asthma

12.0%11.3%Frequent mental distress

18.5%14.4%Self-rated health status – poor or fair

AlternatePrimaryOverall Health Status



Health Access and Utilization for Primary vs. Alternate Respondents

13.5%

12.4%

14.8%

5.8%

12.9%

21.3%

15.5%

31.2%

8.3%

Total

1.048

1.162

1.136

0.965

1.134

1.058

1.043

1.089

1.102

Odds Ratio

12.0%12.7%Unmet mental health care needs

13.8%13.3%No cholesterol check past 5 years

15.2%14.4%Unmet medical care needs

7.4%4.5%Language other than English

12.2%13.5%No regular place of care

22.8%20.2%No regular source of care

15.9%15.2%Difficulty pay prescriptions

33.0%29.9%Difficulty pay premiums, co-pays …

9.8%7.1%Currently Uninsured

AlternatePrimaryHealth Access & Utilization



Lifestyles, Nutrition and Exercise for Primary vs. Alternate Respondents

15.6%

21.8%

18.9%

31.5%

Total

0.976

1.048

0.981

0.895

Odds Ratio

14.2%16.6%Binge drinking in last 30 days

24.3%19.9%No leisure time physical activity

19.7%18.2%Current smoker

29.0%33.5%5 or more servings fruits/vegetables

AlternatePrimaryLifestyles, Nutrition and Exercise



Community and Neighborhood Factors for Primary vs. Alternate Respondents

8.1%

5.5%

13.3%

29.8%

38.3%

26.9%

Total

1.249*

1.366*

1.269*

1.179*

1.296*

1.292*

Odds Ratio

9.2%7.3%Missed rent or mortgage payment in last 12 months

5.3%5.6%Moved at least twice in past two years

15.0%12.1%Worried food would run out before had money to buy

37.1%24.5%Gangs are a serious problem in neighborhood

46.7%32.0%Afraid to go out at night in neighborhood due to violence

28.4%25.8%Never involved in school, community or neighborhood activities

AlternatePrimaryCommunity and Neighborhood



Nearest Neighbor conclusions

Differences between Primary and Alternate respondents did not require weighting adjustmentDesign experiment needed to determine Primary and Alternate similaritiesOperational complexity of “Nearest Neighbor”



Multiple Modes

Todd Rockwood, Ph.D.




Mode Effects in SHAPE III

Embedded experiment in which sampled elements randomized to either Mail or Telephone ModePurpose

Evaluate question areas in which mode effects are presentMeasure magnitude of mode effects if present



Analysis

Simple comparison of responses between modesIntent of the analysis is to evaluate simple differences that would emerge if a study did not use a mixed mode design

Basic differences that would be found if a study was done only be telephone or mail

Multi-variate in which demographic and sampling characteristics that show significant differences between mode

Controls for differences that emerge relative to differences in individuals who respond to mail versus telephone surveys



Questionnaire Domains Evaluated

Health StatusDepressionScreening Tests Insurance/Health Service UtilizationReceipt of Public/Medical AssistanceCommunity EvaluationDiscrimination



Demographics

Mail TelephoneN 2,102 703

Male 36% 42%Female 64% 58%

X2 p.01

White 84% 82%Hispanic/Latino 2% 3%African American 10% 10%American Indian 1% 1%Asian/PI 3% 3%Other 1% 1%Two Plus Race/Ethnic ity 1% 1%



Demographics

Mail Telephone18-24 6% 5%25-34 16% 13%35-44 19% 20%45-54 21% 22%55-64 18% 17%65+ 21% 23%

Less than HS 4% 7%HS/GED 17% 19%Some College 32% 30%College Grad+ 47% 44%

X2 p .01

Born in US 93% 89%Not Born in US 7% 11%

X p.01



Sampling

Mail TelephoneRegular Sample 46% 48%Oversample 54% 52%

MPLS-NearN/Camden 10% 9%MPLS-NE/Uni/LF 13% 9%MPLS-Ctr/Phllips/Ph 16% 12%MPLS-Cal I/SW/Nokomis 9% 9%Suburban-1st ring 23% 27%Suburban-2nd ring 31% 34%

X2 p .01



Variables Included in Multi-Variate Analysis

GenderEducation

High School or Less/More than High SchoolBirthplace

Born in US/Not Born in USSampling Geography



Health Status

Few differences between modesMail Mode

Higher rates borderline diabetes and borderline hypertension

Phone ModeHigher rate MD told them they are overweight (w/in past year)

Item Type Number of ItemsBi-Variate # Significant

Multi-Variate # Significant

Not-Threatening 8 2 2Moderate Threat 3 1 1Threatening 0 0 0

Total 11 27% 27%



Health Status

0%

5%

10%

15%

20%

25%

30%

35%

Boderline Diabetes BoderlineHypertension

Overweight

Condition

Per

cent

Yes

Mail Telephone



Depression

Major differences between modesMail mode manifests more depression




Total 9 78% 56%



Depression

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Nothing Cheer Up Nervous Hopeless

Issue

Perc

ent (

Non

e of

the

Tim

e)

Mail Telephone



Screening

Major differences between modesPhone Mode higher rates of screening in past year for:

Blood CholesterolFBOTSigmoidoscopy/Colonoscopy




Total 3 100% 100%



Screening

0%

10%

20%

30%

40%

50%

60%

70%

Cholesterol FBOT Sig/Colon

Perc

ent Y

es

Mail Telephone



Insurance/Health Services

Moderate differences between modesPhone Mode

Higher rates of insuranceFewer problems with money relative to insuranceMore likely to get care at clinics




Total 6 67% 67%



Insurance/Source of Care

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Insured Difficulty Paying Clinic

Per

cent

Yes

Mail Telephone



Public Assistance

Few differencesMail Mode

Higher use of food shelvesOverall more public assistance (mean .37)

Phone ModeHigher use of school lunchesOverall lower public assistance (mean .20)



Not-Threatening 0 0 0Moderate Threat 0 0 0Threatening 11 2 2Count [11] 1 1

Total 11 18% 18%



Public Assistance

0%

1%

2%

3%

4%

5%

6%

7%

Food Shelves School Lunches

Per

cent

Yes

Mail Telephone



Major DifferencesPhone Mode

Live in better communities and are more socialMail Mode

Live in communities with fewer gang problemsAll items but 1 demonstrate strong social desirability in response

Community Assessment




Total 10 100% 70%



Community Assessment

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Interaction withNghbors

Help EachOther

Trusted Raise Kids

Per

cent

Agr

ee

Mail Telephone



Discrimination

Mail mode higher report of DiscriminationJobMortgageRestaurantCourts

Mean number Mail (.35) Telephone (.19)



Not-Threatening 0 0 0Moderate Threat 0 0 0Threatening 10 4 4Count [10] 1 1

Total 10 40% 40%



Discrimination

0%

5%

10%

15%

20%

25%

At Work Restaurant Court System

Per

cent

Yes

Mail Telephone



Summary

Health StatusLittle Difference

DepressionMajor Difference

Mail mode much more likely to profile a population dealing with depression

ScreeningMajor Differences

Phone mode much more likely to profile a population that screens for health problems



Summary

Insurance/Health ServicesMajor Differences

Mail mode more likely to be un-insured and have monetary issues relative to insuranceMail mode less likely to receive care at a clinic

Public/Medical AssistanceFew Differences

CommunityMajor Differences

Telephone mode lives in utopia



Summary

DiscriminationModerate Differences

Mail mode higher rates of discriminationDirect Mode Effects

Half of the items demonstrate significant differences by modeMuch higher for items that are dealing with threatening or moderately threatening topics


Multi-Variate # Significant Percent

Not-Threatening 12 4 4 33% / 33%Moderate Threat 18 15 12 83% / 67%Threatening 30 14 12 47% / 40 %

Total 60 33 28 55% / 47%

sample and design changes, the methodological results

Documents