1 msu model grant overview dennis martell sandi smith michigan state university

1

MSU Model Grant Overview

Dennis Martell

Sandi Smith

Michigan State University

MSU MODEL GRANT

• What Makes MSU a Model Program?• 1. Unique make up of team

– Service, Survey, and Academic Experts• 2. Effectiveness of global campaign to

reduce extreme drinking.• 3. Unique application of social norms

messaging on celebratory events.• 4. Unique focus on use of protective

behaviors to reduce harm.

3

Outline of the Presentation

• Overview of evaluations• Intervention design and evaluation• Social norming models• MSU’s social norming program and

evaluation design• Formative research on celebratory

drinking• Dissemination and enhancement

strategies

4

Evaluations in the Life Course of Programs

Adapted from: Rossi, P., and H. Freeman, Evaluation: A Systematic Approach

OutputOutcome/ImpactCost/BenefitCost Effectiveness

Decision to Act

Needs AssessmentFormative Evaluation

ProcessConformity to PlanTarget Reach/Coverage

Program Life Course

Implementation

Institutionalization

Assessmentof Utility

Outcome

Planning Monitoring

Problem EvaluationsConceptualization & Design

5

To Evaluate a Program's Outcome, We Need to Know . . .

What was the program supposed to accomplish? (Problem to change) How was it to accomplish this? For whom?

Why was the intervention supposed to have the intended effect? (Conceptual Model)

Did change occur? (Outcome/Impact) How much? Is the change associated with the program elements? Can alternative explanations for the change be ruled out? How much did it cost?

6

The Social Norming Conceptual Model of

High-Risk Drinking

Harm

Consumption

Perception ofNorms:

Descriptive

Perception ofNorms:

Injunctive

FamiliarityWith theCulture

DesireTo be

Normal

Alcohol poisoningPhysical injuryAuto accidentsViolenceSexual assaultSexual risk behaviorAcademic performance

TypeQuantity AbsorptionDurationPace

ProtectiveBehaviors

Setting limits 1 drink per hourDesignated driver Drink look-a-likeWatch out for friends Stay with same groupStay with same alcohol Eating food

7

The MSU Social Norming Intervention Strategy for High-Risk Drinking

Harm

Consumption

Perception ofNorms:

Descriptive

Perception ofNorms:

Injunctive

FamiliarityWith theCulture

DesireTo be

Normal

ProtectiveBehaviors

Intervention:Posters

Table Tents

Intervention: Flyers

Newspaper Ads

AOP

8

The Initial Question:

Was there a discrepancy between the perceived norm around drinking and the actual drinking norm?

i.e., Was a Social Norming approach an appropriate intervention strategy?

9

Difference Between Perceived Norm and Actual Behavior Before Starting Social Norming Campaign

6.1

5.4

Perceived Norm(Mean)

Actual Norm (Mean)

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

Number of DrinksLast Time Partied/Socialized

2000

Source: MSU NCHA 2000

The average perceived number

of drinks consumed by the typical student the

last time s/he partied is 13%

greater than the actual average

}

Difference Between Perceived Norm and Actual Behavior Before Starting Social

Norming Campaign

10

Overall Actual %

0-4 = (46.6%)

% Who Believe Typical Student Drank 0-4 Drinks Last Time Partied/Socialized by Gender and Status, 2000

Grad % 0-4 = 57.8%

Undergrad % 0-4 = 43.7%

Source: MSU NCHA 2000

2007 National Social Norms Conference 11

Examples of Ads, etc.

12

Data Collection Strategy for Evaluation

• Formative – Focus groups for ads

• Monitoring/Process – Web survey near end of each semester– Undergrads only– N = 1,000 – 1,200 @– Perceptions of others, celebration event drinking,

protective behaviors, familiarity with ads• Monitoring/Outcome

– NCHA (web) every 2 years (Feb.-March)– Grad and Undergrad– N = 900 – 1,400– Overall drinking, harm, protective behaviors

13

The Evaluation Questions

• Did the interventions reach the targeted audience?

• Did change occur in normative perceptions? • Is change associated with the intervention? • Did change occur in protective behaviors? • Did change occur in drinking behavior?• Did change occur in amount of harm?• Is there reason to believe this isn’t just

coincidence?

14

Did the Interventions Reach the Targeted Audience?

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

• Males as likely to report seeing ads as females

• Freshmen more than Sophomores more than Juniors more than Seniors

• Average number times reported seeing ads varied from 2.8 (Fall, 06) to 7.5 (Spring, 04)

• No difference between males and females

• Generally, Freshmen saw more times than Sophomores more times than Juniors more times than Seniors

Source: Celebration Surveys 2-9, n=1,211; n=1,040; n=1,277; n=1,073; n=1,334; n=1,110; n=891; n=1,405

% of Undergrads Reporting Seeing at Least 1 of the Norming Ads During the Semester, by

Semester and Year

15

Are Respondent Reports of Seeing Ads Reliable?

22.826.4

42.3

51.3 52.3

34.2

41.2

3.3 3.4 3.30

10

20

30

40

50

60

Ad 1 Ad 2 Ad 3 Ad 4 Ad 5 Ad 6 Ad 7 FakeAd 1

FakeAd 2

FakeAd 3

PERCENT

Source: Spring Celebration Survey, 2005

% of Respondents Who Claimed to Have Seen Each Ad/Poster


Percent

Students’ Latitude of Acceptance/Rejection of Ad Claims

Ave. Number Drinks R’s Believe Typical Student Drank

“For each percentage, please indicate if you find the statement to be very believable, somewhat believable, or not believable at all.”

"The percentage of MSU students that typically drink 4 or fewer drinks when they party is ___% "

Source: Spring Celebration Survey, 2005, n=1,073

Actual % of Respondents who drank 0-4 Last Time

17

Did Change Occur in Normative Perceptions?

Average Number of Drinks the Typical Student Believed to Have Consumed the Last Time Partied/Socialized

Source: MSU NCHA 2000-2006

}A 19%

Decrease

}

A 6.8% Decrease

5.485.63

4.644.50

6.24 6.36

5.93 5.87

2000 2002 2004 20063.00

4.00

5.00

6.00

7.00

Undergrads Grads

Did Change Occur in Normative Perceptions?

18

Percent

% of Respondents Who Believe the Typical Student Drank 0-4 Last Time Partied, by Gender and Student Status

Source: MSU NCHA 2000 - 2006


Did Change Occur in Use of Protective Behaviors?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

2000 2002 2004 2006YEAR

% W

ho A

lway

s or

Usu

ally

a. Alternate drinks

b. Plan not to exceedlimitc. Choose not to drink*

d. Have designateddriver*e. Eat before or during*

f. Have friend saywhen had enough*g. Keep track of drinks

h. Pace drinking

i. Drink look alikes

% of Undergrads Who Always or Usually Use Various Protective Behaviors, by Year

20

Change in Use of Protective Behaviors

30.0

35.0

40.0

45.0

50.0

55.0

60.0

65.0

70.0

2000 2002 2004 2006

Did none

Did 1 or more

Among those who drank 5+ drinks last time partied


22.624.4

20.222.1

11.7

17.4 17.616.1

65.7

58.3

62.2 61.8

10

15

20

25

30

35

40

45

50

55

60

65

70

2000 2002 2004 2006

Never/Not in past 30 1-2 days 3+ days

Did Change Occur in Drinking Behavior?

PercentOf All

Students

Number of Days in Previous Month Drank Alcohol

5.9% decrease from 2000 in percentage of students drinking 3 or more days per month

22

Did Change Occur in Drinking Behavior?

5.42

4.98 4.90 4.97

5.63

5.38 5.32 5.35

4.64

3.313.17 3.093.00

3.50

4.00

4.50

5.00

5.50

6.00

2000 2002 2004 2006

Year

Mea

n #

Dri

nks

All Students

Undergrads

Grad & Other

Mean Number of Drinks Last Time Partied/Socialized, by Year

23

Is Drinking Related to the Likelihood of Harm?

• Physical/Psychological– Injury to self as result of alcohol– Injury to other as result of alcohol– Involved in fights– Did something later regretted – Forgot where you were/what did – Had unprotected sex– Had forced sex

• Academic

– Lower grade on exam or project– Lower grade in course– Incomplete or dropped course

Types of Harm Asked About in NCHA

24

Is Drinking Related to the Likelihood of Physical/Psychological Harm?

Risk of At Least One Undesirable Event by Number Drinks Consumed Last Time Partied

0%10%20%30%40%50%60%70%80%90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Number Drinks Last Time Partied

% H

ad A

t Le

ast

One

Und

esira

ble

Eve

nt in

Las

t

Sch

ool Y

ear

25

% of Students Experiencing Harm by Drinking Levels

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

HeavyModerateLight

Odds Heavy vs. Mod 2.4 4.0 2.8 1.7 1.9 1.5 2.2 1.6

Odds Mod vs. Light 18.4 43.2 33.7 7.9 12.4 12.5 5.7 7.4

Odds Heavy vs. Light 43.4 171.0 94.8 13.2 24.1 18.5 12.6 11.8

Heavy = Drink 10 or more days per month and 5+ drinks last time partiedLight = Drink 2 or fewer days per month and 2 or fewer drinks last time partiedModerate = Everything else

Source: MSU NCHA 2000, 2002, 2004, 2006; n = 4,155


Did Change Occur in Amount of Harm?

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

50.0%

55.0%

2000 2002 2004 2006

Year

% o

f R

esp

ond

ents

Did something later regretted*

Forgot where you were/what did

Injury to self as result of alcohol

Had unprotected sex*

Involved in fights

Injury to other as result ofalcoholForced sex*

Source: MSU NCHA 2000-2006

Among those who drink This school year, as a consequence of your drinking, have you experienced. . .

8.5% reduction from 2000 to 2006

34.8% reduction from 2000 to 200617.0% reduction from 2000

to 200678.3% reduction from 2000 to 2006

27

Change in Academic Harm, by Year

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

2000 2002 2004 2006

YEAR

Per

cen

t

Alcohol effectedAcademicPerformance

Lower gradeexam or project

Lower grade incourse

Incomplete orDropped Course

28

Summary

Evidence of initial misperception of drinking norm Evidence norming campaign reached targeted

audience Evidence perception of norm changed Evidence use of protective behaviors changed Evidence drinking behaviors changed Evidence for reduction of drinking related harm

• Program is working

29

Is There Reason to Believe This Isn’t Just Coincidence?

• Dejong et al., 2006. “A multisite randomized trial of social norms marketing campaigns to reduce college student drinking” Journal of Alcohol Studies 67: 868-879.

• From 2000 to 2003, found experimental group of colleges implementing social norming campaign had no or very small increases in perceptions of drinking norms, actual consumption and harm, while control group of colleges and universities not implementing social norming campaigns had substantial increases in all three.

• The magnitude of the effect at experimental sites correlated positively with the intensity of the norming campaign.

• The secular trend in the time period was for drinking and harm to increase.

• Evidence MSU’s results are not just coincidence because of a secular trend.

Respondents’ Assessments of Social Norming Ads Distributed in 2003-05

Ad % Who

Saw

Mean #

Times

Saw

% Saw as

Believable

% Saw as New

Information

Tailgating ad 40.7% 2.7 42.4% 60.5%

Designated driver ad 36.6% 2.9 73.9% 41.8%

Basketball playing ad 27.3% 2.6 52.9% 68.7%

Pool playing ad (Fall) 36.9% 2.6 45.3% 56.0%

Moderation ad (Fall) 41.2% 2.9 59.6% 58.1%

Pool playing ad (Spring) 61.8% 3.1 39.3% 54.4%

Moderation ad (Spring) 52.3% 3.1 54.6% 54.3%

Spring Break: snow 18.8% 2.3 59.4% 41.1%

Spring Break: luggage 14.8% 2.3 62.8% 41.0%

MSU basketball 26.3% 2.4 55.9% 44.2%


Ad/Poster % Who

Saw

Mean #

Times

Saw

% Saw as

Believable

% Saw as New

Information

Sidelines ad (football) 31.8% 2.6 61.1% 50.0%

Spartans Think (fall) 48.0% 2.5 60.2% 63.3%

Spartans Do (fall) 50.5% 2.5 68.7% 68.4%

Spartans Do (spring) 53.8% 2.8 76.0% 61.4%

Spartans Think (spring) 52.0% 2.5 59.7% 56.7%

St. Patrick’s ad (spring) 46.4% 3.0 62.3% 55.2%

Courtside ad (spring) 32.6% 2.6 68.8% 43.4%

Courtside ad (Izzone) 44.9% 2.1 57.4% 40.3%

Halloween ad 25.4% 2.0 68.5% 39.9%

Halloween Poster 29.3% 3.5 73.6% 52.4%


Ad/Poster % Who

Saw

Mean #

Times

Saw

% Saw as

Believable

% Saw as New

Information

Global ad (Spring) 25.5% 2.5 77.7% 29.0%

Courtside ad 18.0% 2.1 78.2% 37.8%

Izzo ad 38.5% 2.8 79.6% 51.8%

Izzo Poster 38.1% 3.1 79.0% 47.o%

Spring Break Ad 42.6% 2.5 72.0% 52.0%

Spring Break Table tent 48.2% ** 73.4% 56.9%

Global Scrambled (Fall) 43.1% 2.9 68.8% 44.3%

Spartan Do TT 59.6% ** 60.7% 70.8%

Halloween TT 44.4% ** 72.3% 47.9%

Halloween ad 48.3% 2.1 79.2% 50.5%

*Ad Believability and Over and Under Estimation of Drinking Norms

This study looked at estimation and accuracy of normative perceptions for students during

both everyday drinking and celebration drinking.

Believability and Estimation

• It found that students who drank less than four

drinks underestimated the norm, and those who drank more than five drinks overestimated the norm.

• Ad believability played a crucial role in this

process. Those who believed the ad more closely estimated alcohol consumption by their peers.

Effectiveness

• The reduction of specific primary harms (NCHA and Celebratory Survey data)

• The increased adoption of protective behaviors across the target population

• The level of acceptance the campaign has received from students, including levels of message believability and message usefulness

Ethnographic and Formative Research Findings

“Celebratory Occasions”

U.S. Department of Education Grant 2001-2003

DRINKING AND DRUNKENNESS

ON CELEBRATORY OCCASIONS

Holiday DRINK DRUNK

Occasion: PREVAL RATE

Halloween 32% (57%)

St. Patrick’s 26% (58%)

Typical Thursday 19% (48%)

DRINK PREVAL = Percent of all students that report drinking

DRUNK RATE = Proportion of drinkers who report getting drunk



Football DRINK DRUNK


Big Gameday 38% (56%)

Other Games 37% (50%)

Typical Saturday 23% (39%)

DRINK PREVAL = Percent of all students that report drinking

DRUNK RATE = Proportion of drinkers who report getting drunk



Weeklong DRINK DRUNK


Spring Break 48% (62%)

Welcome Week 37% (72%)

End of Semester 23% (55%)

Typical Week 47% (48%)

Figure 4. Percentage Drinking Less or More than Planned

0

5

10

15

20

25

30

Events

Per

cent

age

Less

More

Non-Drinking Norm Prevalence:

76% End of Semester

74% St. Patrick’s

68% Halloween

62% Big Game

60% Welcome Week

52% Spring Break

Non-Drunkenness Norm Prevalence:

(Moderately or not at all)

86% End of Semester

85% St. Patrick’s

82% Halloween

79% Big Game

71% Welcome Week

70% Spring Break

Types of Drinkers:

“Anytime” Drinkers -- 54%

Drink on typical days and special days

“Celebration” Drinkers -- 35%

Drink on special days but not typical days

Types of Drinkers:

“Non-Celebration” Drinkers -- 2%

Drink on typical days but not special days

Seldom Drinkers -- 9%

Drinkers who didn’t drink on the special or

typical days measured in survey

Table 5. Percentage Distribution of Drinking Types, by Demographic Characteristics

n Anytime Celebration Seldom Non-Celebration ……

Gender Female Male

514

431

49.8%

58.9%

35.0%

34.1%

12.5%

4.9%

2.7%

2.1%

19.03

p<.001

Status Freshman Sophomore Junior Senior

180

212

242

313

47.2%

54.7%

55.0%

56.5%

39.4%

35.8%

33.1%

31.9%

11.7%

6.6%

8.7%

9.6%

1.7%

2.8%

3.3%

1.9%

8.66

N.S.

Race White African American Other

793

61

91

56.6%

32.8%

45.1%

33.7%

37.7%

40.7%

7.9%

26.2%

6.6%

1.8%

3.3%

7.7%

42.45

p<.001

Drank in YesH.S. No

666280

57.8%44.6%

34.1%36.1%

6.0%16.4%

2.1%2.9%

30.6p<.001

“Focused” Findings

• Identified Protective Behaviors showed a reduction in harm independent of consumption

• Normative climate seemed to approve of drinking and higher than normal consumption on celebratory occasions

• Individuals identified as ‘celebratory’/event specific drinkers!

Protective Behaviors:

Promising protective behaviors not necessarily related to consumption: (Celebration ‘02)

– Staying with same group of friends-

– Remaining in one location-

– Consuming only one type of alcohol.

Findings from the Social Norming

Study of Student Alcohol Use2003-05

Student approval of activities

Respondents were asked about their approval of various activities, and also their estimates as to whether ‘most MSU students’ approve the same behavior.

• The following slides show the results as a function of level of student drinking

Perception of various celebrations: Celebration Survey data

0%

10%

20%

30%

40%

50%

60%

70%

80%

WelcomeWeek

Tailgate Halloween St. Patrick'sDay

SpringBreak

EstimateActual

Approval & estimates of celebration drinking

76%69%

24%

68%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Estimate: MSU Celebrationdrinking

Yes

No

Having more than 5 drinks

-100%-80%-60%-40%-20%

0%20%40%60%80%

100%

OVERALL LEVEL L LEVEL M LEVEL H

Dis

appr

ove

App

rove

I approve Most approve

Having more than one drink/hour

-100%-80%-60%-40%-20%

0%20%40%60%80%

100%


Dis

appr

ove

App

rove


Getting wasted

-100%-80%-60%-40%-20%

0%20%40%60%80%

100%


Dis

appr

ove

App

rove

I Most

Celebration drinking

-100%-80%-60%-40%-20%

0%20%40%60%80%

100%


Dis

appr

ove

App

rove


Results of Note: Injunctive Norms

• 77% of MSU students approve of drinking more than usual during celebration events.

• 74% of MSU students approve of getting drunk during welcome week.

• 73% of MSU students report they approve of getting drunk during Halloween.

Celebrations Surveys 2003-2006

Dissemination Strategies for the Model Grant

• Develop and evaluate a website that will provide interactive instructions for researchers and practitioners wanting to replicate our program

• Submit content to Wikipedia.org on social norms• Create and evaluate a handbook on the

development and operationalization of a social norms marketing campaign

• Provide service as consultants to other institutions of higher education via hosting several teleconferences/webinars

• Present our model at workshops and conferences

Enhancements

• Enhancements to the program include:– the development of messages that shift

toward subjective norm use distributed through social-interactive community sites (Facebook)

– the use of the Audience Response System—(ARS) or "clicker technology" to enhance live presentations

1 msu model grant overview dennis martell sandi smith michigan state university

Documents

social norming campaign

actual drinking norm

actual norm

perceived norm

model program

actual behavior

actual average

enhancement strategies