when users pledge to take green actions, are they solving a decision problem? michael johnson...

When Users Pledge to Take Green Actions, Are They Solving a Decision Problem?

Michael Johnson

University of Massachusetts Boston

Boston MA 02125

[email protected]

Susan Fussell ([email protected])

Jennifer Mankoff ([email protected]) Deanna Matthews ([email protected])

Leslie Setlock ([email protected])

Carnegie Mellon University

Pittsburgh PA 15213

INFORMS Fall Conference, Washington, D.C.October 15, 2008

Wednesday, October 15, 2008

StepGreen: Decisionmaking 2

Acknowledgements

Funding: Intel Corporation, “Leveraging Computational Technologies to

Support Behavior Change”, Jennifer Mankoff, principal investigator

National Science Foundation grants NSF IIS-0745885 and NSF IIS-0205644, Jennifer Mankoff, principal investigator

Research assistance: Victoria Yew



Problem Motivation Excessive energy consumption is a primary cause of

global warming Americans consumed 100 quadrillion BTUs of energy (U.S.

Department of Energy 2006) Energy consumption primarily linked to individual activities:

lighting, heating and cooling (Bin and Dowlatabadi 2005, U.S. Environmental Protection Agency 2006)

Numerous Web-based initiatives exist to encourage environmentally responsive behavior Daily actions: GreenSpeak.org Green travel: EarthRoutes.net Carbon footprint calculations and green living: Yahoo!Green

Social networking sites enable individuals to meet, collaborate and act collectively Facebook MySpace



Research questions Can we design an application to encourage

reductions in daily activities that affect global warming that integrate: On-line social networking, to attract people who are not

necessarily ‘green’ Displaying progress towards energy reduction goals, individually

and in groups Tracking actual energy consumption through sensors Multiple technology platforms

Result is StepGreen, an initiative that combines scholarship, outreach and social change: Mankoff, Matthews, Fussell and Johnson (2007) Mankoff et al. (2008)



StepGreen: A Web-based system in support individual action to combat global warmingSocial network site ‘badge’ StepGreen.com actions for commitment

Cumulative effects of actions taken

Source: www.stepgreen.org



Alternative disciplinary views of StepGreen Technology: a flexible, robust system will induce

behavioral change and attract many users IS/IT policy: a popular on-line application will provide

novel insight into IT adoption, usage and outcomes. Decision sciences: a Web-based decision support

system will help ordinary users make better and more efficient decisions about high-impact daily actions

Public policy: learn about long-term social and environmental impacts of individual change as compared to national-level policy.



Action selection is a decision problem Behavioral change is a multi-step process:

Persuade users that action on a particular topic is urgent Learn consequences of actions along multiple dimensions Explore alternatives by examining tradeoffs in attribute space Choose one or more actions that optimize utility Observe actual impacts of actions Update preferences for action attributes



Action selection is a decision problem, cont’d Action representation is important to

decisionmaking: Present choices in lists of varying sizes

One at a time List

Present choices in differing ways: Text descriptions Tables, charts and graphs

Dynamics in action representations: Static Interactive

What is the effect of length and information content of alternative visual representations of actions on decision

time and decision quality?



Theoretical Foundations Human-computer interactions (Dix, et al. 2004) Role of visual representation in decision-making

Lurie and Mason (2007) Miller (2004) Mandel and Johnson (2002)

Decision aids for large/complex decision problems Payne et al. (1988) Eiselt and Sandblom (2004)

Decision support systems for consumer choice Häubl and Trifts (2000) van der Heijden (2006)

Decision-making styles and barriers Bruine de Bruin, Parker and Fischoff (2007) Scott and Bruce (1995)



Research gap: decision aids and DSS for public-sector problems

Limited literature on decision support and visualization especially by unsophisticated users, or those in vulnerable or underrepresented groups (Johnson 2006)

Can specific decision aids and visualization strategies enable users to make decisions regarding lifestyle choices more effectively, or with higher levels of

satisfaction?



Previous experiment: design Surveys:

Effects of human actions on global warming Decision-making styles Satisfaction with range of choices provided and rankings made

Evaluate text-based actions according to: Length (‘terse’ vs. ‘verbose’) Information content (‘relevant’ vs. ‘irrelevant’) Action category (e.g. Heating, Lighting, Appliances, Water)

Actions are partitioned into two sets of non-dominated alternatives: ‘Superior’ (8 or 10) ‘Inferior’ (2 of 10)

Users rank top four actions out of 10 available in four categories



Previous experiment: text representationWater Consumption



Previous experiment: results

Decision quality shows significant variation with action categories

Few statistically significant associations with decision length or quality: Question length Information content Decision style Satisfaction with the range of decision alternatives Satisfaction with the ranking decisions

No evidence of learning about environmental impacts



Will alternative visualization methods make a difference? Hypothesis: users prefer graphical representations

of actions to text representations and will make better decisions.

New design: Four action categories Four visual representations of action characteristics and impacts:

‘Terse’/’relevant’ text Symbols Value path Bar chart



Experiment data: classifications, values Actions and categories:

Actions came from literature review and common sense Categories inspired by ‘card-sorting’ exercises

Appliances Heating/Cooling Lighting and Appliances Water Consumption

Impacts: Carbon emissions

Change in energy usage: Energy Star (http://www.energystar.gov/) Estimates of carbon savings: Energy Information Administration (

http://www.eia.doe.gov/emeu/aer/txt/ptb1207b.html) Dollar costs/savings: Energy Information Administration (

http://www.eia.doe.gov/emeu/aer/txt/ptb0810.html) Time costs/savings: rules of thumb Quality of life: subjective assessments



Actions: Text (Appliances)



Actions: Symbols

Action

Emissions impact [Pounds of CO2

per year]Financial impact

[$/ year]Time [minutes per

year]

Quality of Life Impact [most negative -> most

positive]

Let computer sleep when idle

Purchase Energy Star refrigerator

Take a hike outside

Air-dry your clothes

Turn off printers

Turn off the TV

Take the stairs

Turn off ceiling fan

Unplug DVD players

Assess appliance efficiency



Actions: Value pathAction Impacts

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Emissionsimpact [pounds

of CO2 peryear]

Financial impact[$ per year]

Time [minutesper year]

Quality of LifeImpact [mostnegative to

most positive]

Attribute Categories

% of Maximum Possible Value

Achieved

Let computer sleepwhen idlePurchase Energy StarrefrigeratorTake a hike outside


Turn off printers

Turn off the TV

Take the stairs


Unplug DVD players

Assess applianceefficiency

`



Actions: Bar chart

Action Impacts

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Assess appliance efficiency

Unplug DVD players


Take the stairs

Turn off the TV

Turn off printers


Take a hike outside

Purchase Energy Star refrigerator

Let computer sleep when idle


% of Maximum PossibleValue Achieved

Emissions impact[pounds of CO2 peryear]

Financial impact [$per year]

Time [minutes peryear]

Quality of Life Impact[most negative tomost positive]



Research framework, cont’d Users are assumed to act according to defined

decision-making styles (Scott and Bruce 1995): Intuitive

“When I make a decision I trust my inner feelings and reactions” Rational

“I make decisions in a logical and systematic way” Dependent

“I often need the assistance of other people when making important decisions”

Avoidant “I often procrastinate when it comes to making important decisions”

Spontaneous “When making decisions I do what seems natural at the moment”

Users learn about environmental impacts of various actions through choice process



Hypotheses

H 1: Participants’ decision quality and decision time vary according to action category.

H 2a: Graphical representation results in better outcomes than text representations

H2b: Decision time and decision quality varies according to specific graphical representations

H 3a: ‘Rational’ decision-making styles are associated with higher-quality decisions

H 3b: ‘Spontaneous’ decision-making styles are associated with lower-quality decisions

H 3c: ‘Rational’ decision-making styles are associated with slower decisions

H 3d: ‘’Intuitive’ decision-making styles are associated with more rapid decisions

H 4: Participants showed an increase in knowledge about impacts of specific actions with respect to global warming

H 5: Gender is associated with decision quality and decision time



Experiment design Within-subjects design – four conditions (Martin

2004) Survey software automatically randomized

presentation orders and recorded decision times Action categories, graphical representations

counterbalanced across participants 32 undergraduate and graduate student participants Steps:

Study overview and consent forms Pretest survey Action choices Posttest surveys Compensation



Results: Descriptive Statistics Participant characteristics:

72% male 91% students (60% undergraduates; 31% graduate students) 84.4% between 18 – 25 years old Most live in households with unrelated mates and no children Diverse racial, ethnic backgrounds

Decision-making styles (means of 1 – to – 5 scaled question responses within categories): Rational: 3.82 Intuitive: 3.59 Dependent: 3.26 Avoidant: 2.68 Spontaneous: 2.87



Action Impact on Global Warming Mean Pre-Test Value*†Clearing Rainforests Medium 3.62 (0.253)Toxic Wastes Low 3.41 (2.53)The Space Program Low 2.44 (0.229)Ozone in Cities Low 3.81 (0.217)Deforestation Medium 4.00 (0.225)Ocean Dumping Low 2.94 (0.233)The Hole in the Ozone Layer Low 4.00 (0.225)Aerosol Spray Cans Low 3.28 (0.221)Burning Fossil Fuels High 4.06 (0.190)Industrial Emissions High 3.97 (0.198)Household Energy Use High 3.19 (0.208)Household Lighting Medium 3.03 (0.218)Household Appliances Medium 2.94 (0.210)Household Heating and Cooling High 3.37 (0.205)Household Water Consumption Low 2.72 (0.212)* Scale: 1 = “Not very much”…5 = “Very much”†Standard error in parenthesesN = 32

Results: Descriptive Statistics, cont’d

Baseline global warming knowledge is generally low



Results: Outcome measures

Dominated responses = number of choices that were dominated in the subject’s responses = 0, 1, 2

Response time = time to make all selections; used log-transformed times due to skewed original values



Results: Effect of action categoryMixed model analyses showed no difference between

domains (appliances, heating, lighting, water) for: Number of dominated choices (F [3, 36.3] = 1.71, p = .17) or

Log response time (F [3, 58.93] = 1.13, p = .35)

No effect on decision quality or time due to action category



Result: Effect of representation type Collapsed outcomes over action categories, giving each

participant one score for each visualization condition Dominated choices:

Log response time:

Condition

Mean Difference (Std. Error) Significance




Text .087 (0.165) 0.604 .130 (0.192) 0.503 -.043 (0.160) 0.788Symbols .043 (0.172) 0.803 -.130 (0.170) 0.451Value Path -.174 (0.162) 0.295

Bar ChartValue PathSymbolsText

Condition





Text .164* (0.048) 0.002 .123 (0.061) 0.057 .121* (0.052) 0.029Symbols -.041 (0.047) 0.391 -.044 (0.059) 0.471Value Path -.002 (0.069) 0.975

Bar ChartValue PathSymbolsText

Representation type has no effect on decision qualityResponse times longer for text than for graphics, and

response times do not differ by graphic type



Results: Pre- and post-test learningAction Impact on Global

WarmingMean Pre-Test

Value*†Mean Post-Test

Value*†Mean Difference Post-Pre-Test*†

Clearing Rainforests Medium 3.62 (0.253) 3.84 (0.246) .219 (0.147)Toxic Wastes Low 3.41 (2.53) 3.50 (0.250) .094 (0.176)The Space Program Low 2.44 (0.229) 2.63 (0.257) .188 (0.145)Ozone in Cities Low 3.81 (0.217) 4.00 (0.211) .188 (0.114)Deforestation Medium 4.00 (0.225) 4.09 (0.226) .094 (0.145)Ocean Dumping Low 2.94 (0.233) 3.13 (0.219) .188 (0.193)The Hole in the Ozone Layer Low 4.00 (0.225) 3.94 (0.220) -.063 (0.1.00)Aerosol Spray Cans Low 3.28 (0.221) 3.44 (0.237) .156 (0.143)Burning Fossil Fuels High 4.06 (0.190) 4.22 (0.199) .156 (0.156)Industrial Emissions High 3.97 (0.198) 4.19 (0.193) .219 (0.098)Household Energy Use High 3.19 (0.208) 3.75 (0.206) .563 (0.174)Household Lighting Medium 3.03 (0.218) 3.38 (0.200) .344 (0.183)Household Appliances Medium 2.94 (0.210) 3.41 (0.224) .469 (0.190)Household Heating and Cooling High 3.37 (0.205) 3.81 (0.182) .438 (0.190)Household Water Consumption Low 2.72 (0.212) 3.06 (0.250) .344 (0.204)* Scale: 1 = “Not very much”…5 = “Very much”†Standard error in parenthesesN = 32

After experiment, users generally perceived greater global warming impact on all actions



Correlations: user characteristics and decision outcomes Mean number of dominated choices per trial

negatively correlated with log of response time ( -0.405 [0.021])

Gender (male = 0, women = 1) is negatively correlated with log of response time (-0.439 [0.012])

Decision-making style has no statistically significant effect on decision-making quality or decision time

Gender not statistically significantly correlated with decision style or decision quality



Correlations: graphical representations and outcomes Some relationships between decision times across

graphical representations: Text and value path (.470 [0.008])

Text and bar chart (.513 [0.010])

Symbols and value path (.603 [0.000])

Symbols and bar chart (.403 [0.051])



So, are StepGreen users solving a decision problem? We think so..but decision context so far provides

little support. For static representations of actions:

Generally, neither length of action descriptions, information content within descriptions or graphical representation of actions had significant effects on decision outcomes.

Why did more information about decision alternatives not help subjects make better decisions? Alternatives ranking too demanding cognitively? Information insufficiently tailored to different needs, interests and

backgrounds of subjects?



Implications for research Design:

For maximum speed, emphasize graphical representations of actions

Particular graphical representation not important

Next steps: Use a human intermediary to help users choose actions Apply new methods to measure decision-making competence



New Experiment: Human-assisted decisionmaking Inspiration:

Risk-perception literature (Florig, et al. 2001): detailed problem representation improves risk ranking

Decision competence literature (Parker, Bruine de Bruin and Fischoff 2007), in which measures of decision efficacy are associated with satisfaction with decisions made

Idea: Use a human intermediary to help users better understand their

own values and preferences and characteristics of action alternatives



Goal: evaluate decision outcomes along two dimensions of interventions Intermediary types:

‘Peer’ intermediary - informal, youthful affect and use a minimum of technical language

‘Expert’ intermediary - more formal, academic affect, use technical language and appear to be an authority on behavioral changes and impacts of actions on climate change.

Intermediation type: Quantitative - Problem-focused, scientific presentation of the

impacts of various actions using figures and descriptions of relevant calculations

Qualitative - individual-focused, interactive, holistic discussion using probing questions to learn about subject attitudes regarding different actions



Scientific versus informal presentation of actions Scientific presentation:

Diagrams will convey the mechanisms by which actions will result in energy savings and a reduction in carbon emissions.

Equations will convey the means by which energy savings and reductions in carbon emissions are computed for ‘typical’ users.

No mention will be made of individual preferences for some classes of actions over others, or the means by which individual lifestyle characteristics influence the impacts of various actions.



Scientific versus informal presentation of actions, continued Informal presentation:

Scripted questions to subjects will determine Categories of actions are most important to them

Constraints that limit consideration of certain action

Motivation for pursuing energy reducing actions

No mention will be made of amounts of energy saved for various actions, or the means by which impacts are computed.



Proposed data analysis

Descriptive statistics: Demographics Measures of decision-making styles Decision-making competency Decision-making outcomes

Hypothesis tests: Impact on decision-making outcomes of

Intermediary type - intermediation type pairs Decision-making styles Decision-making competency



See you next year!

Questions?



References Bin, S. and H. Dowlatabadi. 2005. Consumer Lifestyle Approach to US Energy Use and

the Related CO2 Emissions. Energy Policy 33: 197 – 208. Bruine de Bruin, W., Parker, A.M. and B. Fischhoff. 2007. Individual Differences in Adult

Decision-Making Competence. Journal of Personality and Social Psychology 92(5): 938 – 956.

Häubl, G. and V. Trifts. 2000. Consumer Decision Making in Online Shopping Environments: The Effects of Interactive Decision Aids. Marketing Science 19(1): 4 – 21.

Lurie, N.H. and C.H. Mason. 2007. Visual Representation: Implications for Decision Making. Journal of Marketing 71: 160 – 177.

Mankoff, J., Fussell, S.R., Johnson, M.P., Matthews, D., Blais, D., Dillahunt, T., Glaves, R., McGuire, R., Setlock, L., Schick, A. Thompson, R. and H.-C. Wang. 2008. “StepGreen: Engaging Individuals in Energy-Saving Actions Online.” Under review for presentation at Computer/Human Interaction Conference 2009, Boston, MA.

Mankoff, J., Matthews, D., Fussell, S.R. and M. Johnson. 2007. “Leveraging Social Networks to Motivate Individuals to Reduce Their Ecological Footprints”, in Proceedings of the 40th Annual Hawaii International Conference on System Sciences (CD-ROM), January 3 – 6, 2007, Computer Society Press, 2007 (10 pages)

Scott, S.G. and R.A. Bruce. 1995. Decision Making Style: The Development and Assessment of a New Measure. Educational and Psychological Measurement 55: 818 – 31.

U.S. Department of Energy. 2006. Annual Energy Review 2005. Washington, D.C.: Energy Information Administration, DOE/EIA-0384. World Wide Web: http://tonto.eia.doe.gov/FTPROOT/multifuel/038405.pdf.

van der Heijden, H. 2006. Mobile Decision Support for In-Store Purchase Decisions. Decision Support Systems 42: 656 – 663.



Previous experiment: choice sets

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Emissions impact[Pounds of co2 per year]

Financial impact [$/year]

Time [minutes per year] Quality of Life Impact[most negative -> most

positive]


% A

ch

iev

ed

Plant a shade tree

Install a programmable thermostat

Purchase energy saving windows

Adjust thermostat each day when out

Ajust set temperature of thermostat by 2degreesAdd a layer of clothing

Close off vents in rarely used rooms

Avoid using the oven

Turn off window air conditioners

Check furnace/air conditioning air filters

Heating/Cooling



Correlations: User characteristics and decision outcomesMean Number of

Dominated Choices per Trial

Log of Average Decision Time

Mean of 4 Rational decision making questions

Mean of 5 Intuitive decision making questions

Mean of 5 Dependent

decision making questions

Mean of 5 Avoidant

decision making questions

Mean of 5 Spontaneous

decision making questions Gender

Mean Number of Non-Dominated Solutions: Text

Mean Response

Time: Text

Mean Number of Dominated Choices per Trial -0.405* (0.021) -0.114 (0.534) -.030 (0.869) -0.044 (0.812) -0.225 (-.225) -0.068 (0.713) -0.005 (0.977) .436* (0.014) -0.337 (0.064)Log of Average Decision Time 0.257 (0.156) 0.033 (0.857) 0.14 (0.444) -0.127 (0.489) -0.323 (0.072) -0.439* (0.012) -0.028 (0.883) .828** (0.000)

Mean of 4 Rational decision making questions -.363* (0.041) 0.033 (0.860) -.390* (0.027) -0.117 (0.523) -.341 (0.056) .069 (0.711) .221 (0.233)Mean of 5 Intuitive decision making questions .133 (0.468) .187 (0.304) .440* (0.012) -.039 (0.831) .239 (0.196) .034 (0.855)Mean of 5 Dependent decision making questions .406* (0.021) .002 (0.993) -.204 (0.264) -.178 (0.337) .247 (0.180)Mean of 5 Avoidant decision making questions .411* (0.020) .128 (0.485) -.189 (0.309) -.036 (0.849)Mean of 5 Spontaneous decision making questions .020 (0.915) .108 (0.565) -.256 (0.164)Gender -.345 (0.058) -.310 (0.090)Mean Number of Non-Dominated Solutions: Text -.170 (0.360)†t-statistics in parentheses*Significant at 5% level**Significant at 1% level



Correlations: Action representations and decision outcomesMean Number

of Dominated Choices per

Trial

Average Decision

Time

Log of Average

Decision Time

Mean Number of Dominated Choices per

TrialAverage

Decision Time

Log of Average

Decision Time


TrialAverage

Decision TimeLog of Average Decision Time


Trial

Average Decision

Time

Log of Average

Decision Time

Mean Number of Dominated Choices per Trial -.254 (0.168) -.170 (0.360) .070 (0.750) .312 (0.087) .345 (0.058) -.254 (0.167) -.113 (0.545) -.088 (0.639) -.037 (0.842) .006 (0.979) -.093 (0.666)Average Decision Time .962** (0.000) -.011 (0.960) .255 (0.166) .246 (0.182) -.207 (0.264) .429* (0.016) .385* (0.033) -.153 (0.410) .364 (0.080) .465* (0.022)Log of Average Decision Time .012 (0.956) .334 (0.067) .326 (0.074) -.322 (0.078) .486** (0.006) .470** (0.008) -.184 (0.322) .424* (0.039) .513* (0.010)Mean Number of Dominated Choices per Trial -.407* (0.048) -.455* (0.026) -.086 (0.690) -.027 (0.900) .009 (0.965) .016 (0.941) -.395 (0.130) -.518* (0.040)Average Decision Time .972** (0.000) -.132 (0.472) .597** (0.000) .591** (0.000) -.148 (0.418) .490* (0.015) .486* (0.016)Log of Average Decision Time -.145 (0.429) .597** (0.000) .603** (0.000) -.271 (0.133) .425* (0.038) .403 (0.051)Mean Number of Dominated Choices per Trial -.271 (0.134) -.300 (0.096) .000 (1.00) -.148 (0.490) -.141 (0.512)Average Decision Time .957** (0.000) -.306 (0.089) .388 (0.061) .359 (0.085)Log of Average Decision Time -.293 (0.103) .356 (0.088) .334 (0.111)Mean Number of Dominated Choices per Trial -.100 (0.640) .051 (0.815)Average Decision Time .947** (0.000)Log of Average Decision Time

†t-statistics in parentheses*Significant at 5% level**Significant at 1% levelN = 32

Text Symbols Value Path Bar Chart

Text

Symbols

Value Path

Bar Chart

when users pledge to take green actions, are they solving a decision problem? michael johnson...

Documents

green actions

social change

jennifer mankoff

department of energy

behavior change

decision sciences

webbased system

support individual action