assessing sustainability literacy

Assessing Sustainability LiteracyAn Instrument to Measure Knowledge of Environmental, Economic, and Social Sustainability

Adam ZwickleTomas KoontzKristina Slagle

Overview Need for a sustainability literacy

measure

Developing an assessment

Results

Where we go from here

Need for a sustainability literacy measure

STARS ER Credit 13: Sustainability Literacy Assessment

“This credit recognizes institutions that are assessing the sustainability literacy of their students. Such an assessment helps institutions evaluate the success of their sustainability education initiatives and develop insight into how these initiatives could be improved.”

Criteria“Institution conducts an assessment of the sustainability literacy of its students. The sustainability literacy assessment focuses on knowledge of sustainability topics, not values or beliefs.”

Need for a sustainability literacy measure Lacking in the scientific community

“Sustainability” has been defined and measured a myriad of ways Often focused merely on the environmental aspect

Immensely applicable Can test for correlations with various values,

attitudes, and behaviors Can serve as an explanatory variable

Developing an assessment Built upon the “triple bottom line”, the

“three legged stool”, the “3 p’s” Environmental (planet) Economic (prosperity) Social (people)

Basic knowledge from each knowledge domain

Developing an assessment Replicated questions used in the past

Coyle, 2005. “Environmental Literacy in America.”

Solicited topics and questions from experts

Held expert focus groups Pilot tested among professors, graduate,

and undergraduate students Narrowed down to 30 questions

Developing an assessment Administered 30 question assessment

Through an online survey Sent via email

Sent to over 10,000 currently enrolled undergraduates 19.3% response rate

Non-response follow up

Developing an assessment Demographics

OSU – average demographics This study - Slight trend towards more

liberal and environmentalist, but differences were non-significant

Non-respondants More likely to get a subset of the

sustainability knowledge questions wrong…(small difference, but significant)

No differences in pro-environmental behaviors or GPA

Developing an assessment Therefore, those choosing to take the

survey were more knowledgeable about sustainability issues to begin with

These findings represent an ‘upper bound’ of the student population’s knowledge

Developing an assessment Needed to reduce the length

Utilized Item Response Theory (IRT) To reduce down to 16 questions

And aside on IRT… Developed in the educational testing

field Used for standardized tests (GRE)

Focuses on the likelihood of answering something correctly… At a given level of the construct of

interest

And aside on IRT…

Information

Correct

Incorrect

50/50threshold

Standard deviations

Theta - The latent construct that is being measured

Probability of answering question correctly

And aside on IRT…

Information

Correct

Incorrect

50/50threshold

“Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans?”

And aside on IRT…

Information

Incorrect

Correct

50/50threshold

Standard deviations

A question that was removed…

Probability of answering question correctly

And aside on IRT…

Very low information

Correct

Incorrect

50/50threshold

“Approximately how much of the oil used in the U.S. in 2010 was imported from other countries?”

Questions - Environmental What is the most common cause of pollution of

streams and rivers? Ozone forms a protective layer in the earth's upper

atmosphere. What does ozone protect us from? What is the name of the primary federal agency

that oversees environmental regulation? What is the primary benefit of wetlands? Which of the following is an example of sustainable

forest management? In the U.S., what do we currently do with the

nuclear waste generated by nuclear power plants?

Questions - Social Which of the following is the most commonly

used definition of sustainable development? The wealthiest 20% of people in the U.S. own

approximately what percent of the nation’s privately held wealth?

Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans?

Higher levels of education generally lead to... Which of the following populations has the

highest rate of growth?

Questions - Economic Many economists argue that electricity prices in

the U.S. are too low because… Which of the following countries has now passed

the U.S. as the biggest emitter of the greenhouse gas carbon?

Which of the following is a leading cause of depletion of fish stocks in the Atlantic Ocean?

Which of the following is the most commonly used definition of economic sustainability?

Which of the following is the primary reason that gasoline prices have risen over the last several decades in the U.S.?

Results Environmental

6 questions

Mean: 4.39/6SD: 1.4873%

Social5 questions

Mean: 3.55/5SD: 1.2371%

Economic5 questions

Mean: 3.03/5SD: 1.2761%

Total16 questions

Mean: 11.08/16SD: 3.21

69%

Example Analysis

Mean1

4

7

10

13

16

9.5710.57 10.95 11.34

FreshmanSophomoreJuniorSenior

ANOVAF(3,1330)=8.09, p<.001

Example Analysis Compare sub score by major

Economic score Economics: 3.82

Aerospace engineering: 4.00 Animal science: 3.15 Finance: 3.02 English: 2.80 Accounting: 2.63

Moving Forward Refine

Weaknesses Some answers change over time Give us your suggestions!

Publish

Apply (put path model here?)

Acknowledgements Funded by

The Ohio State Office of Sustainability http://sustainability.osu.edu/

OSU’s School of Environment & Natural Resources

http://senr.osu.edu/

The many faculty and graduate students who contributed their time and expertise

Questions? Environment and Social Sustainability

Lab www.ess.osu.edu Contains:

This presentation The knowledge questions with multiple choice

answers

Lab email essl@osu.edu