a user assessment of decision support tools
DESCRIPTION
69th SWCS International Annual Conference July 27-30, 2014 Lombard, ILTRANSCRIPT
A User Assessment of Decision Support Tools for Addressing Nonpoint Source Pollution in the Saginaw Basin Watershed:
Environmental Learning Using Computer Interactive Decisions (ELUCID) and the Great Lakes Watershed Management System (GLWMS)
Laura Young, Research AssistantSaichon Seedang, Research Specialist
Michigan State University Institute of Water Research
69th Soil Water Conservation Society International Annual ConferenceMaking Waves in ConservationJuly 29, Lombard, Illinois
CONTEXT & SYSTEMS OVERVIEW
Context
Awarded Great Lakes Restoration Initiative (GLRI) grant
Enhance technical assistance
Accelerate BMP implementation
Reduce non‐point source pollution
Trainings
• May 2014• Overview and walkthrough of systems• Tutorials• Evaluation
ELUCID
Environmental Learning Using Computer Interactive Decisions
Developed with input from various stakeholders
Flint River Watershed
ELUCID
Engage and inform various users groupsField staff, watershed groups, teachers, the public
ELUCIDSubwatershed Prioritization Field‐level Prioritization
ELUCID
Seamlessly linked with other systemsEPA databases, GLWMS
Great Lakes Watershed Management System
Evolved through various multi‐partner efforts
Four GLRI priority watersheds (Saginaw Bay)
Great Lakes Watershed Management System
Subwatershed prioritization for NPS pollutants
Great Lakes Watershed Management System
Provides baseline NPS pollution model estimates at field‐scales
Great Lakes Watershed Management System
Conducts field‐scale scenario evaluations of land cover changes or best management practices (BMPs)
Great Lakes Watershed Management System
FRAMEWORK FOR EVALUATION
Assessment PurposeTo evaluate:
– Perceived usefulness of system2
• Prioritizing high risk sites• Engaging producers
– Impacts of usage on efficiency at work– Integration into conservation work processes1
– Ease of use1– Transparency1– Anticipated Usage– Identify additional features needed to improve DSS’s– Participants’ understanding of material presented at the training
1. Inman, D., M. et. al. 2011.
2. Ives., B., M. Olson, J. Baroudi. 1983.
Survey Instrument
• Types of Questions– ELUCID, GLWMS, Training– Five‐point Likert Scale (51)– Multiple Choice (7)– Short Answer (12)
• Administered online through Survey Gizmo
RESULTS
Survey Population
Sample size: 17 trainees
Response Rate: 82%– 14 completions– 1 partial completion– 2 did not take survey
Profession
Will help me work with producers to place BMPs on high risk areas
Will help me work with producers to place BMPs on high risk areas
Will increase my ability engage with a producer
Will increase my ability engage with a producer
A producer will more motivated to implement a BMP on a high risk area
after reviewing data in ELUCID
A producer will more motivated to implement a BMP on a high risk area
after reviewing data in GLWMS
ELUCID
GLWMS
Perceived Usefulness
ELUCID
GLWMS
Will help me work with producers to place BMPs on high risk areas
Will help me work with producers to place BMPs on high risk areas
Perceived Usefulness
A producer will be more motivated to implement a BMP on a high risk area
after reviewing data in ELUCID
A producer will be more motivated to implement a BMP on a high risk area
after reviewing data in GLWMS
ELUCID
GLWMS
Perceived Usefulness
Will increase my ability to engage with a producer
Will increase my ability to engage with a producer
ELUCID
GLWMS
Perceived Usefulness
Will make my work easier
Will make my work easier
Will reduce the amount of effort I spend on certain aspects of my work.
Will reduce the amount of effort I spend on certain aspects of my work.
Will reduce the amount of time I spend on certain aspects of my work.
Will reduce the amount of time I spend on certain aspects of my work.
Perceived Impacts on Efficiency
ELUCID
GLWMS
Will make my work easier
Will make my work easier
Perceived Impacts on Efficiency
ELUCID
GLWMS
Will reduce the amount of effort I spend on certain aspects of my work.
Will reduce the amount of effort I spend on certain aspects of my work.
Perceived Impacts on Efficiency
ELUCID
GLWMS
Will reduce the amount of time I spend on certain aspects of my work.
Will reduce the amount of time I spend on certain aspects of my work.
Perceived Impacts on Efficiency
ELUCID
GLWMS
ELUCID will easily integrate into my conservation work process.
The GLWMS will easily integrate into my conservation work process.
Perceived Potential for Integration
ELUCID fills a gap in the current set of tools I frequently use to do my job.
The GLWMS fills a gap in the current set of tools I frequently use to do my job.
ELUCID
GLWMS
ELUCID fills a gap in the current set of tools I frequently use to do my job.
The GLWMS fills a gap in the current set of tools I frequently use to do my job.
Perceived Potential for Integration
ELUCID
GLWMS
ELUCID will easily integrate into my conservation work process.
The GLWMS will easily integrate into my conservation work process.
Perceived Potential for Integration
ELUCID
GLWMS
Future Work
• Follow‐up survey– Barriers to targeted conservation– How respondents have used the system
• Focus groups• Examining Google Analytics data with evaluation results
ReferencesInman, D., M. Blind, I. Ribarova, A. Krause, O. Roosenschoon, A. Kassahun, H. Scholten, G. Arampatzis, G.
Abrami, B. McIntosh, P. Jeffery. 2011. Perceived effectiveness of environmental decision support systems in participatory planning: Evidence from small groups of end‐users. Environmental Modelling & Software. 26, 302‐309.
Ives, B. M. Olson, and J. J. Baroudi. 1983. The Measurement of User Information Systems. Communications of the ACM. 26 (10): 785‐793.
Matthews, K.B., M. Rivington, K.L. Blackstock, G. McCrum, K. Buchan, and D.G. Miller. 2010. Raising the bar? The challenges of evaluating the outcomes of environmental modelling and software. Environmental Modelling & Software. 26: 247‐257.
Rhee, C. and H. Raghav Rao. 2008. Evaluation of Decision Support Systems. International Handbook on Information Systems.
QUESTIONS?
www.iwr.msu.edu/flintriverelucid.iwr.msu.eduwww.iwr.msu.edu/glwms
Laura [email protected]
517‐353‐8587