watershed hero | scenario-based elearning hamood alrawili, carrie blaustein, ora tanner, and ana...

Watershed Hero|scenario-based eLearning

Hamood Alrawili, Carrie Blaustein, Ora Tanner, and Ana Vizcaino

EME6613 under the guidance of Dr. Yiping Lou12.3.14

Hamood

Carrie

Section 1

Ora

Ana


Introduction"It [scenario-based elearning] allows us to immerse learners in real problems, provide them with guidelines and principles to deal with those problems, and show them the consequences of their choices in a safe, efficient manner.”

~Ruth Clark

Whole Group

Hamood

Carrie

Whole Group

Ora

Project Background• Providing clean water and improving urban infrastructure are two of the three National Academy of Engineer’s Grand Challenges.• A goal of USF Water Awareness Research and Education (WARE) is to promote and engage students with STEM education. • Stormwater runoff causes flooding, contaminated drinking water, and excessive algae growth.• Understanding watersheds is crucial to designing a sustainable solution for these problems.

Ana


Rationale•The elearning module will provide accurate, engaging, practical, and relevant scenario-based learning to generate greater interest and allow students to review and apply watershed concepts at the local level.

•Providing interactivity, game-style assessments, engaging real-life graphics, and custom audio support this effort.


Target Learners1st Implementation Hillsborough County traditional middle school science students (8th grade). These students often perform below grade level on the reading FCAT. Implemented November 20, 2014 during Great American Teach-In at Adams Middle School.* Students completed the module as a whole class.


Target Learners2nd Implementation Hillsborough County magnet program middle school science students (6th, 7th, and 8th grade) Implemented December 2, 2014 in an afterschool enrichment program at Ferrell Girls’ Preparatory Academy. Students completed the module individually in computer lab.


Project Goals1. Utilization of the Engineering Design Process as a way

for students’ guidance and instruction2. Use scenario-based learning that students can relate to

generate greater interest on their part3. Design and implement instruction and interactive

activities that promote interest in STEM content4. Integrate evaluation tools that award points/money to

students to assess their knowledge of the materials provided

5. Challenging students in a game environment to earn points will enhance their motivation to learn and to finally place as many rain gardens along the watershed as possible


Learning Objectives1. Define what a watershed is and identify key

components.2. Locate and select the boundaries of a watershed using

Web 2.0.3. Examine major sources of pollution in a watershed and

pollution effects.4. Utilize the engineering cycle to design a stormwater

runoff management system using green infrastructure.5. Evaluate the effect of a rain garden system on water

quality and flooding due to stormwater runoff.


Next Generation Science Standards

A Science Framework for K-12 Science Education provides the blueprint for developing the Next Generation Science Standards (NGSS). The watershed module is designed to elicit the starred items from the eight practices below:*1. Asking questions (for science) and defining problems (for engineering)*2. Developing and using models*3. Planning and carrying out investigations*4. Analyzing and interpreting data 5. Using mathematics and computational thinking*6. Constructing explanations (for science) and designing solutions (for engineering) 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information



Theoretical Framework• SAM: The Successive Approximation Model

• Kirkpatrick’s Levels of Evaluation: Reactions, Learning, Transfer, Results• Scenario Based e-Learning: domains, trigger, case data, strategies• Design for How People Learn: memory, attention, motivation (elephant)

Design Based Research• “a systematic but flexible methodology aimed to improve

educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world setting and leading to contextually-sensitive design principles and theories.”*

• Five basic characteristics : (a) pragmatic; (b) grounded; (c) interactive, iterative, and flexible; (d) integrative; and (e) contextual*

• Good design-based research: (1) goals of designing learning environments and learning theories intertwine, (2) continuous cycles of design, enactment, analysis, and redesign (3) leads to sharable theories with implications (4) research guides function in authentic settings, (5) relies on methods that document and connect enactment to outcomes+


*Wang & Hannafin (2005). Design-Based Research and Technology-Enhanced Learning Environments. (Wang and Hannafin2005_DesignBased Research.pdf)+Designed-Based Research Collective (2003). Design-Based Research: An Emerging Paradigm for Educational Inquiry. Educational Researcher, 32, 5-8. http://www.tophe.net/papers/dbrc03.pdf (Links to an external site.)

https://usflearn.instructure.com/courses/998134/files/39106303/download?verifier=JUYKgJ9kEWcYswyTrmdWGX0LmzPcihNeggjDgxRD&wrap=1

http://www.tophe.net/papers/dbrc03.pdf

http://www.tophe.net/papers/dbrc03.pdf

Approach1. Introduce trigger and pedagogical guide (virtual coach)

to define problem and generate interest2. Utilize game-like interactions (i.e. animations, graphics,

sound effects, score, challenge) to keep learner engaged3. Promote 21st century skills, such as critical thinking and

problem solving to promote STEM education4. Encourage water awareness by teaching students about

water pollution in watersheds and efficient management of stormwater runoff

5. Instill in students a research culture that supports sustainable solutions and increased interest in STEM fields

6. Teach students resourceful ways to reduce flooding in urban green spaces


Learning Theories• Our scenario-based learning module aligns with cognitivist and constructivist frameworks.

• Students will be guided through an actual problem where they can apply the engineering design cycle to interact and earn points that can be used for the final round challenge of designing a rain garden stormwater management system.

• Students will reflect on problems and solutions to construct their own understanding.

• Cognition, motivation, retention, and memory principles were applied as described in“Design for How People Learn.


Section 2Section 3

Hamood

Whole Group

Ora

Ana

Instructional Design Principles

Mayer’s Instructional Design Principles employed:

Modality principle: People learn better from animation and narration than from animation and on-screen text.

Multimedia principle: People learn better from words and pictures than from words alone.

Carrie



More Instructional Design Principles


Segmenting principle: People learn better when a multimedia lesson is presented in learner-paced segments rather than as a continuous unit.

Personalization principle: People learn better when the words are in conversational style rather than formal style.

Temporal contiguity principle: People learn better when corresponding words and pictures are presented simultaneously rather than successively.

Coherence principle: People learn better when extraneous words, pictures, and sounds are excluded rather than included.


More Instructional Design Principles


Signaling principle: People learn better when the words include cues about the organization of the presentation.

Voice principle: People learn better when words are spoken in a standard-accented human voice than in a machine voice or foreign-accented human voice.

Redundancy principle: People learn better from animation and narration than from animation, narration, and on-screen text.

Spatial contiguity principle: People learn better when corresponding words and pictures are presented near rather than far from each other on the page or screen.

Research Questions1. Which eLearning design strategies make learning relevant and does this relevance enhance interest, engagement, and learning?

a. Does scenario-based e-learning promote interest and learning?b. Do game-like animations, graphics, sounds, challenge, and a scoring system promote interest and learning?c. Do realistic characters, photos, sounds, and a local setting create relevance that enhances learning? d. Does reflection improve comprehension and retention of science content?e. Does using the Engineering Design Cycle as a model guide students through analyzing and solving problems better than just presenting a problem and requiring a solution?


Design & Development Process

Development platform: • Articulate Storyline 1 and 2• Adobe PhotoshopGroup collaboration channels and processes:• Google Docs• Email• Canvas Group• Big Blue ButtonConsultation with practitioners:• Email• Meeting in classSME Review: Ryan Locicero, Summer Hires, Jennifer Butler, Kristy Carew, Mirko Soko (main client)• Email• In class



Design & Development ProcessPeer Evaluation:Based on peer feedback changes were made in navigation, audio, and feedback improvementsOne-on-one learner evaluation: Designers’ three children provided feedback which led to improvements in navigation, content chunking, additional gaming components, redesign of rain garden system activityField Implementation:• Twelve students at Adams Middle School tested the product, but did not complete it.• Student dialogue, student behaviors, and additional notes were taken through process.• Made changes in feedback and navigation within the module based on classroom observation.• Surveyed Mirko based on his classroom observations of three other class periods.• Six middle/high school students completed module and survey individually.• Fourteen 6th-8th grade students (all female) in an afterschool program completed the module and provided additional feedback for future design iterations

Design & Development Process

Roles of group members:Hamood Alrawili: watershed pollution activity, communication, revisionsCarrie Blaustein: watershed boundaries activity, communication, update plan and timelines, beta testing, revisionsOra Tanner: concept, framework, objectives, design, drafts, integration of modules, narration and sound effects, trigger event, rain garden system construction (culminating gaming activity), engineering design cycle, one-on-one learner evaluation, classroom observation, beta testing, revisionsAna Vizcaino: watershed characteristics, formative evaluation plan, students and teacher surveys, revisions



Iteration 1 Design Proof/Revisions

Focus of evaluation: The purpose of the design proof was to provide a first look at our module transferred from the storyboard into the development platform that we were going to use to show it as a final product. This step initiated the iteration process.Changes applied were related to narration to make it more enthusiastic/appealing to the target audience; fix glitches on slides; change the wording of titles to be more learner friendly; and made us reconsider the number of activities that were going to be added to the module.

Participants: Classmates from EME6613 Development of Technology-Based Instruction, and our instructor


Iteration 2 Alpha Tests/Revisions

Focus of evaluation: The purpose of the evaluation is to ensure that the module provides effective learning of the watershed and the Engineering Design Cycle. We also focused on usability to make sure students can easily navigate throughout the course (buttons, menus, etc.) as well as user experience (how engaged the learners feel). We surveyed learners to discover the learner outcomes i.e. how engaging the course was, and whether they learned from it.

Participants: We surveyed three learners between the ages of 13 and 18 (2 males, 1 female). All three participants were children of one of the designers from our group; they participated from the comfort of their home. The participants were within the grade level of the intended target audience for the module.

In order to make the course engaging in a classroom group environment, numerous interactions and effects were added.


Iteration 3 Beta Tests

Focus of evaluation: The purpose of the evaluation is to ensure that the module provides effective learning materials. We also focused on usability to make sure students can easily navigate throughout the course (buttons, menus, etc.); we surveyed learners to discover the learner outcomes i.e. how engaging the course was, and whether they learned from it.

Participants: Mirko had taught the module in his classes for 12 middle school students, and Ora took notes about their behavior and interactions through the process.We surveyed Mirko based on his students responses. Three students in a federal college-access program, three children of the designer, and fourteen 6th-8th grade female students in an after school program also completed the module and survey individually

Section 2Carrie

Section 4

Whole Group

Ora

Ana

Final Product Illustrations

• After conducting several iterations of testing with SMEs and target learners, modifications were made to the elearning module based on user feedback.•The following slides illustrate the changes that were made.Hamood



Final Product IllustrationsInform Students of the Objectives


Final Product IllustrationsProvide Feedback


Final Product IllustrationsElicit Performance


Final Product IllustrationsPresent the Content

Video presentation of information On-demand presentation of information through pop-ups


Final Product IllustrationsAssess Performance


Final Product IllustrationsStimulate Recall of Prior Knowledge


Survey Results

Section 2Hamood

Carrie

Whole Group

Section 5

Ana

Discussion of Results• Overall, there was a very positive response to the Watershed eLearning module.• 80% of students found the module easy to navigate• 80% liked the look and feel of the module and felt the game-like components made learning fun• 80% of students believe they have a better understanding of the watershed as a result of interacting with the module• 85% of participants felt being given an actual problem and discovering how to solve it made learning easierOra


Qualitative Data1st Implementation- Adams Middle School

• N=12• Low performing students in 8th grade science class• Teacher implemented as whole class activity• Music (although very low), opening animation, bright colors captured students attention at the offset• Real-life avatar appeared to help students make an immediate connection• Game-like components facilitated students in persisting with content in light of struggling due to poor reading skills• No off-task behavior observed during implementation of module• Teacher reported that two of the most challenging students (who are usually off task and causing problems) were the most engaged during the lesson


Qualitative Data2nd Implementation- Ferrell Girls’ Preparatory Academy

• N=14• High performing magnet students in 6th-8th grade afterschool program•Students worked on module individually in computer lab• Music, opening animation, bright colors captured students attention at the offset• Real-life avatar appeared to help students make an immediate connection• Game-like components facilitated 6th grade students in persisting with content in light of struggling due to content being above their ability• No off-task behavior observed during implementation of module. All students completed module and survey.


Qualitative DataResponses from open-ended question: If you would change anything about the module, what would it be?

• “be more specific because I did not understand”• “To have more details in the instructions.”• “I would add a bar somewhere that shows where you are in the engineering design process so the student can know where they are in the process.”• “I would change the maps because it was a little hard but it was FUN”• “more videos less text”• There were a number of students who said they would not change anything and they liked everything about the module.


Qualitative DataResponses from open-ended question: How do you like the e-learning module? What parts do you like and what parts do you not like or had difficulty with?• “I like that I became more engaged with the subject. I like how the problems were actually realistic.”• “The graphics, the problem, and the solutions are what I liked”• “I liked the character and interactivity with buttons/ things.”• “I liked the e-learning module and I thought that it was fun and engaging.”• “i had no difficulty. I like the colors and the guide”• “Its good,I liked when I had to earn the money and buying that water thing”• “I like everything about it, how it teaches you quikly and how it shows me new things!”


Limitations• Module was tested as a whole class with the original target audience however students only went through Boundary activity. We have no data from teacher or students as to how effective the learning was other than classroom observation by the researcher.• Individual responses to survey were from a different school (magnet program) with different grade levels (6, 7, and 8) and a different population (single gender). Also the module was not completed in a science context.• Data may be skewed due to a number of participants not understanding survey questions (comprehension barrier due to grade level).• Content is not a part of the prescribed curriculum so motivation to implement or for students to learn material can be diminished.


Recommendations of Design Principles

• Design Feedback and Reflection: Use the real-life avatar to provide intrinsic feedback throughout entire module, especially Pollution activity, in addition to text feedback, sound effects, pop ups, etc.• Use avatar body language, replies or attitude meter so learners see results of their actions within the module.• Based on the classroom observation, increase the frequency of the feedback to provide a high level of guidance to novice users.• Delay feedback for the final process-oriented scenario of designing and building rain garden system.• Provide more explicit step-by-step directions in Boundaries, Pollution and Rain Garden activities.


Future Design & Research

• Future Design: Make the entire module a multi-level game• Future Design: Make the Engineering Design Cycle the module interface• Future Research: Effects of game-based science learning on struggling readers in the middle grades• Future Research: Effects of game-based science learning on girls in the middle grades


watershed hero | scenario-based elearning hamood alrawili, carrie blaustein, ora tanner, and ana...

Documents