Transcript
Page 1: Engaging Students in Distance Learning

http://www.pages.drexel.edu/~jw65/

Engaging Students in Distance Learning

Jim WatersSusan Gasson

The iSchool at Drexel

Page 2: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

If students have already paid us, why should we care if they are engaged ? Altruism – the long view? Pragmatism – they can always leave

taking their tuition money with them Student feedback and tenure decisions? Word gets out! – social networks

Students can vote with their virtual feet and Their very real wallets

Page 3: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Agenda How do I know if students are engaged? What is the effect of:

Question design? Course scaffolding? Instructor moderation? Being an entertaining instructor?

So what?

Page 4: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Research Study Analyzed course interactions via discussion board on

Blackboard learning system. 12 online MS courses (info. systems./info. Science) 313 Students, 11,497 messages Posts to discussion board + small group discussions

Analyzed Thread depth, thread length, participants Cognitive content of message Interactive intent of message Patterns of message sequences

Examined student outcomes related to interaction Pre and Post questionnaires Demographics and Attitudinal data

Page 5: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Three modes of learning engagement

Individual Participation Active Course Involvement Iterative Social Engagement

Fluid: students can move between modes reacting to drivers

Page 6: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 1: Individual Participation The semi-transparent participant

Interacts with materials Internalizes knowledge Contractual obligation postings Broadcast messages Superficial learning Hermit!

Page 7: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 2: Active Course Involvement Demonstrates (some) genuine interest

Interacts with peers (after a fashion) Translates community knowledge Relates posts to own experience or knowledge Internalizes community knowledge Ego-centric approach Small group or clique interactions

Page 8: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 3: Interactive Social Engagement Motivated for interactive learning

Committed to greater group learning Interacts freely with peers Looped learning cycles Iterative internalizations/externalizations Social construction of knowledge

Page 9: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

How can we tell what is going on?

Page 10: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 1: Course Participation

How Do We Know A Project Is On Track? Instructor 10/21/07 2:36 PMRE:How Do We Know A Project Is On Track? S18 10/25/07 12:09 PMRE:How Do We Know A Project Is On Track? S10 10/25/07 6:29 PMRE:How Do We Know A Project Is On Track? S21 10/25/07 8:30 PMRE:How Do We Know A Project Is On Track? S17 10/26/07 7:38

PMRE:How Do We Know A Project Is On Track? S22 10/28/07 6:19 PMRE:How Do We Know A Project Is On Track? S2 10/28/07 7:04 PMRE:How Do We Know A Project Is On Track? S4 10/28/07 10:24 PMRE:How Do We Know A Project Is On Track? S10 10/28/07 10:26 PMRE:How Do We Know A Project Is On Track? S7 10/28/07 10:46 PMRE:How Do We Know A Project Is On Track? S8 10/29/07 12:59 AMRE:How Do We Know A Project Is On Track? S11 12/14/07 11:34 AM

Page 11: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 1: Course Participation

Page 12: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 2: Active Course InvolvementUnanswered questions Instructor 10/5/07 3:23 PM

RE:Unanswered questions S1 10/5/07 6:59 PMRE:Unanswered questions S17 10/6/07 3:41 AM

RE:RE:Unanswered questions S19 10/8/07 12:33 AMRE:RE:RE:Unanswered questions S20 10/8/07 10:52 AM

RE:Unanswered questions S13 10/6/07 10:52 AMRE:RE:Unanswered questions S18 10/7/07 4:14 PM

RE:RE:RE:Unanswered questions S6 10/9/07 9:45 PMRE:Unanswered questions S12 10/6/07 11:04 AM

RE:RE:Unanswered questions S20 10/7/07 10:34 AMRE:Unanswered questions S9 10/7/07 6:49 AMRE:Unanswered questions S21 10/7/07 4:36 PMRE:Unanswered questions S10 10/7/07 5:31 PMRE:Unanswered questions S4 10/7/07 10:59 PMRE:Unanswered questions S19 10/8/07 12:07 AMRE:Unanswered questions S8 10/8/07 2:21 PM

BOK as a communications/marketing tool S12 10/9/07 1:18 PMRE:Unanswered questions S8 10/8/07 2:48 PMRE:Unanswered questions S14 10/12/07 2:33 PM

RE:RE:Unanswered questions S12 10/12/07 3:22 PMRE:Unanswered questions S15 10/12/07 3:26

Page 13: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 2: Active Course Involvement

Page 14: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 3: Interactive Social Engagement

Page 15: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Mode 3: Interactive Social Engagement

Page 16: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

What makes students engage at higher

levels in a distance-learning course?

Page 17: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Question Frequency vs. response

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00LI

S-1

b

LIS

-1a

IS-3

IS-2

IS-4

IS-1

b

LIS

-2c

LIS

-2b

CO

M-1

b

CO

M-1

a

IS-1

a

LIS

-2a

Questions

Posts/question

Linear (Posts/question)

Page 18: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Question Design

I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project]

Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives?  Is this a "real" model that could be used on "real" projects? [Fast or slow]

I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding]

Page 19: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

And the Winner is

I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding]

Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives?  Is this a "real" model that could be used on "real" projects? [Fast or slow]

I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project]

Page 20: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

GoodI want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [Cooking up a new project]

150 posts Several sub-threads extremely deep (7 or 8 levels) Critique, feedback, support and facilitationWell-placed faculty moderation, nudges rather than cattle

prods Well-bounded but open-ended: students define problem Deliberately pitched as a cooperative task Concrete (well-defined) task Students negotiate the task meaning collaboratively

Page 21: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

AverageI would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [Fact-finding]

85 postsModerate sub-thread depth (mostly 3 or 4 levels) 31% were messages from Instructor to students20% were messages from students to InstructorWell-placed faculty moderation, focus on challenging assumptions. Reasonably open-ended problem Far less cooperative inter-student activity Not pitched as a cooperative activity Students not answering a common question, but question is

defined

Page 22: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

BadCritically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or slow]

46 postsLimited sub-thread depth - mostly 2 (question then single response)45% were messages from Instructor to students37% were messages from students to Instructor18% were student-student messages Faculty intervention much more critical (didactic) Five questions in one: 1 was open-ended 4 bounded Very little cooperative inter-student activity Not pitched as a cooperative activity

Page 23: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Good questions tended to be

Early rather than later First question in the week Early weeks better than later weeks

Open (scope), but bounded (problem structure) Permitted students to call upon their personal experience

with IT or organizations Permitted many ways to approach the issues

Negotiated rather than defined Permitted collaborative interpretation Allowed students to contribute by defining their own take

on the question. Relevance to students helps – war stories,

company policies and approaches

Page 24: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Bad questions tended to be

Following a highly-interactive question or later in the course Later questions much less interactive and constructive

across courses than earlier questions Cognitively complex

Containing multiple parts that needed to be considered in turn, or

Overly abstract, so students could not draw on their personal experience.

Socially isolating Fewer opportunities for interpretation and collaboration

in answering the question.

Page 25: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

How does course scaffolding affect

student engagement?

Page 26: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Scaffolding Something to hang construction on Solid foundation for task

Materials Discussion Support for knowledgeable peers to contribute

Task requires an extension of prior abilities Provides a structure on which students can

build knowledge Task must be just beyond current unaided

skills but doable with help

Page 27: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Poor ScaffoldingYou've been asked to read the ALA Code of Ethics plus two other codes of ethics of your choice.  What did you learn from this process?  Did any common themes or concerns tend to emerge?  What did you relate to in the ALA Code of Ethics?  Were there things that seemed problematic, or that you disagreed with?

Codes of Ethics Instructor 1/28/08 3:15 PMRE: Codes of Ethics S5 1/31/08 7:00 AMRE: Codes of Ethics S15 1/31/08 5:49 PMRE: Codes of Ethics S13 1/31/08 9:17 PMRE: Codes of Ethics S14 2/1/08 12:05 AM

RE: Codes of Ethics S16 2/1/08 12:54 PM RE: Codes of Ethics S9 2/1/08 1:08 PM

RE: Codes of Ethics S17 2/3/08 1:20 PM RE: Codes of Ethics S11 2/2/08 3:26 PMRE: Codes of Ethics S18 2/3/08 4:23 PMRE: Codes of Ethics S19 2/4/08 5:33 PMRE: Codes of Ethics S6 2/5/08 6:24 PMRE: Codes of Ethics S1 2/5/08 11:03 PM

Page 28: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Better ScaffoldingCan ethical behavior really be codified by a professional organization?Can ethical behavior be enforced? How?

Page 29: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Unscaffolded Learning materials

A long list of codes of ethics web-pages Three abstract ethics articles

A body of solid material but this did not directly relate to the posted question or give a framework for answering the question

Page 30: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Scaffolded Learning materials

A description of ethical models Worksheet for ethical decision making

Actions and consequences Responsibilities and obligations A theoretical and pragmatic platform from which

discourse could be built Three sparse pages of bullet-points

Materials contained less information but provided a structure for thinking.

Page 31: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

How does instructor moderation affect

student engagement?

Page 32: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Moderated or leave-alone ?

Two sections of an IS course delivered at the same time – same basic syllabus

~Same number of students (23/24) Selected six “identical” questions on each

section Different Instructor approach Heavy moderation vs. lightweight

moderation

Page 33: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Question Heavy Moderation

Low Moderation

Systems Analyst as problem solver 69 74

Agile methods 96 97Project design 150 97

Requirements Analysis 96 83Fact Finding 85 90Data Modeling Practice 182 180Average 112 103Tot messages 238 268Tot words 26270 57128 Average words/student

message110.38 213.16

Questions and Approach (messages)

Page 34: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Agile methods Heavy Moderation Low Moderation

Total Messages 96 97Instructor – student

messages 16 (17%) 0Student messages 80 97Student-student

messages 28 73Student-instructor

messages 52 24Deep thread messages

(students) 65 44Deep sub-threads ( 4

levels or greater) 10 8Deep sub-threads w/o

instructor intervention 2 8

Page 35: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Instructor interaction and student posts

0

500

1000

1500

2000

2500

INST-2

INST-4

INST-8

INST-8

INST-6

INST-8

INST-3

INST-4

INST-7

INST-3

INST-5

INST-1

Instructor DB postsStudent Posts

Page 36: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Are students more engaged when the

Professor is entertaining?

Page 37: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Deconstructing the Entertaining Professor Highly knowledgeable industry professional

Very Popular Instructor High level of interaction with students Regular internet chat presence Projects personality into discussion

High percentage of social interactions Voluntarily discusses hobbies, weather, music,

Disneyworld, cooking, children, Dickens, vintage cars, pets, gardening, insects, Star Wars, birds, Nintendo, Scrabble, foreign films, beer ….

Injects lots of jokes

Page 38: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Student participationStudents Discussion

Board Visits

Posts Topic Threads

Posts/Thread

23 24095 2745 67 40.9

Student Posts

StudentPosts/Thread

FormalQuestions

StudentPosts/Question

AverageThreadDepth per question

1648 24.7 30 67 8.6

Page 39: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Never mind the width, feel the quality

Threads show evaluation and analysis Some hypothesis formation Fairly advanced cognitive activity Some student-student interactionBut, some chaff among the wheat

5% Un-focused Anecdotes 32% fluff posts: “LOL,” “Awesome [dead rock star] story!”

“OMG,” “Pictures of gardening implements,” “Lawyer Jokes” 50% contribute to learning 33% student-Instructor messages

Still, pretty successful overall High student satisfaction Grades were comparable with Prof. Serious

Page 40: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Introducing Professor Serious

Same course – Same Syllabus Highly knowledgeable industry

professional Very skilled Instructor Low level of direct interaction with students Strong Topic-focus Little social interaction

Page 41: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Student participationStudents Discussion

Board Visits

Posts TopicThreads

Posts/Thread

24 13079 1458 352 4

Student Posts

StudentPosts/Thread

FormalQuestions

StudentPosts/Question

AverageThreadDepth per question

1334 3.79 14 95.3 5

Page 42: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Quality ? More collaborative learning Messages longer and more detailed Fairly advanced cognitive activity Much stronger student-student interaction

Stronger awareness of value of peer interactionsSo what?

High task-Focus 2.5% fluff posts 80% substantive knowledge-building posts 15% student-Instructor messages

Successful overall Moderate student satisfaction Grades were comparable with Prof. Entertaining

Page 43: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

The tale of the tape Instructor participation of Prof. Entertaining

inflates thread depth slightly More posts, but smaller % focus on knowledge-building

Student-student posts more productive than student-instructor posts

Overall productive (knowledge building) activity was about the same for the two Professors

Student satisfaction slightly lower for Prof. Serious than Prof. Entertaining Fewer posts, but greater % focus on knowledge-building

But Stronger thought leaders for Prof. Serious Explicit kudos for peers in Prof. Serious

Page 44: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

ConclusionsQuestion design can be crucial to

engagement Discussion needs to be framed as

collaborative not competitive Relevance to students helps – war stories,

company policies and approaches No payoff for frequency of questions

Course scaffolding aids engagement Focus and framing of questions Must support task in concrete manner

Page 45: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

ConclusionsSuccessful course moderation hinges

on quality not quantity or frequency Knowing when to intervene Does not mean “being absent”

Being entertaining is not essential for success Tradeoff between popularity with students

and peer-engagement by students May shift focus from peers to instructor Excessive interaction is a lot of work

Page 46: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Students think they are learning more from peer-interaction

this was so helpful because often I was struggling with the same thing so I could learn from their errors and gain new information from the answers to their questions

I was moved to comment on how refreshing the lack of competition in the communications for the online classes seemed to me. It was a discussion and a sharing of experiences

Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback

No question that the on line discussion was critical to getting me through the class. There were mostly questions about how to..I've never done this before.

I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.

Page 47: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Related References Waters, J. '

Social Network Behavior, Thought-Leaders and Knowledge Building In An Online Learning Community', Proceedings of  Hawaii Intl. Conference on System Sciences (HICSS-41), Knowledge Management Track, Jan. 2008.

Gasson, S. and Waters, J. “How (not) to construct ALN course questions that encourage student participation in peer collaboration and knowledge construction,” 40th Hawaii International Conference on System Sciences, Hawaii, January 2007.

Waters, J., and Gasson, S. "Social Engagement in an Online Community of Inquiry," 27th International Conference on Information Systems (ICIS), Milwaukee WI, 2006.

Waters, J. “Determinants of Engagement in an Online Community of Inquiry,” The 12th Sloan-C International Conference on Online Learning, November 2006, http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot

Waters, J., and Gasson, S. "Strategies Employed By Participants In Virtual Learning Communities," Hawaii Intl. Conference on System Sciences (HICSS-38), Collaboration Systems and Technology track, IEEE Software Society, Hawaii, January 2005, p. 3b.

A full list of publications, with full copies of articles, is available at http://www.pages.drexel.edu/~jw65/publications.htm

Page 48: Engaging Students in Distance Learning

Jim Waters, Susan Gasson Drexel University, 2009

Credits Dr. Susan Gasson

MBA, PhD from University of Warwick (Warwick Business School), UK. Prior to that, she worked in systems design , IS management, and IS software architecture

consultancy. Researches collaborative processes of design, problem-solving, learning. Associate Professor at Drexel University, Philadelphia (USA). Involved in online education since 2000(!).

Dr. Jim Waters BA in Psychology from Warwick University, UK (1979), MSc in Occupational Psychology from

University of Hertfordshire, UK (1991), MS in Information Systems (2002) and PhD (2009) from the iSchool at Drexel University.

Prior to that, he enjoyed a substantial career in systems design, management and IS consultancy.

Doctoral Candidate at the iSchool at Drexel University, graduating Summer 2009. Principal research interests lie in Online Collaborative Knowledge Building, Technology-Supported Learning, Student Role-Behavior in Online Learning Communities and HCI.

This material is based upon work supported by the National Science Foundation under Grant No. IIS-0347595. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.


Top Related