adaptive learning systems: a review of adaptation

59
Adaptive Learning Systems Towards “Adaptation Engine” Peter Brusilovsky School of Information Sciences University of Pittsburgh, USA

Upload: peter-brusilovsky

Post on 18-Nov-2014

374 views

Category:

Technology


6 download

DESCRIPTION

A review of adaptation approaches in adaptive learning systems and a discussion of their implementation in modern e-learning.

TRANSCRIPT

Page 1: Adaptive Learning Systems: A review of Adaptation

Adaptive Learning Systems ���Towards “Adaptation Engine”

Peter Brusilovsky School of Information Sciences University of Pittsburgh, USA

Page 2: Adaptive Learning Systems: A review of Adaptation

Caveat Emptor

Page 3: Adaptive Learning Systems: A review of Adaptation

Overview

•  Adaptation Technologies (what can be adapted and how) – Origins – Review – Place in the “Big Picture”

•  How it could be implemented – “adaptation engine”

Page 4: Adaptive Learning Systems: A review of Adaptation

Key Aspects of Adaptive Systems

•  Adapting to what? – User knowledge – User interests – User individual traits

•  What can be adapted? – Adaptive sequencing of educational tasks – Adaptive content presentation – Adaptive ordering of search results

Page 5: Adaptive Learning Systems: A review of Adaptation

Technologies: The Origins

•  Pre-Web AES Technologies

–  ITS Technologies

– AH Technologies

•  Web Technologies

•  Post-Web Technologies •  Brusilovsky, P. and Peylo, C. (2003) Adaptive and intelligent Web-based educational systems.

International Journal of Artificial Intelligence in Education 13 (2-4), 159-172.

Page 6: Adaptive Learning Systems: A review of Adaptation

Pre-Web Technologies

Adaptive Hypermedia Systems Intelligent Tutoring Systems

Adaptive Hypermedia

Intelligent Tutoring

Adaptive Presentation

Adaptive Navigation Support

Curriculum Sequencing

Intelligent Solution Analysis

Problem Solving Support

Page 7: Adaptive Learning Systems: A review of Adaptation

Pre-Web Technologies

•  Intelligent Tutoring Systems –  course sequencing –  intelligent analysis of problem solutions –  interactive problem solving support –  example-based problem solving

•  Adaptive Hypermedia Systems –  adaptive presentation –  adaptive navigation support

Page 8: Adaptive Learning Systems: A review of Adaptation

How to Model User Knowledge

•  Domain model – The whole body of domain knowledge is

decomposed into set of smaller knowledge componens (skills, concepts, topics, etc)

•  Student model – Overlay model

•  Student knowledge is measured independently for each knowledge unit

– Misconceptions (bugs)

Page 9: Adaptive Learning Systems: A review of Adaptation

Simple overlay model

Concept 1

Concept 2

Concept 3

Concept 4

Concept 5

Concept N yes no

no

no yes

yes

Page 10: Adaptive Learning Systems: A review of Adaptation

Simple overlay model

Concept 1

Concept 2

Concept 3

Concept 4

Concept 5

Concept N yes no

no

no yes

yes

Page 11: Adaptive Learning Systems: A review of Adaptation

Weighted overlay model

Concept 1

Concept 2

Concept 3

Concept 4

Concept 5

Concept N 10 3

0

2 7

4

Page 12: Adaptive Learning Systems: A review of Adaptation

Bug models

Concept A

Concept B

Concept C

•  Each concept/skill has a set of associated bugs/misconceptions and sub-optimal skills

•  There are help/hint/remediation messages for bugs

Page 13: Adaptive Learning Systems: A review of Adaptation

Course Sequencing

•  Oldest ITS technology –  SCHOLAR, BIP, GCAI...

•  Goal: individualized “best” sequence of educational activities –  information to read –  examples to explore –  problems to solve ...

•  Curriculum sequencing, instructional planning, ...

Page 14: Adaptive Learning Systems: A review of Adaptation

ELM-ART: Exercise Sequencing

Web

er, G

. and

Bru

silo

vsky

, P. (

2001

) ELM

-AR

T: A

n ad

aptiv

e ve

rsat

ile s

yste

m fo

r Web

-bas

ed in

stru

ctio

n. In

tern

atio

nal

Jour

nal o

f Arti

ficia

l Int

ellig

ence

in E

duca

tion

12

(4),

351-

384.

Page 15: Adaptive Learning Systems: A review of Adaptation

Beyond Sequencing: Generation

Kum

ar, A

. (20

05) G

ener

atio

n of

pro

blem

s, a

nsw

ers,

gra

de

and

feed

back

- ca

se s

tudy

of a

fully

aut

omat

ed tu

tor.

AC

M

Jour

nal o

n E

duca

tiona

l Res

ourc

es in

Com

putin

g 5

(3),

Arti

cle

No.

3.

Page 16: Adaptive Learning Systems: A review of Adaptation

Adaptive Problem Solving Support

•  The core of Intelligent Tutoring Systems •  From diagnosis to problem solving support •  Low-interactive support

–  intelligent analysis of problem solutions •  Highly-interactive support

–  interactive problem solving support

Page 17: Adaptive Learning Systems: A review of Adaptation

Intelligent analysis of problem solutions

•  Intelligent analysis of problem solutions •  Support: Identifying misconceptions (bug

model) and broken constraints (CM) •  Provides feedback adapted to the user model:

remediation, positive help •  Low interactivity: Works after the (partial)

solution is completed •  Examples: PROUST, ELM-ART, SQL-Tutor

Page 18: Adaptive Learning Systems: A review of Adaptation

Example: ELM-ART

Page 19: Adaptive Learning Systems: A review of Adaptation

Interactive Problem Solving Support

•  Classic System: Lisp-Tutor •  The “ultimate goal” of many ITS developers •  Several kinds of adaptive feedback on every step

of problem solving –  Coach-style intervention –  Highlight wrong step –  What is wrong –  What is the correct step –  Several levels of help by request

Page 20: Adaptive Learning Systems: A review of Adaptation

Example: WADEIn

http://adapt2.sis.pitt.edu/cbum/ Bru

silo

vsky

, P. a

nd L

obod

a, T

. D. (

2006

) WA

DE

In II

: A c

ase

for a

dapt

ive

expl

anat

ory

visu

aliz

atio

n. In

: M. G

oldw

eber

and

P. S

alom

oni (

eds.

) Pro

ceed

ings

of

11t

h A

nnua

l Con

fere

nce

on In

nova

tion

and

Tech

nolo

gy in

Com

pute

r Sci

ence

E

duca

tion,

ITiC

SE

'200

6, B

olog

na, I

taly,

Jun

e 26

-28,

200

6, A

CM

Pre

ss, p

p. 4

8-52

.

Page 21: Adaptive Learning Systems: A review of Adaptation

Example-Based Technologies •  While focused on problem solving, ITS research

developed several adaptive example-based learning approaches

•  Example-based problem solving support –  Adaptively suggesting relevant examples for given

problem and student state of knowledge (ELM-ART) •  Adaptive worked out examples

–  Steps could be presented with different level of details (fading with knowledge growth)

–  Example steps could be replaced with problem steps

Page 22: Adaptive Learning Systems: A review of Adaptation

Adaptive hypermedia

•  Hypermedia systems = Pages + Links

•  Adaptive presentation

–  content adaptation

•  Adaptive navigation support

–  link adaptation

•  Could be considered as “soft” sequencing

–  Helping the user to get to the right content

Page 23: Adaptive Learning Systems: A review of Adaptation

Adaptive navigation support •  What could be done with links to provide

personalized guidance? •  Direct guidance •  Restricting access

–  Removing, disabling, hiding •  Link Ranking •  Link Annotation •  Link Generation

–  Similarity-based, interest-based

Page 24: Adaptive Learning Systems: A review of Adaptation

Adaptive Annotation: InterBook  

1. Concept role 2. Current concept state

3. Current section state 4. Linked sections state

4

3

2

1

√"

InterBook system

Page 25: Adaptive Learning Systems: A review of Adaptation

Adaptive Annotation: NavEx

Yudelson, M. and Brusilovsky, P. (2005) NavEx: Providing Navigation Support for Adaptive Browsing of Annotated Code Examples. In: C.-K. Looi, G. McCalla, B. Bredeweg and J. Breuker (eds.) Proceedings of 12th International Conference on Artificial Intelligence in Education, AI-Ed'2005, Amsterdam, the Netherlands, July 18-22, 2005, IOS Press, pp. 710-717

Page 26: Adaptive Learning Systems: A review of Adaptation

Adaptive Text Presentation���in PUSH (stretchtext)

Höö

k, K

., K

arlg

ren,

J.,

rn, A

., D

ahlb

äck,

N.,

Jans

son,

C. G

., K

arlg

ren,

K.,

and

Lem

aire

, B. (

1996

) A g

lass

box

app

roac

h to

ada

ptiv

e hy

perm

edia

. Use

r Mod

elin

g an

d U

ser-A

dapt

ed In

tera

ctio

n 6

(2-3

), 15

7-18

4.

Page 27: Adaptive Learning Systems: A review of Adaptation

Adaptive Animation in WADEIn

Page 28: Adaptive Learning Systems: A review of Adaptation

Adapting to Individual Traits

•  Source of knowledge –  educational psychology research on individual

differences •  Known as cognitive or learning styles

– Field dependence, wholist/serialist (Pask) – Kolb, VARK, Felder-Silverman classifiers

Page 29: Adaptive Learning Systems: A review of Adaptation

Style-Adaptive Hypermedia

•  Different content for different style – Pictures for visually oriented – Little success, a lot of negative evidence

•  Better idea: different interface organization and navigation tools for different styles – Adding/removing maps, advanced organizers,

etc.

Page 30: Adaptive Learning Systems: A review of Adaptation

Example: AES-CS

Interface for field-independent learners Tria

ntaf

illou

, E.,

Pom

port

is, A

., an

d D

emet

riadi

s, S

. (20

03) T

he d

esig

n an

d th

e fo

rmat

ive

eval

uatio

n of

an

adap

tive

educ

atio

nal s

yste

m b

ased

on

cogn

itive

sty

les.

Com

pute

rs a

nd E

duca

tion,

87-

103.

Page 31: Adaptive Learning Systems: A review of Adaptation

Example: AES-CS

Interface for field-dependent learners

Page 32: Adaptive Learning Systems: A review of Adaptation

Web Impact: Early Migration

•  Intelligent Tutoring Systems (since 1970) – CALAT (CAIRNE, NTT) – PAT-ONLINE (PAT, Carnegie Mellon)

•  Adaptive Hypermedia Systems (since 1990) – AHA (Adaptive Hypertext Course, Eindhoven) – KBS-HyperBook (KB Hypertext, Hannover)

•  ITS and AHS – ELM-ART (ELM-PE, Trier, ISIS-Tutor, MSU)

Page 33: Adaptive Learning Systems: A review of Adaptation

Technology Fusion Adaptive Web Adaptive Educational

Systems

Adaptive E-Learning

Page 34: Adaptive Learning Systems: A review of Adaptation

Web Age Technologies

Adaptive Hypermedia Systems Intelligent Tutoring Systems

Information Retrieval

Adaptive Hypermedia

Adaptive Information

Filtering

Intelligent Monitoring

Intelligent Collaborative

Learning

Intelligent Tutoring

Machine Learning, Data Mining

CSCL

Page 35: Adaptive Learning Systems: A review of Adaptation

Native Web Technologies •  Availability of logs

–  Log-mining –  Intelligent class monitoring –  Class progress visualization

•  One system, many users - group adaptation! –  Adaptive collaboration support

•  Web is a large information resource - helping to find relevant open corpus information –  Adaptive content recommendation

Page 36: Adaptive Learning Systems: A review of Adaptation

Adaptive Collaboration Support

•  Peer help / peer finding •  Collaborative group formation •  Group collaboration support

– Collaborative work support – Forum discussion support

•  Awareness support

Page 37: Adaptive Learning Systems: A review of Adaptation

Educational Recommenders

•  Motivated by research on IR and Recommender Systems

•  Content based recommender systems •  Collaborative recommender systems •  Social recommender systems (based on

social links) •  Hybrid Recommenders

Page 38: Adaptive Learning Systems: A review of Adaptation

Modeling User Interests

•  Concept-level modeling – Same domain models as in knowledge

modeling, but the overlay models level of interests, not level of knowledge

•  Keyword-level modeling – Uses a long list of keywords (terms) in place of

domain model – User interests are modeled as weigthed vector

or terms – Originated from adaptive filtering/search area

Page 39: Adaptive Learning Systems: A review of Adaptation

How it Fits Together?

Page 40: Adaptive Learning Systems: A review of Adaptation

Popular View on Adaptive Learning: Big PIcture

•  A learning course (system) is an organized collection of learning content (objects)

•  Students learn by moving from one content item to another interacting with each one depending on item nature (watch a movie, answer a quiz)

•  Results are stored and used for learner modeling and analytics

Page 41: Adaptive Learning Systems: A review of Adaptation

A View on Adaptive Learning

•  Adaptive learning could be achieved by adaptively selecting the next best content

•  The job of adaptation engine is to use data about student (obtained before and during the course) to suggest next content item

Page 42: Adaptive Learning Systems: A review of Adaptation

What is (Partially) Correct •  This is a valuable adaptation context, exactly the

place to use adaptive sequencing •  Sequencing is an effective adaptation approach,

comes in several well-explored brands: –  Mastery learning –  Remedial sequencing –  Proactive sequencing

•  But – any personalized guidance technology that can guide the learner to the most appropriate content could be used in this context and there are other ways to do it –  Adaptive navigation support –  Recommendation with a ranked list

Page 43: Adaptive Learning Systems: A review of Adaptation

Lessons Learned I •  Approaches that combine system-driven

adaptation with user ability to select content work better for “mature” learners that purely system-driven “Deus ex machina” approaches while sequencing is critical for younger kids –  If you want to apply sequencing, consider other

guidance approaches as well •  There are other approaches to support self-

regulated learning related to adaptation and they work really well – open learner model! –  If you build learner model, make it open!

•  Thanks, David, for explaining why we need it!

Page 44: Adaptive Learning Systems: A review of Adaptation

Exercise area

List of annotated links to all exercises available for a student in the current course grouped into topics

QuizGuide = OLM + ANS

Page 45: Adaptive Learning Systems: A review of Adaptation

•  Topic-based Topic-based+Concept-Based

Concept-based vs Topic-based ANS

Page 46: Adaptive Learning Systems: A review of Adaptation

Lessons Learned II

•  The largest impact is achieved by personalized guidance to complex activities (i.e., problems), while juggling static content has low impact –  If you focus on sequencing, make sure you

have advanced learning content •  Selection of activities based on learning

style is not (yet) an efficient approach, –  If you want to build style-based adaptation, use

more complex approaches

Page 47: Adaptive Learning Systems: A review of Adaptation

What is Usually Missed •  Learning objects are not necessary static files •  Most efficient learning “content” is interactive (might

not even look like content, stored in files, copied) –  Interactive simulations –  Worked-out examples –  Problems

•  This is exactly the place to apply “within-content” adaptation –  All kind of problem-solving support “tutors” –  All kinds of adaptive presentation such as adaptive

animation and examples •  There is a place for adaptation even beyond content

–  Adaptive collaboration support

Page 48: Adaptive Learning Systems: A review of Adaptation

Lessons Learned III •  Within-content adaptation is important

–  Adaptive presentation significantly increases comprehension while decreasing learning time

–  Provides vital problem-solving support where students needs most help

–  Engages learners in interactive activities •  There is no “single place” for adaptation

–  Every type of content might use different approaches for adaptation and use own appropriate “engine”

–  Different engines might need different information about learner and on different granularity levels

•  ITS is a great technology for content-level adaptation, but existing monolithic ITS should be re-engineered to fit the traditional learning architectures

Page 49: Adaptive Learning Systems: A review of Adaptation

Requirements for AL architecture

•  Support adaptation on several levels – Adaptive guidance (item to item) – Within-item adaptation – Adaptation beyond “items”, i.e., collaboration

•  Data for learner modeling should be collected from all kinds of interactions

•  Learner model produced from this data should be available for all components

Page 50: Adaptive Learning Systems: A review of Adaptation

ADAPT2 Architecture Portal

Activity Server

Student Modeling Server

Value-added Service

Brusilovsky, P. (2004) KnowledgeTree: A distributed architecture for adaptive e-learning. In: Proceedings of 13th International World Wide Web Conference, WWW 2004, New York, NY, 17-22 May, 2004, ACM Press,

Page 51: Adaptive Learning Systems: A review of Adaptation

User modeling server CUMULATE���

Event Storage

Inferenced UM

UM requests

Application ExternalInference Agent

InternalInference Agent

UM updates

Event reports

Event requests

Page 52: Adaptive Learning Systems: A review of Adaptation

All Pieces in Place?

Page 53: Adaptive Learning Systems: A review of Adaptation

Next Challenges: Architecture

•  Post-Web Learning technologies are more diverse, but we need to find how to fit them into the architecture

•  Educational games •  Virtual and Augmenter Reality •  Mobile learning •  “Real World” learning

Page 54: Adaptive Learning Systems: A review of Adaptation

Next Challenges: Adaptation

•  Most of existing adaptation technologies are based on knowledge engineering – Cognitive analysis – Metadata indexing

•  Works well, but expensive •  How we could use large volume of data

collected from many students to deliver and improve adaptation?

Page 55: Adaptive Learning Systems: A review of Adaptation

Social Personalization for AES

•  Starting with technologies based on shallow processing of social data

•  Social navigation support for open corpus resources – Knowledge Sea II

•  Open Social Student Modeling with Social guidance – Progressor – MasteryGrids

Page 56: Adaptive Learning Systems: A review of Adaptation

Knowledge Sea II

Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based group modeling. In: L. Ardissono, P. Brna and A. Mitrovic (eds.) Proceedings of 10th International User Modeling Conference, Berlin, July 24-29, 2005, Springer Verlag, pp. 463-472, also available at http://www2.sis.pitt.edu/~peterb/papers/FarzanBrusilovskyUM05.pdf.

Page 57: Adaptive Learning Systems: A review of Adaptation

Progressor

Hsiao, I. H., Bakalov, F., Brusilovsky, P., and König-Ries, B. (2013) Progressor: social navigation support through open social student modeling. New Review of Hypermedia and Multimedia 19 (2), 112-131.

Page 58: Adaptive Learning Systems: A review of Adaptation

MasteryGrids

Loboda, T., Guerra, J., Hosseini, R., and Brusilovsky, P. (2014) Mastery Grids: An Open Source Social Educational Progress Visualization. In: S. de Freitas, C. Rensing, P. J. Muñoz Merino and T. Ley (eds.) Proceedings of 9th European Conference on Technology Enhanced Learning (EC-TEL 2014), Graz, Austria, September 16-19, 2014.

Page 59: Adaptive Learning Systems: A review of Adaptation

The Challenge for Social Personalization

•  Use large volume of learner community data to build more advanced adaptation approaches to replace or enhance “content-based” adaptation

•  Example: Finding latent groups, meta-adaptation