Demo Navigation Service (WP7)

Hans Hummel

Open University of the NetherlandsTC meeting Bolton (Jan 8, 2007)

Aims & planning

(TC Task 7.5; started June 1, 2006; lead by OUNL)• Aim: Delivering navigation service to advise next most suitable learning activity (and –path).• Status: Currently Phase 1 (proof of concept) June 2006-June 2007, including pilot at OUNL with CMS (Moodle) and service designed in PHP / MySQL.• Techniques: Predictions based on combination of collaborative filtering (matrix with finished learning activities by

other learners) and group profiles (motivation, time, interest)

• Testing: First TC pilot (!) from Oct 2, 2006 to Jan 24, 2007; started with 66 students (2x 33), currently around 160 (2x 80) students registered

• Test and evaluate the effect of a personalized recommendation service on learning behavior in learning networks

• The group with the personalized advise is expected to become more efficient, with possible indicators being:

study time (faster), results on grades (higher, faster), completion (more, faster)

Focus (one at a time) or variety (less), satisfaction (more)

Experimental design (1)

Experimental design (2)

Exp. Group

100 students

Using indirect social navigation

(collaborative filtering information) and profile

information (time, motive, interest)

to personalizerecommendation for

next ‘course’

Personalized advise

Control Group

100 students

Randomized list of ‘courses’

No advise

Effectiveness

Efficiency

Satisfaction

Efficiency and Effectiveness:• Effectiveness: Students with PRS will be more satisfied with

learning, will participate more, will get closer to completion, will get higher progress test and exam results

• Efficiency: Students with PRS will achieve the same test results and completions in less time

Variety:• Students with PRS will persevere more to finish one course at a

time, because they study content personalized to their profile• For students with PRS, the learning path will be more ‘linear’.

Assumptions

Planning Phase 1

October 06Maintain

Experiment

November 06

Data CollectionExperiment

December 06 Data Collection

Experiment

January 07 First Results

Proof of Concept

February 07 Results Proof

of Concept

March 07 Service

Development

April 07 Service

Development

May 07 Testing

June 07 First release

recommend course based on

MATRIX-> A | B | C

[available]

PRS Advise

[not available]

Check if course is part of matrix A

[not part of A]

Remove study timecheck if part of matrix B

[yes]

recommend course based on direct

subdomain interest

use personal interest in subdomain;

courses unfinished?

Get recommendation

Already finished course by user available?

Remove study motivecheck if part of matrix C

[not part of B] [not part of C]

recommend coursebased on indirect

subdomain interest[topdomain]

[no][yes] [yes] [yes]

Recommendation created

with a first prediction strategy

PROVIDE INFORMATION ABOUT PRIOR KNOWLEDGE

UPLOAD DOCUMENTS

Documentation of the recommendation service (Version: 11/23/06):

Interface / object layer: The recommendation service is clustered into two main layers. The basic communication functions are located in the interface layer. The interface layer is responsible for getting all required data from connected services and profiles.On top of this layer is the object layer located. The object layer is in charge of creating a reasonable profile for the requesting learner. This contains a collection of required profile information and suitable learner groups for recommendation.

Profile class: The Profile class is like the Position class part of the interface layer between the recommendationEngine and the connected objects in 10CC domain model. It is responsible for getting required data regarding for the profile of the learner. The function getProcessData(learnerID) submit a learner id to the actor class and returns a list called profileData. The list profileData contains required information for the recommendationEngine from the actor class.The functiongetStudiedLearningActivities(learnerID) submit a learner id to the process Log class and returns a list with finished learning activities. Already completed learning activities will be stored in the listOfExcludedNodes which is part of the learner class.

Position class: The Position class is like the Profile class part of the interface layer between the recommendationEngine and connected objects in 10CC domain model. It is responsible for getting the current position of the learner regarding to the LN. It works as an interface between the positioning service and the recommendation service. It provides data for recommendationEngine that is taking into account for the recommendations.

The function getPosition(learnerID,learningGoal) will submit a learner id and a learning goal to the positioning service. It returns a list of nodes that are stored in the array listofExcludedNode which is part of the learner class. Additional to this list could the positioning service provide some metadata. This metadata information is stored in the array listOfMetaData in the Learner class.In the ISIS pilot is no positioning class implemented, because no positioning service was available.

Learner: The Learner class gathers all required data about learners. It is used for creating a character for the requesting learner as well as input for the learner group class. It requires the Profile and Position class from the interface layer to get required data for the requesting learner. The result of the Position class could be null, but any feedback from this class is required as input for the recommendation strategy. The Profile class delivers always information to the learner class.The function getPosition(learnerId) is requesting the getPosition(learnerId) function in the Profile class to get data. The same happens with the getProfile() function. Both functions of this class provide required parameter like learnerID or a learningGoal to the classes of the interface layer.

CurrentLearner: The CurrentLearner class is an instance of the learner class. It represents the requesting learner and provides all information for this learner for the recommendationEngine.

Learner Group: The LearnerGroup class generates a list of helpful learners if possible. It gathers all required data about helpful learners regarding to the current learner. It uses the learner class to select matching learners and provide a list with people to the RecommendationEngine.The function lookForHelpfulLearners() checks the LN for learners with similar learning goals, interest or reason to study (studytime, studymotivation). The cardinality relationship describe that there could be no or more than one suitable LearnerGroup available for the current learner. Recommendation Engine: The RecommendationEngine is the heart of the recommendation service.It calculates which recommendation technique is suitable for the current learner. Therefore a specific recommendation strategy is implemented. This strategy decides which prediction technique is most suitable to cover the needs of the current learner. For a recommendation it takes into account the data given by CurrentLearner and the LearnerGroup.The functions getCurrentLearner() uses the CurrentLearner class as a starting point for the recommendation strategy. The function getSuitableLearnerGroup() look for learners with similar profile and already done learning activities. The function chooseRecommendationStrategy() uses the recommendation strategy to come up with a prediction for the current learner. The function priory’s suitable learning activities regarding to the recommendation strategy.

The function provideListOfRecommendations() delivers the output of the recommendationEngine to the learner.

The experimental Group finish more Courses.

1 614 19

2936

4354

69 7387

2 617

2936

44

6675

8495

120

0

20

40

60

80

100

120

140

time checkpoints

Com

plet

ed C

ours

es(G

roup

Lev

el)

control group 1 6 14 19 29 36 43 54 69 73 87

experimental group 2 6 17 29 36 44 66 75 84 95 120

06.10

13.10

20.10

27.10

03.11

10.11

17.11

24.11

01.12

08.12

15.12

Individuals in the Experimental Group finish more Courses successful.

0,00

0,50

1,00

1,50

2,00

2,50

time checkpoints

Avg

of c

ompl

eted

C

ours

es p

er L

earn

er(In

divi

dual

Lev

el)

control group 0,10 0,40 0,60 0,80 1,10 1,20 1,30 1,50 1,90 1,80 1,80

experimental group 0,20 0,40 0,80 1,20 1,30 1,40 1,80 2,00 2,00 2,20 2,00

06.10

13.10

20.10

27.10

03.11

10.11

17.11

24.11

01.12

08.12

15.12

The Experimental Group need less Studytime to finish Courses successful.

0

50

100

150

200

250

time checkpoints

Avg

of

Tim

e(G

rou

p L

evel

)

control group 67,5 143,4 163,2 165 195,4 202,7 211,7 194,2 185 186,4 196,5

experimental group 129,2 129,2 158,8 143,1 150,8 162,5 170,2 172 173,2 178,4 185,3

06.10 13.10 20.10 27.10 03.11 10.11 17.11 24.11 01.12 08.12 15.12

Individuals in the Experimental Group need less Studytime to finish Courses successful.

0,00

2,00

4,00

6,00

8,00

10,00

12,00

time checkpoints

Avg

of T

ime

per

Cou

rse

per L

earn

er(In

divi

dual

Lev

el)

control group 5,60 9 6,5 6,6 7,5 6,5 6,4 5,5 5 4,7 4,5

experimental group 10,80 8,1 7,6 6 5,4 5,2 4,7 4,5 4,2 4,1 3,8

06.10

13.10

20.10

27.10

03.11

10.11

17.11

24.11

01.12

08.12

15.12

Usage of Recommendation Technique

3345

71

9097

113

131 137 143151

164

0 0 2 5 7 12

3038

4759

77

0

20

40

60

80

100

120

140

160

180

time checkpoints

ontology 33 45 71 90 97 113 131 137 143 151 164

collaborative f iltering 0 0 2 5 7 12 30 38 47 59 77

06.10 13.10 20.10 27.10 03.11 10.11 17.11 24.11 01.12 08.12 15.12

Total Amount of Starters per Week

1216

25 25 26

3133

3537

40 41

1216

2124

2831

3638

4144

47

0

5

10

15

20

25

30

35

40

45

50

time checkpoints

control group 12 16 25 25 26 31 33 35 37 40 41

experimental group 12 16 21 24 28 31 36 38 41 44 47

06.10 13.10 20.10 27.10 03.11 10.11 17.11 24.11 01.12 08.12 15.12

7.5: Risks

• Proof of concept delivers no useable results

• Collaborative filtering is not properly working because of little students

• Discuss interface definition between the navigation and other services, esp. positioning• Discuss SOA design of a navigation service

Joint publications on:- Visualisation of Navigation (courses and paths) in Learning Networks (with UPF)- Profiling of Learners, integration with positioning service in Learning Networks (with L3S)

7.5: Collaboration

Links

http://elo.ou.nl(hah + OUPW)

http://moodle.psychologie.ou.nl/moodle/(demo + de&MO$)

http://dspace.ou.nl/handle/1820/830(draft article on combined approach)