investigating teachers' understanding of ims learning design: yes they can!

www.icoper.org

1 Oct 2010EC-TEL 2010 — Michael Derntl <[email protected]>

Investigating teachers’ understanding of IMS Learning

Design: Yes they can!

Michael Derntl1, Susanne Neumann1, Dai Griffiths2, Petra Oberhuemer1

1 University of Vienna, Austria 2 University of Bolton/CETIS, UK

License: CC-BY-SA

EC-TEL 2010, 1 Oct 2010, Barcelona, Spain

30 Sept 2010EC-TEL 2010 — Michael Derntl <[email protected]> 2

www.icoper.org

Context

ICOPER Best Practice Network – adoption, evaluation and recommendations for TEL standards and specs

Instructional modeling WP: Recommendations regarding the use of IMS LD

Various stakeholders addressed: standardization bodies, technology providers, HE managers, teachers

Here: Do teachers understand the specification’s structure and elements?


www.icoper.org

IMS Learning Design (LD)

IMS LD is a specification that enables the formal description of learning and teaching processes

A course or lesson described (authored) using IMS LD is called IMS LD unit of learning (UoL)

A UoL is a machine-readable package that can be played over and over with different people, technology, …

IMS LD is geared towards use in computer-managed environments


www.icoper.org

IMS LD Elements

IMS LD distinguishes Components: activity, activity structure, property, role, environment Method: act, role-part, condition

The method weaves the components into a process following a stage-play metaphor

Act 1 Act 2 Act n

Role-Part 1 Role-Part n

Role Activity Environment

Components

Method

Activity Structure

Role-Part 2

Property

Condition

Acts are executed one after the otherActs are executed

one after the other

All role-parts in the active act are concurrently

executed

All role-parts in the active act are concurrently

executed


www.icoper.org

Study setup

Workshops with teachers Introduce IMS LD elements + the “tool” to work with Show an example Give a task to solve

+ collect additional structured data


www.icoper.org

Study setup: participants

Total 21 participants from 10 European countries

Avg. ~10yrs teaching experience, 4/5 in HE

5

2

16

19

0% 25% 50% 75% 100%

IMS LDauthoring

IMS LDrunning

Yes

No


www.icoper.org

Study setup: The task

The following scenario is an online learning scenario. The instructor gives a presentation to learners via a presentation conferencing service about coal-burning power plants as well as their positive and negative side effects. After the presentation, learners can choose to do one of the two following activities: set up controversial questions regarding coal-burning power plants (to be

used during a later discussion round), or collect credible sources on the World Wide Web regarding new developments

in coal-burning power plants (to be used during a later discussion round).

The outputs learners created in the previous activities will be used during the subsequent discussion activity: The learners engage together with the instructor in the discussion using the questions and materials. After the discussion, learners rate their own agreement with the statement “Coal-burning power plants have a future in Europe’s energy production" (scale of 5, where 5 is the highest agreement). Learners, who rated themselves with a level of at least 3, will next do an activity involving the creation of a (digital) poster on one positive aspect of coal-burning power plants. Learners, who rated themselves at a level of 2 or lower, will next create a (digital) poster on one negative aspect of coal-burning power plants.


www.icoper.org

Study setup: the authoring tool


www.icoper.orgStudy setup: the tool

IDs

LEARNING ACTIVITYSUPPORT ACTIVITY

ID:

TITLE:

Description:

ACTIVITY

Properties to display: Properties to change/update: Environments to display:

IDsIDs IDs


ID:

TITLE:

Description:

ACTIVITY


IDsIDs

ID:

TITLE:

ENVIRONMENT

Learning objects: Services:

Chat

Forum

Mail

Announcement

ID:

TITLE:

ENVIRONMENT


ChatChat

ForumForum

MailMail

AnnouncementAnnouncement

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:ACTIVITY:

ACTIVITY STRUCTURE:ACTIVITY STRUCTURE:

ENVIRONMENT:ENVIRONMENT:

Sequence (activities to be executed one after another)

Selection – How many activities must be selected:

ID:

TITLE:

TYPE:

ACTIVITY STRUCTURE

ACTIVITIES: (separate referenced activity IDs with commas)

IDs

Sequence (activities to be executed one after another)Sequence (activities to be executed one after another)

Selection – How many activities must be selected: Selection – How many activities must be selected:

ID:

TITLE:

TYPE:

ACTIVITY STRUCTURE

ACTIVITIES: (separate referenced activity IDs with commas)

IDs

ID:

TITLE:

Description:

PROPERTY

Initial value:

ID:

TITLE:

Description:

PROPERTY

Initial value:

ID:

TITLE:

ROLESTAFFLEARNER

ID:

TITLE:

ROLESTAFFLEARNER

ID ID ID

ID ID ID

ID

C O N D I T I O NTITLE:

Show:

Hide:

Change Property:

IF: THEN:

ID ID IDID ID ID

ID ID IDID ID ID

ID


Show:Show:

Hide:Hide:

Change Property:Change Property:

IF: THEN:


www.icoper.org


www.icoper.org

Data analysis

Matching solutions with a correct prototype solution Decomposing solution into 60 actions that needed to be performed –

the “checklist” Two experts score each solution on the checklist


www.icoper.org


www.icoper.org

Results: action layer

95%

94%

94%

94%

92%

90%

90%

90%

89%

82%

81%

79%

73%

95%

86%

85%

80%

80%

69%

67%

0% 25% 50% 75% 100%

Activity Structure: Link activities *

Role Part: Assign target *

Role Part: Assign role *

Act: Role-part in correct act *

Activity: Create

Role: Create

Activity Structure: Create

Act: Only role-parts and cond.

Environment: Define services *

Activity: Link environment *

Environment: Create

Activity Structure: Select type *

Role Part: Create

Condition: Create

Property: Create

Condition: IF correct *

Condition: THEN correct *

Condition: ELSE correct *

Store output from activity *

Use input in activity *

Leve

l A(m

ean

= 8

8%)

Leve

l B(m

ean

= 8

0%)


www.icoper.org

Results: element layer


www.icoper.org

Results: element layer

92%

90%

88%

87%

87%

86%

85%

85%

0% 25% 50% 75% 100%

Act

Role

Activity Structure

Role-Part

Activity

Property

Condition

Environment


www.icoper.org

Results: components, method, UoL

87%

88%

0% 25% 50% 75% 100%

Components

Method

Calculation of overall quality of solutions Based on action layer, incl. dependent actions Average conformity 78% (± 20%, min=23.8%, max=100%)


www.icoper.org

Findings

Role-parts “Difficult” element, linking roles with other elements Particularly for jointly performed activities Tended to link only “dominant role” with activities

ID:

TITLE:

ROLESTAFFLEARNER

ID:

TITLE:

ROLESTAFFLEARNER

Instructor

R2

IDs


ID:

TITLE:

Description:

ACTIVITY


IDsIDs

Discussion

A4

Discuss the questions that learners have prepared in advance, and current developments in coal-burningpower plants. After the discussion rate your agreement.

P1, P2 E2P3

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:




R2A4

Discussion - Instructor

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:




R2A4

Discussion - Instructor


www.icoper.org

Act #

FindingsConditional activities Poster activities: role parts + conditions in method Some additional activity structures (selection)

IDs


ID:

TITLE:

Description:

ACTIVITY


IDsIDs

A5

Create poster on positive aspect

Create a digital poster about one positive aspect of coal-burning power plants.

IDs


ID:

TITLE:

Description:

ACTIVITY


IDsIDs

A6

Create poster on negative aspect

Create a digital poster about one negative aspect of coal-burning power plants.

ID:

TITLE:

Description:

PROPERTY

Initial value:

ID:

TITLE:

Description:

PROPERTY

Initial value:

P3

Agreement with statement on futureof coal-burning power plants

Each learner's agreement withstatement “Coal-burning power plants have a future in Europe’s energy production”.

0

ID ID ID

ID ID ID

ID


Show:

Hide:

Change Property:

IF: THEN:

ID ID IDID ID ID

ID ID IDID ID ID

ID


Show:Show:

Hide:Hide:

Change Property:Change Property:

IF: THEN:

Vote on coal-burning powerplant >= 3

If the valuein theproperty P3 is at least 3 or higher

A5

A6IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:



ENVIRONMENT:ENVIRONMENT:R1

A5

Learner -- poster positive aspect

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:




A5

Learner -- poster positive aspect

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:




A6

Learner -- poster negative aspect

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:




A6

Learner -- poster negative aspect

ID:

TITLE:

ROLESTAFFLEARNER

ID:

TITLE:

ROLESTAFFLEARNER

Learner

R1


www.icoper.org

FindingsActivity I/O Using properties to handle “activity flow”

ID:

TITLE:

Description:

PROPERTY

Initial value:

ID:

TITLE:

Description:

PROPERTY

Initial value:

P1

Discussion questions

This property contains the questions thatlearners created as input for thediscussion activity.

None

IDs


ID:

TITLE:

Description:

ACTIVITY


IDsIDs

A2

Set up controversial questions

Prepare controversial questions on coal-burning powerplant effects that will be used during the discussion.

IDs


ID:

TITLE:

Description:

ACTIVITY


IDsIDs

Discussion

A4

Discuss the questions that learners have prepared in advance, and current developments in coal-burningpower plants. After the discussion rate your agreement.

P1 P1


www.icoper.org

Findings

Referencing elements Activity – environment, Activity structure – activity Sometimes in components and method

IDs


ID:

TITLE:

Description:

ACTIVITY


IDsIDs

A1

Presentation on coal-burning power plants

E1

The instructor gives a presentation on coal-burningpower plants. Login to the presentation conferencingservice to participate.

ID:

TITLE:

ENVIRONMENT


Chat

Forum

Mail

Announcement

ID:

TITLE:

ENVIRONMENT


ChatChat

ForumForum

MailMail

AnnouncementAnnouncement

E1

Presentation Conferencing Service

Presentation Service(Presentation Slides)

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:




A1

Presentation - Instructor

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:




A1

Presentation - Instructor

IDID

IDID

ROLE-PARTTitle:

ROLE: ACTIVITY:

ACTIVITY STRUCTURE:

ENVIRONMENT:

IDID

IDID

ROLE-PARTTitle:



ENVIRONMENT:ENVIRONMENT:R2 E1

Presentation Env - Instructor


www.icoper.org

Findings

Issues protocol

Greatest troubles

Number of Nominations

3

2

1

3

1

1

2

1

8

4

5

2

3

2

1

1

0 2 4 6 8 10 12

Role-Parts

Conditions

Properties

Activity Structures

Environments

Acts

Activities

Learner Groups

Number Solved

Number Unsolved


www.icoper.org

Findings

Perceived troubles Environment (7 nominations), Property (5), Condition (4) Distinguishing elements:

activity structure vs act activity vs activity structure activity vs environment


www.icoper.org

Conclusions

Teachers can solve moderately complex design task with IMS LD after a 45-minute introduction

Some conceptual structures present obstacles to teachers (role parts, document flow, conditional activities)

Some call for a new specification (e.g. SLD 2.0) Others call for due diligence when juggling with standards


www.icoper.org

Recommendation

Keep IMS LD unchanged. Improve UI metaphors in tools for difficult concepts Provide templates to build on (e.g. design patterns)


www.icoper.org

Full report

ISURE: Report on usage and recommendations of specification for instructional models

ICOPER Deliverable D3.2, to go public in Dec 2010@ http://icoper.org/results/deliverables


www.icoper.org

Call for papers:

CD4TEL – Computational Design for Technology Enhanced Learning

at ICCSA 2011

Submission deadline: 31 Dec 2010

http://elearn.pri.univie.ac.at/cd4tel

Call for papers:

CD4TEL – Computational Design for Technology Enhanced Learning

at ICCSA 2011

Submission deadline: 31 Dec 2010

http://elearn.pri.univie.ac.at/cd4tel

investigating teachers' understanding of ims learning design: yes they can!

Education