tele report version 7 - lmi.ub.es report report on technology enhanced learning environments (teles)...

TELE Report

Report onTechnology Enhanced Learning Environments (TELEs)

that Support Self-Regulated Learning

Version 7.0

June 30, 2005

Universität zu Köln / Germany,Vrije Universiteit Amsterdam / The Netherlands,Universidade Católica Portuguesa / Portugal,Aalborg University / Denmark,CNR Istituto per le Tecnologie Didattiche / Italy,Nottingham Trent University / U.K.,Universitat de Barcelona / Spain ,Université de Technologie de Compiegne / France,Universitetet i Bergen / Norway.

© The TELEPEERS consortiumThis report was compiled within the project “Self-regulated Learning in Technology EnhancedLearning Environments at University Level: a Peer Review (TELEPEERS)”. The project is beingcarried out with the support of the European Commission (Grant agreement 2003-4710-/001-001EDU-ELEARN). The content of this project does not necessarily reflect the position of the EuropeanCommission, nor does it involve any responsibility on the part of the European Commission.

TELE Report

2

1. Introduction 3

2. Container Systems With Tutor Support Group 1

2.1 Digital Portfolio – Keeping online personal repositories Vrije Universiteit Amsterdam 62.2 DiViDU – Streaming video support for reflection Vrije Universiteit Amsterdam 92.3 ILIAS – A Learning Management System Universität zu Köln 132.4 Weblogs – Personal knowledge publishing Universitetet i Bergen 16

3. Content Systems With Tutor Support Group 2

3.1 Advanced Teacher Education in Informatics – ICT-based Teacher Trainin program Universidade Católica Portuguesa 193.2 Cognitive psychology course – Enhanced web based support Nottingham Trent University 233.3 Digital Video Course – Blended Learning with new media Universitat de Barcelona 273.4 Teacher training course on ET – Using a CMCS in teacher training CNR Istituto per le Tecnologie Didattiche 29

4. Content Systems Without Tutor Support Group 3

4.1 Sunpower – CD-ROM to train business English Universität zu Köln 334.2 Databases – A model for online studies Université de Technologie de Compiegne 374.3 Programming Tutorials – Additional online exercises Université de Technologie de Compiegne 404.4 SWIM – Online tutorial for information seeking Aalborg University 43

5. Concluding remarks

5.1 Complexity of the task 465.2 Consindering learners, teachers and other protagonists 475.3 Evaluating the Peer Review Evaluation Tool 505.4 Outlook 52

6. List of contributors 53

Annex 1: Peer Review Evaluation Tool (PRET) 54

Annex 2: Summary of quantitative RET results 64

TELE Report

3

1. Introduction

During its first year (January 1, 2004 to December 31, 2004) the major objective ofthe TELEPEERS project was to identify Technology Enhanced LearningEnvironments (TELEs) that support self-regulated learning. To achieve this objective,we developed a Peer Review Evaluation Tool (PRET) which was to help us inevaluating to which extent the TELEs that we were to identify actually did supportself-regulated learning. At the same time, we selected TELEs that looked promisingwith regard to their capacity to foster self-regulated learning.

The development of the Peer Review Evaluation Tool (PRET) was based on athorough review of the literature on self-regulated learning and on cognitive and non-cognitive aspects of learning processes. At several project meetings and workshops,the concept of the Peer Review Evaluation Tool was discussed in depth which leadto several revisions of the PRET.

In the process of preparing and carrying out the peer reviews, it became clear thatevaluating the TELEs was a multi-faceted task and that using the Peer ReviewEvaluation Tool, we could not embrace all the facets. As peer reviewers, weconsidered ourselves experts on TELEs, but we could only try to assess the capacityof each TELE for supporting self-regulated learning, i.e. we evaluated the TELEs aslearning objects. We could not assess to what extent the TELEs would indeed helpstudents to self-regulate their learning. This aspect we intend to clarify using anevaluation form for students (TELESTUDENTS). We have been developing thisstudent form and are presently giving it to students of our TELEs.

In developing the evaluation tool, it was important to us to take into considerationcross-cultural, contextual issues. In our opinion, there seems to be a delicateinterdependency between wanting a tool that is general enough to work acrosstechnologies and settings - to facilitate dissemination - and subscribing to aconstructivist view that claims use to also be situated. To work out a balancebetween being concrete and descriptive, and general and conclusive is a challenge.

The questionnaire form that we developed is easy to apply and will yield a largeamount of information, but a narrative approach might be more appropriate todiscover more subtle individual and cultural differences. We will therefore also bearthis approach in mind.

Basically, the PRET consists of three parts:Part A: Technical description of TELE,Part B: Detailed evaluation of support for self-regulated learning,Part C: Global evaluation of support for self-regulated learning.

While Part A was used to describe the TELE in question with respect to technicalaspects, PART B served as a grid to evaluate in detail the capacity of the TELE tosupport self-regulated learning. It consists of 43 statements that relate to the threestages of a learning process:

(1) Planning,(2) Executing and monitoring and(3) Evaluating.

TELE Report

4

For each of the phases, we distinguished between(a) cognitive aspects,(b) motivational aspects,(c) emotional aspects and(d) social aspects.

The evaluators were asked to indicate on a six-point scale to what extent theythought that self-regulated learning was supported by the TELE in question.

In the last part (PART C), a summary was made of the responses to the single items.The reviewer was also encouraged to comment on the TELE in a more open wayand to identify factors that contribute to the strength of the TELE as well as factorsthat represent weaknesses of the TELE, and to make suggestions for improving theTELE.

The version of the PRET used in our studies is attached as Annex 1. Updatedversions will be found on the TACONET website (http://www.lmi.ub.es/taconet).

The concept of Technology Enhanced Learning Environments is a rather wide one. ACD-ROM with a programme to learn a foreign language which is used by individualsin their spare time is a TELE. If the same CD-ROM is used in a course at school,university or other educational institution, then the whole course is the TELE. A TELEshould therefore not be reduced to its technological component, rather, it is thecomplete learning environment.

The category system that we suggest (see above) is one that applies to the TELE asthey were evaluated in the project. In other contexts, they may be used in a differentway. For example, Sunpower is a multimedia program on CD-ROM which wasevaluated as a stand-alone TELE (content system without a tutor). But it is also easyto imagine it being used in class, with a teacher or tutor. Likewise, we evaluated ateacher training course on Educational Technologies that used FirstClass as a

With Tutor Support Origin Without TutorSupport

Origin

Containersystems

Group 1

Digital PortfolioDiViDU (reflection/assessment module)ILIASWeblogs

AmsterdamAmsterdam

CologneBergen

Contentsystems

Group 2

Advanced TeacherEducation in InformaticsCognitive psychologycourseDigital Video CourseTeacher Training Courseon EducationalTechnologies

Lisbon

Nottingham

BarcelonaGenoa

Group 3

SunpowerDatabasesProgrammingTutorialsSWIM

CologneCompiègneCompiègne

Aalborg

TELE Report

5

content system with a tutor. It might also be used as a container system which is thenfilled with content.

The TELEs in group1 are basically Computer Mediated Communication or LearningManagement Systems (CMCS or LMS). Group 2 contains four courses that relysignificantly on technological components. Group 3 contains four computerprograms, most of them Internet-based, which can be used by individual studentswithout the help of a tutor. In the following sections, a detailed description of theseTELEs as well as their evaluations will be given.

Evaluations of the TELEs were conducted in a peer review manner. Each TELE wasreviewed by one expert in the field of TELEs at the institution where it wasimplemented and used, and by at least one expert at one of the partner institutions.Most of the evaluations were carried out in October and November 2004 and theresults were presented and discussed at the TACONET conference in Barcelona(Nov.20, 2004).

Category system for TELEs that were peer reviewed in the TELEPEERS project

2. Container Systems With Tutor Support. Group 1 peer reviewed by

2.1 Digital Portfolio Nottingham Trent Unviersity2.2 DiViDU (reflection/assessment module) Universität zu Köln2.3 ILIAS Université de Technologie de

Compiègne2.4 Weblogs Vrije Unviersiteit Amsterdam,

Aalborg Unviersity

3. Content Systems With Tutor Support. Group 2 peer reviewed by

3.1 Advanced Teacher Education in Informatics Universitat de BarcelonaCNR Istituto per la TecnologieDidattiche

3.2 Cognitive psychology course Vrije Unviersiteit AmsterdamIstituto per la TecnologieDittatiche

3.3 Digital Video Course Universidade CatólicaPortuguesa

3.4 Teacher training course on ET Universidade CatólicaPortuguesa,Universitat de Barcelona

4. Content Systems Without Tutor Support. Group 3 peer reviewed by

4.1 Sunpower Nottingham Trent University,Université de Technologie deCompiègne

4.2 Databases Universität zu Köln4.3 Programming Tutorials Universität zu Köln4.4 SWIM Universitetet i Bergen

TELE Report

6

2. Container Systems With Tutor Support. Group 1

2 .1 D ig i ta l Por t fo l io – Keep ing on l ine persona l repos i tor iesVrije Universiteit Amsterdam

An increasing number of educational institutions make use of portfolios. A portfolio isa purposeful collection of student work that exhibits the student’s efforts, progressand achievement in one or more areas (Tromp, 2003). Although it is a collection ofdocuments, a portfolio is tangible evidence of the wide range of knowledge,dispositions and skills that a student possesses as a growing professional. Thefunction which is eventually assigned to the portfolio is closely linked to theeducational concept existing within the institution and the way this is implemented atcurriculum level (Ritzen, 2001).

Brown Daniels (2002) describes the three most common types of portfolios used ineducation:- the working portfolio, which contains projects the student is currently working on

or has recently completed.- the display portfolio, which showcases samples of the student's best work.- the assessment portfolio, which presents work demonstrating that the student has

met specific learning goals and requirements.

2.1.1 Example Case study of an Assessment Portfolio

The faculty of Human Movement Sciences of the Vrije Universiteit Amsterdamintroduced the use of portfolios in their one year teacher training programme. Thiscompetence-based program assesses students on the basis of achievedcompetencies. The student is responsible for his/her own learning process and usesthe portfolio as a tool to analyse competencies, store achievements, demonstrateskills, discuss achievements with the teacher and peers etc. At the start of thecourse, the student describes his/her starting level for each competency (based onprevious professional experiences e.g. placements, teaching jobs etc.). On the basisof this, the student explains how he/she will acquire the required competencies of thecourse. In a meeting with his/her coach group (four other students and a coach)his/her learning aims and plans are discussed. The group meets halfway through thecourse to discuss each other’s progress.

At the end of the course, an assessment takes place to establish whether the studentis sufficiently competent. The student uses the portfolio to publish products he/shehas made during the year (videos of classroom teaching, essays, presentations,posters etc) and to demonstrate he/she has achieved the required level ofcompetency.

2.1.2 Software and implementation

In the above case study, the electronic portfolio system of the Digital University wasused. In this system, each student has exclusive rights to manage his own portfolio.Other people, including peers, tutors, assessors and teachers cannot access theportfolio, unless they are invited by the owner of the portfolio. This software will nolonger be supported by the Digital University from the summer of 2005.

TELE Report

7

In general, a portfolio can be implemented using generic tools (e.g. HTML, Word incombination with learning environments such as Blackboard or collaborativeenvironments such as ProjectPlace) or customized portfolio systems (Folio, Concord,Blackboard Content System). Gibson and Barrett (2003) define criteria for choosingbetween these two directions. These include among others:

- the amount of creativity a user should have in building and structuring his/herportfolio (easier with standard applications) versus a preference to standardiseportfolio usage (easier with purpose-built portfolio systems).

- the importance of having communication and collaboration functionality available(often the case with purpose-built portfolio systems).

- financial considerations (purpose-built systems are more expensive).

The success of introducing a digital portfolio does not only depend on the structureand content of a portfolio or the type of software that is used. A portfolio has to begiven a structural place in the curriculum: its purpose has to be clear to students andperceived as being worthwhile. Teachers may find it time consuming to work withportfolios: they also need time and guidance to develop appropriate mechanisms forusing them effectively in their teaching.

For more information on the use of portfolios at the Vrije Universiteit Amsterdam:[email protected]

2.1.3 Evaluation within the TELEPEERS Project

This concept was peer-reviewed in the European TELEPEERS project using thePeer Review Evaluation Tool (PRET). Portfolios score very highly on all aspects inthe cycle of self-regulation:• they allow for explicit reflection over a period of time on the development of one’s

own knowledge, competencies and monitoring of progress in achieving requiredcompetencies.

• they enable and encourage the use of peer assessment.• the student is in charge of the portfolio:- by choosing the way he/she uses the portfolio to demonstrate progress- by allowing access to peers, tutor and other parties.

2.1.4 Bibliography

Brown Daniels, M. (2002) Electronic Portfolios in the K-12 Classroom. EducationWorld [Online]. Available: http://www.educationworld.com/a_tech/tech/tech111.shtml

Gibson, D. & Barrett, H. (2003). Directions in Electronic Portfolio Development.Contemporary Issues in Technology and Teacher Education, [Online serial], 2(4).Available: http://www.citejournal.org/vol2/iss4/general/article3.cfm

Ritzen, M, and Kosters J. (2001) Combining Different Aims in a Portfolio System: AWebbased Portfolio and the Various Ways in which it can serve the Student. Paperpresented at Ed-Media 2001 conference, Tampere, Finland, June 2001.Available: http://www.efa.nl/publicaties/teksten/finlandtrans.html.

TELE Report

8

Tromp Meesters, M. and van Drunen, M. (2003) Study Manual Module 6 - PortfolioERM. Institute for Environmental Studies. Vrije Universiteit Amsterdam (internaldocument).

2.1.5 Peer review results

Comments on Part C of PRET or on PRET in generalNottingham Trent UniversityThe Digital Portfolio scores high on all dimensions of the PRET. It encourages self-regulated learningand allows students to self-monitor their work as well as seeking feedback from tutors and peers. It isrelatively easy to use and this might well encourage use even by students who are not very technicallyproficient.

Factors that contribute to the strength of the TELEVrije Universiteit Amsterdam- The TELE scores very highly on all aspects in the cycle of self-regulation: it allows for explicit

reflection over a period of time on the development of own knowledge, skills and attitudes, andmonitoring of progress in achieving required competencies.

- It can be customized for use in different learning contexts and different disciplines.- It enables and encourages the use of peer assessment.- The student is in charge:

- by choosing the way he uses the TELE to demonstrate progress- by allowing access to peers, tutor and other parties.

- Easy to use technology.- With this particular case study: there is a clear purpose to the use and instructions on its usage.- Nottingham Trent UniversityIt is a course requirement to complete the programEasy to useAbility to upload a wide range of file formatsAbility to select items to displayAbility to give viewing rights to selected usersEasy to obtain feedback from tutors and peerswww based

Factors that represent weaknesses of the TELE Vrije Universiteit Amsterdam- In general: a portfolio has to be given a structural place in the curriculum; its purpose has to be

clear to students and perceived as being worthwhile. Students may find it time consuming.- This TELE could not be integrated with other systems e.g. learning content systems; student

grade systems etc.- This particular TELE will not be developed further; it did not have a viable future according to the

Dutch (higher) educational community.

Nottingham Trent University Encourages extensive writing rather than a concise style Requires relatively high maintenance by tutors

Suggestions for improving the TELEVrije Universiteit AmsterdamWith this particular system; the way to give access to other users could be improved

Nottingham Trent UniversityMore attractive interfaceVisible rating / importance scales

TELE Report

9

2.2 DiViDU Streaming video support for reflectionVrije Universiteit Amsterdam

The crucial role of reflection in the training of professionals is becoming more andmore recognised. Professionals gain competencies in a life-long process of learningin authentic contexts. To be able to learn from experience in (often complex) practicalsettings, systematic reflection on one’s experiences and actions is necessary. Thedevelopment of reflective skills to aid the process of self-regulation and self-monitoring is therefore a professional competency that is required of futureprofessionals.

DiViDU is an internet application based on the use of (streaming) video to teachstudents to reflect on their skills and professional development. Although the systemhas so far mostly been tested and used in teacher training and dentistry, it wasdeveloped on the basis of generic didactical models which may be used to designdigital learning environments for different types of professional training, such asmedicine, dentistry, physiotherapy or social work.

The DiViDU system allows students to watch video clips of typical professionalsituations, record observations, exchange views and discuss alternative approaches.With easy-to-use video recording equipment, students record their own professionalexperiences (e.g. a dentist in training dealing with a nervous patient) and give themto peers or teacher trainers to comment on.

DiViDU can be used by teachers both for classroom and distance teaching. They canuse available learning activities, edit learning activities to suit their specific goals ordevelop new learning activities. They may use the video clips that are available in thevideo catalogue, upload their own clips, or ask permission to use video clips providedby students. Additional study materials and sources can also be included.

The software is structured around three modules, serving different educationalobjectives and used at different stages of professional training:

• Analysis (to learn to analyse professional situations)• Reflection (to learn to reflect on one’s own behaviour in a professional context)• Assessment (to learn to assess one’s own professional competencies).

2.2.1 Software and implementation

The internet-based DiViDU system contains a video archive of professional situationsand allows for the storage of students’ own recordings of their professionalexperiences. More importantly, group discussions on the content of the videos arealso stored in the DiViDU system, so that students can read and reread thecomments of peers and lecturers on their performances.

TELE Report

10

DiViDU was developed in 2003 by the University of Amsterdam, de EducatieveFaculteit Amsterdam, the Academisch Centrum Tandheelkunde Amsterdam, the VrijeUniversiteit Amsterdam and the University of Twente as part of a Digital University-funded project. The Digital University is a consortium of ten universities in TheNetherlands. It focuses on the development and application of digital educationalproducts and knowledge for higher education.

The system is still being developed and further tested in a variety of educationalscenarios in the academic year 2004-2005. At the moment, an English version of thesoftware is being developed. Interest in such a version may be expressed to thecurrent project leader of DiViDU (Judith Janssen) at [email protected]. Moreinformation can also be obtained via [email protected]

2.2.2 Evaluation within TELEPEERS project

This system was peer-reviewed in the European TELEPEERS project using the PeerReview Evaluation Tool (PRET). It was evaluated as a system which supports self-regulated learning to a high degree, in particular its reflection module. It enables acyclical approach of the process of reflection (based on Korthagen’s reflection modelwith additions from Fortkamp and Schon).

Cyclical model of Korthagen (2002), with additions from Fortkamp/Schon

Design principles for activities within the reflection model include:

- The subjects of reflection are an individual’s actions and behaviours in practicalsituations.

- Reflection is not an individual activity; feedback and discussion with peers iscrucial.

- Reflective practice contains learning activities from Fortkamp’s five phases.

2.2.3 Educational benefits offered by DiViDU

‘indeterminate zone ofprofessional practice’

Recording experiences with video:- video of own practice- video of other students

Exchange of :- experiences- opinions

TELE Report

11

- the purpose of the software: help students to develop reflective and self-regulating skills

- backed by strong theoretical framework on reflection- innovative use of ICT (streaming video)- potential for use in different learning contexts- suitable to reflect on different types of skills in a range of disciplines- suitable to be used in different educational settings and delivery methods: face-to-

face education, distance learning, blended learning- enables and encourages collaborative feedback- allows for storage and re-use of typical professional situations- tutor development module: assignments can be easily changed.

2.2.4 Bibliography

• Bimmel P. and Janssen J., ed. (2004) DiViDU, Digitale Video voor DigitaleUniversiteit. Onderwijskundige uitgangspunten van de 3 modellen.Ontwerpdocument als output van Werkpaketten 1,2 en 6. Intern documentDigitale Universiteit.

• Fortkamp, J. (2002). Leren reflecteren in beeld. Literatuuronderzoek MUST,Universiteit Twente. Online:http://www.edith.nl/web/lectoraat/assets/reflectie%20in%20beeld.pdf

• Korthagen, F. (2001) Linking Practice and Theory: The Pedagogy of RealisticTeacher Education. Paper presented at the Annual meeting of the AmericanEducational Research Association. Available:http://educ.queensu.ca/~ar/aera2001/Korthagen2001.pdf


Factors that contribute to the strength of the TELEVrije Universiteit AmsterdamTime and place independentVideo clips show day to day practice; to tell you need a lot of words!!Tutor development module; assignments can easily be changedStorage and re-use of typical practical situations (online video catalogue)Storage and re-use of assignments and modules (online modules of different teachers)We show (in analysis) authentic (class-room) situationsNot “one solution”; good examples of active learning of critical moments,different repertoires.

Universität zu KölnFeedback by tutor and peersImprovement is encouraged and monitored (reflection module)Students can choose what video clip he/she wants to use to demonstrate specific aspects of his/herperformance.

Factors that represent weaknesses of the TELEVrije Universiteit AmsterdamDifferent players, real one is necessaryInstructions for making video clipsInterface beta versionChallenge students’ ICT skills

Universität zu KölnTechnical quality of videos (in shooting and viewing)

TELE Report

12

Suggestions for improving the TELEVrije Universiteit AmsterdamMake interface more student friendlyDevelopment of learning tasksFind other partners/disciplines for useBetter integration in curriculum

Universität zu KölnImprove quality of videos

TELE Report

13

2 . 3 I L I A S – A L e a r n i n g M a g a g e m e n t S y s t e mUniversität zu Köln

ILIAS - An Integrated System for Learning, Information Seeking andCooperative Working(Integriertes Lern-, Informations- und Arbeitskooperationssystem)

ILIAS was developed at the University of Cologne in the framework of the VIRTUSproject. The VIRTUS project of the Department of Business Administrations,Economics and Social Sciences at the University of Cologne (1997 – 2001) aimed atthe development of virtual university systems (see http://www.virtus.uni-koeln.de ). Itwas intended to supplement traditional learning environments in higher educationwith a web-based working environment - the ILIAS learning platform. Aftertermination of the project, it was decided to make ILIAS an open source system.ILIAS open source was supported by the Bertelsmann Foundation, the Heinz NixdorfFoundation, the Sal. Oppenheim Foundation and the Department of Science andResearch of the State of Northrhine-Westphalia.

ILIAS is an Internet-based authoring environment for course designers as well as anInternet-based learning environment for students at university level (seehttp://www.ilias.uni-koeln.de/ilias3/start.php). As such it qualifies as a ComputerMediated Communication System (CMCS) as well as a Learning ManagementSystem (LMS). The system was developed with the explicit goal of enhancing self-regulated and collaborative learning. The authors believe that these skills willbecome increasingly important in the future, particularly in the context of life-longlearning (which demands high skills in self-regulated learning) and virtual enterprises(which demand high cooperative working skills).

As a technology enhanced learning environment (TELE), ILIAS supportscollaborative work and communication, as well as monitoring and controlling one’sown learning. The system admits four groups of users:

• Learners who make use of the learning material available in the system,• Guests who may access a limited array of learning materials and

functionalities,• Authors who create course content (learning units) and• Administrators who are responsible for maintaining the system.

2.3.1 Personal desktop

ILIAS supports self-regulated learning by helping the individual student to organisehis/her learning activities. The TELE provides the student with a personal-desktop

where he/she can create a personal profile and find and up to date informationon courses which he/she is attending. It also allows him/her to create and organisehis/her own notes, to mark specific dates in the calendar and to store and organisebookmarks.

2.3.2 Repository

The repository allows the student to access and exchange information in theframework of ILIAS as well as to communicate with their peers and with coursetutors.

TELE Report

14

Clicking on the Course option, the student will be given an overview of the coursesthat are available in ILIAS. After having registered for a specific course he/she willhave access to all the information and learning material belonging to that course.

Self-regulated learning is enhanced by working through Learning Modules and bydoing specific exercises. For self-assessment, the student is given the opportunity totake tests on the learning material he/she has been working with (with tutorfeedback).

Communication and collaborative learning is facilitated by giving students theopportunity to take part in chats and topic-centred discussions forums. The systemalso allows for creating and managing virtual groups to exchange ideas andthoughts.

2.3.3 Search

Searching for information is encouraged and facilitated by an elaborate help system.

2.3.4 Mail

The mail system allows the student to communicate from within the ILIAS system viae-mail, particularly with his/her peers and students. He thus does not have to leavehis/her working and learning environment when he/she wants to contact one ofhis/her fellow students or one of the tutors.

2.3.5 Characteristics

ILIAS allows users to create, edit and publish learning and teaching material in anintegrated system with their normal web browsers. Tools for cooperative working andcommunication are included as well. The current version of ILIAS offers the followingfeatures (see http://www.ilias.uni-koeln.de/ilias3/start.php):

• Personal desktop for each user with information about last visited courses,new mail or forum entries

• Learning environment with personal annotations, test, glossary, print function,search engine and download

• SCORM 1.2 and AICC compliance• Course management system• Communication features like mail system, forums and chat• Group system for collaborative work and organising users and resources• Integrated authoring environment (Editor) to create courses even without any

HTML knowledge• Support of metadata for all levels of learning objects• Context-sensitive help system for learners and authors• User and system administration interface

2.3.6 Availability

ILIAS is an Open Sourse Code system. It may be downloaded for free athttp://www.ilias.uni-koeln.de

TELE Report

15

System languages are: Chinese, Czech, Danish, Dutch, English, French, German,Greek, Indonesian, Italian, Lithuanian, Norwegian, Polish, Portuguese, Russian,Spanish, Swedish and Ukrainian

2.3.7 Peer Review

Factors that contribute to the strength of the TELEUniversität zu KölnThe TELE allows for feedback by peers and tutorsCommunication with peers and students is well facilitatedThere are a number of feedback mechanisms that help students to monitor their learningThe TELE supports collaborative learning and workingThe TELE is open source and therefore free of charge

Factors that represent weaknesses of the TELEUniversität zu KölnNavigation takes some time to get accustomed toMotivational and emotional factors are not explicitly taken into consideration

Suggestions for improving the TELEUniversität zu KölnA path diagram to explain the structure of the program might be helpful

The Université de Technologie de Compiègne had agreed to conduct a peer reviewof ILIAS. Due to unsurmountable difficulties that were encountered when trying toinstall ILIAS, an evaluation at Compiègne was not possible.

TELE Report

16

2.4 Weblogs- Personal knowledge publ ishing Universitetet i Bergen

Web-logs (or blogs for short) are personal, but public, web spaces for selfexpression. Web logs are often seen as online diaries and can also be used as digitalportfolios, but they distinguish themselves first and foremost by being public. Studentweblogs are often filled with meta-comments about the student’s own learningactivities, and in this way are tools that support a self-reflective practice in self-regulated learning. Learning journals have long been used as a means for self-exploration, self-discovery, and self-disclosure, and weblogs are in this respectsimilar to journals. Being public, weblogs also serve as a means for public disclosurewhich adds a social dimension, as well as providing the opportunity for an openreflective practice within the community of interest (or even the shaping of acommunity of practice).

2.4.2 Features of a weblog

The main component of a weblog are the entries (content)s that are authored by theowner(s) of the weblog. These entries are displayed in reverse chronological orders(the last entry is always on the top). When you write an entry you can also label it bya category that reflects the content of the newly authored entry. Categories are meta-cognitive tools that structure your writings and your database into similar knowledgeobjects that are useful according to your own understanding of a domain. Wheneveryou write a new entry you can also decide wether you will allow visitors to addcomments to your weblog. Allowing comments to an entry is like attaching adiscussions group to it where the topic of your writing can be discussed. Last, there isan mechanims called trackback that enables other webloggers to send a notice to aweblog whenever they are commenting on an entry in their own weblog. This enablesthree things. First, visitors to an weblog can navigate to other webloggers writingabout the same topic, second, a weblog owner will know about who else out therelinks to him or her, third, this can build a web of interconnected ideas.

Next you will typically have a callendar in your weblog showing you dates markedwhen entries have been posted at a particular date. Another important feature of aweblog is the possibility to search it for information. When the information put into theweblog grow you will build your own personal knowledge database, annotated byothers through commenting, which is being indexed and thus searchable by your selfor visitors to you weblog. Links to other sites and other weblogs are also a typicalfeature found in weblogs. This is usually called a blogroll and give the blog owner away to make public the sites that are interesting and frequently used and read. Thisis a way for the learner to explicitly show the sources he frequently reads, and alsohelps the student meta-cognitively to be aware of the online resources he frequentlyuses. There are also links directly to archived entries. One archive for each month.All entries belonging to a category can also be archived together making it simple forvisitors to read about their category of interest while the owner has easy access toher or his own writings.

TELE Report

17

2.4.3 Benefits of using weblogs in education

First, a blog has the function as a personal, but public, learning log for reflection. It isa very suitable tool to demonstrate one's own learning progress and personaldevelopment. The public nature allows for feedback leading to collaboration andsometimes the building of communities that share common interests. Blogs are alsoowned by the learners themselves meaning they have total ownership of their ownlearning tool. They can customize and personalize it the way they want to, and thisalso gives them a strong motivation to use the tool. The tool is easy to use (“push-button-publishing”) which means that it is a powerful tool in the hands of both novicesand experts, bringing agency (a tool to influence and express oneself through) to alarge group of people. Multimedia support is also possible through uploading ofimages and other media files which also makes it suitable for a range of differentsubjects. Finally, weblogs are being adapted by many to a whole range of tasks thatare mostly done by large LMS (Learning Management Systems) systems today.

2.4.4 Implementation

A lot of the weblogging tools are Free and Open Source Software (FOSS). Thismeans there are no costs associated with the deployment of weblogging softwareeither for the product or the license. They do not demand powerful and expensiveservers (or clients), and most systems are translated into all the major languages.The systems lacking translation are easily translated due to the FOSS feature of a lotof the systems.

2.4.5 Technical requirements:

Every weblog system requires a web server with cgi capabilities. The apache webserver is the most frequently used web server and the easiest one to use if you wantto start blogging. Another option is to use Zope which is a combined applicationsserver and web server, although this limits your choice of weblog application to Zopeproducts (e.g. COREBLog or SimpleBlog). In addition you will often need a databaseto store the date in your weblog. MySQL is an open source database which nicely fitsmost weblog systems. More specific requirements can be found in the installationinstructions of each system.

2.4.6 Popular free weblogging tools:Jroller (Java, http://www.rollerweblogger.org/page/project )Wordpress (PHP based, http://wordpress.org/ )bBlog (PHP, http://www.bblog.com/ )COREBlog (Python, Zope product)b2evolution (http://b2evolution.net/ )SnipSnap (java, http://www.snipsnap.org/space/start )Serendipity (php, http://www.s9y.org/ )


Factors that contribute to the strength of the TELEAalborg University, Universitetet i Bergen, Vrije Universiteit AmsterdamThe TELE is easily transferable to other educational contexts. The technology is very low key, as isthe concept of a weblog itself (technology-poor/didactical-rich). It puts ownership of the TELE in thehands of the student, which is very motivating in itself. He or she decides how to use it, for whichpurpose and for which audience. It is a very suitable tool to demonstrate one’s own learning progressand personal development, and therefore is probably most useful in the executing/monitoring phaseof a cycle of self-regulation.

TELE Report

18

purpose and for which audience. It is a very suitable tool to demonstrate one’s own learning progressand personal development, and therefore is probably most useful in the executing/monitoring phaseof a cycle of self-regulation.The social aspects of a weblog are consistently highly scored: it is a very good tool for sharingknowledge and opinions, for giving feedback and for creating a network of peers, whether it bestudents or professionals.

Aalborg UniversityThe biggest strength and at the same time biggest weakness is the open-ended structure. Thisevaluation focuses on the strength of the open structure and the practice we can observe in thespecific course. All students have access to valuable inspiration from the teacher as well as fromother students. The individual student also has a tool for reflection because the blog by nature is anongoing report of the concerns of the owner. This, however, doesn’t ensure reflection, self-regulationand learning; it only works if the teachers and all students keep on contributing to the blogs.

Factors that represent weaknesses of the TELE

Aalborg University, Universitetet i Bergen, Vrije Universiteit AmsterdamTo make effective use of a weblog within the context of a course requires a lot of effort, both from thetutor and students.The tutor will have to set the example by using a weblog him/herself, and will have to develop a clearset of tasks that have to be carried out within a weblog. It also requires a clear place and purposewithin the broader curriculum.Students need to be motivated to maintain their weblog. It may be time intensive, lacking clearpurpose, and they may have some inhibitions in giving a wide audience insight into their self-reflectionactivities and progress.Also, within the context of a course, the social aspect of a weblog may be too time consuming,therefore the personal logging aspect may dominate.

Suggestions for improving the TELEUniversitetet i BergenFoster a culture of knowledge sharing, shed inhibitions about showing weaknesses etc. – these areaspects students should become familiar with.It's up to the students/teachers to appropriate the tool.It is important to foster a culture of knowledge sharing and this is something students should becomefamiliar with. Producing effective spam filters will also be important as long as the blogs are public,Last, the incorporaion of multimedia content is an area for improvement.

TELE Report

19

3. Content Systems With Tutor Support Group 2

3.1 Advanced Teacher Education in Informatics - ICT-based teacher trainingprogramUniversidade Católica Portuguesa

3.1.1 Context

Two Institutes of Portuguese Catholic University (UCP), Institute for DistanceLearning (IEFD) and Institute for Education (IEDU), decided to partner together in theoffering of a new MA degree on Education Informatics primarily aimed at teachers.

The opportunity arose when the Ministry of Education of Portugal decided tointroduce two new ICT subjects in the national curriculum. These subjects aremandatory for all students in grade 9 (90 minutes/week) and grade 10 (180minutes/week). The curriculum reform was to be launched as of September 2004 andthere was an estimated need for 1,800 trained teachers for the teaching of the newICT subjects. The priority would be granted to experienced teachers who wouldundergo some additional training in ICT didactics. No formal offering existedaddressing this need in the in-service training market and thus IEFD and IEDUdecided to take this opportunity and provide a qualified offer in this field.

The decision to move forward was taken in May 2003, the programme wasadvertised from June 2003 onwards and the first trimester opened in October 2003.The selection committee approved 143 students from about 230 applicants. This is arecord number of students for a Masters in Portugal.

Table 1: Course Statistics

Candidates 230

Enrolled 143

Classes 5

Tutors per course (1 per class) 5

Table 2: Geographic Distribution of Students

North 30

Central and South 106

Azores 4

Madeira 3

TELE Report

20

3.1.2 Objectives of the TELE

The Course Objectives were:

• To open new graduate teaching in an area of vital importance for the country• To specialize teachers of the basic and secondary education levels• To develop studies and research related to ICT• To allow the training of students that live and work in places where access to

some resources is difficult• To invest in innovative methodologies (massification of graduate studies)

For these objectives a two-year comprehensive curriculum was proposed:

• 1st Year:o 1st Term: Learning Psychology; Basic Notions of Hardware and

Networks (30h + 30h)o 2nd Term: Design of multimedia materials for web; Multimedia

Communication (30h + 30h)o 3rd Term: Common Software (advanced level); Internet and Education

(30h + 30h)

• 2nd Year:o 4th Term: Didactic of ICT; Research Methodologies in Education (30h +

30h)o 5th Term: Distance Learning and eLearning; Project Seminars (30h +

80h)o Dissertation

The TELE was designed to support these objectives. It uses a blended learningeducational model supported by collaborative learning and self-assisted learning asthe main methodologies.

3.1.3 The TELE Model

The TELE is web-based and uses two different commercial platforms: (1) Netforma,developed by Vector21, a Portuguese company, and (2) Blackboard. The use ofthese two different platforms served two main objectives. On the one hand it was thefirst experience in distance learning for the University and thus was an opportunity tofield evaluate different platforms and decide which criteria would be important tochoose a platform for a wider roll-out; on the other hand it matched one of theobjectives of the course itself, which was to familiarize students with distancelearning methodologies.

The model was designed to take advantage of a reflective posture on relevant pastexperience (teaching trajectory and ICT- more than 30% of students were alreadyteaching computer literacy at school level).

TELE Report

21

The technologies, the learning objects and the eLearning platforms used weredesigned to foster collaborative learning ventures. Meta-motivation and meta-cognition were two key features of this new programme, where active participation inforums (a tool that underwent considerable technological improvement in the courseof the experiment) was intended to be the main tutorial and learning instrument. Atthe outset the majority of students tended to use forums as chats – a lot ofconversation but little collaboration but, gradually, forum interventions became morestructured and directed at knowledge construction. Surprisingly questions and issuescovered in forums overflowed to other areas in the platform (e-mails, chats, virtualcommunities) and especially outside the platform (personal e-mail; MSN).

The pursuit of meta-motivational enhancement and strengthening of vocationalidentities emerged as strategic elements of personal and social learning.

One of the evidences of the success of the model may be the low drop-out rate of theMasters, even when compared with international benchmarks:

Number of Students attending each term

The TELE blended educational model is characterized by a monthly class sessionand self study periods in between. This self study period is supported by activetutoring where tutors are responsible for:

• Motivating students;• Providing counselling;• Managing the forums;• Clarifying questions through forum, chat or e-mail;• Conducting on-going evaluation (personal assignments; group assignments;

forums; etc.)• Conducting final evaluation;• Preparing and coordinating class sessions.

143

131

124121

110

115

120

125

130

135

140

145

inicial trim.1 trim.2 trim.3

Series1

Series2

TELE Report

22

3.1.4 Sites

http://elearning-ucp-ie.vector21.comhttp://lms.iefd.ucp.pt


Factors that contribute to the strength of the TELE

Universidade Católica Portuguesa

Close relationship between technology and conceptual model of trainingThe use of active tutoring in ODLPossibility of strong socialization among the different actors in the TELE

Universitat de Barcelona

The collaborative environment allows the learner to communicate with peers and tutors, and also toplan and prioritise tasks. It is a friendly interface and allows traditional records of activities. The TELEalso allows the tutor to promote SRL.

Factors that represent weaknesses of the TELEUniversidade Católica PortuguesaLack of a good tracking system to allow self-evaluation and analysis of work done by studentsThe need for a multimedia platform with some level of sophistication to access to all facilitiesWhen using a sophisticated platform both students and tutors should have previous training in theiruse, bearing in mind its effectiveness.

Universitat de BarcelonaThe TELE leaves an important role to promote SRL to the tutor (but for us is not a weakness of thecourse itself)There is only a traditional way of recording activities - the TELE does not allow the promotion of peer-evaluation (only by sending an email).

Few chances to choose learning patterns and strategies.

Suggestions for improving the TELEUniversidade Católica PortuguesaThe provision of better tracking systems in technologyAllow the students to choose different paths in building their own curriculum

Universitat de Barcelona

The TELE should have a space to record the learner activities, the partners’ review and its evolution.

Give the opportunity for peers to access personal files or documents.

(*) This, connected with the opportunity of choosing patterns and strategy, will elevate the cost of thecourse’s production (more interfaces designed, more contents, more space for each student, etc.).

TELE Report

23

3 . 2 C o g n i t i v e p s y c h o l o g y c o u r s e – E n h a n c e d W e b b a s e d s u p p o r tNottingham Trent University

3.2.1 Using web-pages to support undergraduate courses

Web pages are one of the main sources of information for students. They provideaccess to news, to academic material and to information about their interests. If youcan use a computer you can access the internet and read web pages. It is the idealenvironment to make information available to people at all times in all places.Students can access web pages when they want and where they want.

Lecturers can use web pages to present material to the students and provide supportfor their courses. For example they can use the medium to make their lecture notesavailable which provides an ever-present source of the material and eliminates theneed for paper handouts. Web pages can go beyond the simple presentation of textand offer a range of support materials that will help students to regulate their studyand develop their learning skills. Some examples of these support materials are;

• suggestions for further reading that allow students to concentrate and extendtheir learning where they feel they need it most, or where they have generatedinterest through their studies.

• mock exam questions enabling students to tailor their further reading andrevision to the requirements of the module assessment.

• examples of assessed work annotated by the tutor to show strengths andweaknesses.

• annotated links to other internet sites.• copies of academic papers.• discussion boards to enable an exchange of ideas between students.

Web pages can create a structured environment for students that encourages themto complete their course, provides support for this aim and also encourages readingoutside the strict requirements of the course. The challenge for lecturers is to providean electronic environment that is more than a book and encourages students toactively engage with the material to develop their own learning path.

3.2.2 Example of using web pages to support cognitive psychology courses

The level one cognitive psychology course at Nottingham Trent University uses webpages to add some extra support for students. The course engages with students ina number of ways with the aim of encouraging their self-regulated learning. Thecourse provides a programme of lectures and workshops to present the informationand encourage debate. The web pages are an additional source of information andencouragement.

The materials for the module are set in the context of more general support forcognitive psychology (http://ess.ntu.ac.uk/miller/cognitive/) which includes paststudent work and links to information on cognitive psychology topics not directlycovered in the modules.

The students can then get access to the level one support materials. Each lecturehas a range of materials including lecture notes (available before and after thelecture), links to other sites and links to explanations and illustrations of key ideas.

TELE Report

24

The outline for a lecture identifies topics to be covered and provides links todemonstrations of the concepts described such as a linked site, with examples toillustrate the concepts being described.

The web pages also provide numerous online resources external to the Universitywebsite, including;

• academic papers,• other Universities’ online tutoring resources which cover similar modules,• video and audio resources• illustrations of concepts/terms, such as perceptual illusions.

These resources provide a structured route into cognitive psychology and it is hopedthat this will provide a starting point for study rather than an end point. That choice isin the hands of the student.

3.2.3 Self-regulation

Web pages can also be used to encourage self-reflection and assessment. On thecognitive psychology web pages students are encouraged to assess their ownprogress and to prepare for module assessment with a range of materials. Theseinclude mock examination papers and examples of student work. Self-assessmentpages allow students to attempt short answer questions and look at the correctanswers and explanations for them.

There are also examples of assessed work that have been annotated by the lecturerto show strengths and weaknesses. These pages highlight the essays to show whichpiece of the answer gained credit and which contained errors. Students can togglethrough the pages to get a picture of the most parsimonious response.

Other ways of encouraging self-regulation include providing interactive questions anddiscussion pages. Web pages can be used to offer a bank of multiple choicequestions that can provide instant feedback. A web environment can also be used toprovide a discussion board that allows student discussion and for tutorialassignments.

An example of a discussion board: a web board was set up to allow students to havespace for informal discussion and academic discussion. The challenge for theacademic staff is to find the best way of motivating the academic discussion andencouraging the students to engage with their peers.

2.2.4 Educational benefits of using web-pages

Web pages offer many benefits for students and lecturers;• They are relatively easy to create using standard software,• They are accessible anywhere and anytime,• They can create a gateway to a wide range of other resources,• They can provide examples of student work,• They can encourage self-reflection,• They can be used as the focus of an electronic community.

TELE Report

25


Comments on Part C of PRET or on PRET in generalNottingham Trent UniversityThe TELE does what it sets out to do. It provides support for a module on a degree course and itoffers routes out from the prescribed material to other sources and other ideas. It also offers examplesof how student work can be presented and allows the student to learn from the successes of otherstudents.

It does not have a facility to be interactive and so does not score highly on the dimensions of thePRET. This suggests the PRET might best be used selectively so that the zero values do not weightthe overall outcome excessively.

CNR Istituto per le Tecnologie DittaticheWe completed part C based on the general feeling we got from the analysis of the TELE, not bymaking an average of the scores given to the single questions.

Factors that contribute to the strength of the TELENottingham Trent UniversityThe easy to use interfaceThe wide range of links for the studentsThe encouragement to look beyond the standard materialThe stimulus to further studyThe www base for the material (accessibility is shown in the response from an international reader toone problem about arrows on a model of memory)

Vrije Universiteit AmsterdamThe web pages which support Cognitive Psychology courses at Nottingham Trent University contain awide variety of information resources which students may find motivating and helpful in tacklingvarious learning tasks.As far as self-regulation is concerned, there is no explicit attention in the TELE to how students canadapt their learning strategies within cycles of planning, executing, monitoring and assessment.However, there are elements present which at various stages of the self-regulating cycle may play animportant role:- At the planning and task analysis level; students can make use of tutor’s notes, examples,

demonstrations and assessment specifications to analyse the requirements of the task andsubsequently decide on strategies to achieve the task objectives. Especially the cognitive andmotivational aspects of the planning stage are supported in the TELE.

- At the executing and monitoring level: the student can use self-tests and examples of otherstudents’ work as a measure to judge their own performance and progress, and consequently,adapt learning strategies.

Self-regulation is therefore an implicit feature of the learning model, through the availability of a varietyof resources which students may use at various stages of their learning.

CNR Istituto per le Tecnologie DittaticheFrom the point of view of SRL, the cognitive aspects of this TELE are the most relevant ones. Somesupport is also given, though in limited measure, to motivational aspects. Social and emotionalaspects are almost not addressed at all.

The TELE provides many resources, commented on by the teacher and related with each other inmeaningful ways.The TELE provides a rich group of materials for self-assessment (though limited to one topic), that is:1) questions with the answers, initially hidden, explained and commented in a meaningful way;2) examples of materials produced by students in previous years, with comments from the teacher

highlighting different features in different ways;3) mock exams.Moreover, the teacher’s comments are not limited to giving the right answer or content explanations,but often exemplify a methodological approach to the considered field.

TELE Report

26

Factors that represent weaknesses of the TELENottingham Trent University It is not interactive and so does not provide individual feedback to students It is not a requirement of the course and so students might not value it Some students require explicit motivation to develop a new learning style, in this case exploringacademic resource on the webThere are no direct measures of its effectiveness or use, though student feedback to the overallcourse includes some positive comments about the website

Vrije Universiteit AmsterdamSelf-regulation characteristics situate themselves mainly at a cognitive, emotional and motivationallevel. The social level is not really supported (e.g. through peer feedback) and not a technical featureof the TELE.Finally, self-regulation is not made explicit through instructional methods or particular technicalfeatures of the TELE. Of course, we can not comment on the self-regulating characteristics of learningactivities which take place outside the TELE, e.g. in tutorials or through particular assignments

CNR Istituto per le Tecnologie DittaticheThe self-assessment material is given for just one topic.No social interactions are possible.Not all topics have materials of any kind online.The structure of the links is complex and not straightforward; it is difficult to find again materialpreviously found; there are some loops in the links.

Suggestions for improving the TELENottingham Trent UniversityA question bank with a self assessment facilityNotice board facility for students

Vrije Universiteit Amsterdam-A clearer structure to the site may be conducive to the amount and intensity of usage.-Making the self-regulating characteristics of the TELE more explicit by setting specific learning tasksat all stages of the learning cycle, aimed at making students more knowledgeable and proficient inskills related to task analysis, strategy selection and self-monitoring.

CNR Istituto per le Tecnologie DittaticheProvide material for all topics, especially self-assessment exercises and commented examples.Improve the readability of the site by adjusting the structure of the links.

We do not suggest that the TELE provide the possibility of social communication, since this wouldchange the nature of this TELE. However, a TELE of this kind could be paired with an environment likeWeb-board where this does allow students the possibility of interacting with each other.

TELE Report

27

3.3 Digital Video Course – Blended learning with new mediaUniversitat de Barcelona

The course “Digital Video” (http://www.cav.ub.es/docencia/dv/textos/) is a non-compulsory course in the Audiovisual Communication Studies, at the Faculty ofTeaching Training of the University of Barcelona. The studies of AudiovisualCommunication at the UB are not traditional face-to-face studies, but they are not e-learning either: they are defined as blended. A simple conception defines blendedlearning as a model that “combines face-to-face and virtual teaching”. This meansthat the class hours are reduced and, at the same time, the importance of thelearning process is increased, justifying an extra teacher’s dedication time. Theteacher must dedicate these free classroom hours in planning different activities: inthe virtual campus, in small groups, tutoring, etc.

The course aims to develop technical competencies connected with the Digital Videoand critical competencies (in connection with the eCPLIse project, http://tv-lmi.ub.es/eclipse/). The class session takes place in a special laboratory (aclassroom where each student has a computer). Material is presented in lectures(one hour per week for 24 weeks) and supported through two workshops during theyear-long programme.

The website provides;• access to lecture material in the form of text-based handouts.• suggestions for further reading that allow students to concentrate and extend their

learning where they feel they need it most, or where they have generated interestthrough their studies.

• mock exam questions enabling students to tailor their further reading and revisionto the requirements of the module assessment.

• examples of assessed work annotated by the tutor to show strengths andweaknesses.

3.3.1 The Digital Video course and the self-regulation learning.

The factors that contribute to the self regulation learning are:

- The specific blended learning model, which combines the TELE with the faceto face sessions and individual monitoring interviews.- The possibility of following up the activities by teacher and peers through thevirtual campus.- The strong social aspects. Specifically the possibility to socialise through theweb, and not only with tutor and peers, but also with external audience (for example,to share the outcomes of the course with other peers of the eclipse project).- There is an intuitive interface, using a powerful LMS, but not used with lots offacilities.- The TELE was used as an informational support, using video facilities withinterest to learning strategy.

3.3.2 Transferability

TELE Report

28

The campus is a commercial one (WebCT). The course’s information (without thecommunicative tools) is easy to transfer. It would be possible to transfer this course(with the project) to non-university target groups (ethnic minorities) in order toempower them in the analysis of the media.


Factors that contribute to the strength of the TELEUniversitat de Barcelona

- The specific blended learning model, which combines the TELE with the face to face sessionsand individual monitoring interviews

- The possibility of following up the activities by teacher and peers through the virtual campus.- The strong social aspects. Specifically the possibility to socialise through the web, and not

only with tutor and peers, but also with external audience (for example, to share the outcomesof the course with other peers of the eclipse project).

Universidade Católica PortuguesaThere is an intuitive interface, using a powerful LMS, but not used with lots of facilities.The TELE was used as an informational support, using video facilities with interest to learning strategy

Factors that represent weaknesses of the TELEUniversitat de Barcelona

- The campus does not allow self-evaluation.- The TELE does not register the different stages of the student’s products.- The peers cannot evaluate each other’s compressions in the TELE.- Doubts about the effective consequences of the chat and forum.

Universidade Católica PortuguesaThe LMS was not used as a collaborative tool.The forum did not have enough communication between students and tutors. This means that theimportant moments of training were centered in face to face moments.

Suggestions for improving the TELEUniversitat de Barcelona- To incorporate more activities into peer evaluation. For example to socialise the student productionthrough making their outcomes public in the ftp space.

- Building feedback mechanisms in the TELE itself

Universidade Católica PortuguesaDifferent approach in using the technology, trying to use the collaborative tools available

TELE Report

29

3.4 Teacher training course on ET- Using a CMCS in teacher trainingCNR Istituto per le Tecnologie Didattiche

Supporting Self-Regulated Learning in a CMC environment

3.4.1 Description of the teacher training course on Educational Tecdhnologies

The TELE analysed by ITD-CNR is the ad hoc configuration of Centrinity First Class1

prepared and used for TD-SSIS, an online teacher training course in EducationalTechnology. First Class is a commercial Computer-Mediated Communication (CMC)system, that is, a web-based environment aiming to allow a group of people (in ourcase teacher trainers and their trainees) to interact in structured way.

The study of SRL in this kind of environment appeared to be particularly promising,since online learning itself compels the students to self-regulate time and work pace,as well as to develop abilities to interact and collaborate with peers, progressivelydecreasing dependence from the teacher.

3.4.2 Context of the course

The Educational Technology course analysed is part of the Specialisation School forSecondary Teaching (SSIS). ITD-CNR has been in charge of this course since theyear 2000, that is, when the SSIS was established. Within the TELEPEERS project,we have been focussing on the online edition of this course run in the spring 2004,mentioned as TD-SSIS in the following.

The benefits of computer-supported collaborative learning are well documented inthe literature, especially in the field of teacher development (Gray et al, 2005). Inparticular, the choice of this method for the TD-SSIS course derived from twoconsiderations. Firstly, the great diversity among students’ background and attitudescalled for a student-centred approach that increase responsibility for learning andalso be more flexible than traditional, transmissive teaching. Moreover, the broad andoperative nature of the course’s learning objectives could better be achieved throughan active learning process where students can question, apply prior knowledge,share and develop new ideas.

TD-SSIS’ main objectives include the development of instructional designcompetence, with focus on evaluation and selection of learning strategies andtechniques. In other words, student teachers are lead to become aware of both thepotential and pitfalls related to educational technology implementation in the schoolsetting.

The approach adopted in the course is inspired by socio-constructivist theories oflearning and is therefore heavily based on collaborative learning activities. Thistheoretical framework obviously influenced the choice of both the software platformused and the configuration worked out.

3.4.2 Reference

1 http://www.firstclass.com/

TELE Report

30

Gray D.E., Ryan M., Coulon A. (2004) The training of Teachers and Trainers:Innovative practices, skills and competences in the use of eLearning, EURODL-E u r o p e a n J o u r n a l o f O p e n a n d D i s t a n c e L e a r n i n g ,http://www.eurodl.org/materials/contrib/2004/Gray_Ryan_Coulon.htm (last retrievedon 15 Jan 2005)

3.4.3 Methodology applied

The course approach was mainly based on co-operative leaning, entailing discussionand collaboration among peers to develop shared understanding and meanings,according to the canonical pillars of co-operative learning online. Some of theactivities were purely reflective, others were aimed at producing artefacts such assmall project works. The course was not designed bearing in mind the developmentof SRL abilities; nevertheless, course tutors tried to foster meta-reflection andmethod appropriation. Qualitative analysis of messages exchanged suggest thatthere are peculiarities to SRL abilities when the learning context is a collaborativeone, based on Computer Mediated Communication (CMC). In addition, it seems thatthere are activities that foster the development of such abilities better than others.For example, role playing activities can trigger the assumption of responsibility on theside of the trainees, helping tutors to step back from their scaffolding role to leave theground to one or two trainees with a penchant for leadership.

3.4.4 Structure and contents

The online course lasted 10 weeks and included three face-to-face meetings: one atthe beginning, one midway through, one at the end of the course. Basic computerskills were course prerequisites, because course content mostly involvedmethodological and pedagogical aspects, rather than technical ones. The courseprogram included seven main streams of activities, on the following topics:1) socialisation and familiarization with the learning environment,2) introduction to educational technology (definitions and history),3) learning theories (mainly behaviourism, cognitivism and constructivism),4) analysis of educational software of various kinds (from constructivist learning

environments to classic drill and practice or reference works);5) practice with online educational activities, with particular focus on webquests;6) in-depth study of one topic chosen by the trainees among “teachers communities

of practice”, “analysis of the educational potential of written interaction in CMCenvironments” and “case study on a European project concerning the definition ofa syllabus for teacher training in ICT”

7) meta-reflection on the learning process.

3.4.5 Evaluation within the TELEPEERS project

This FirstClass configuration was peer-reviewed using the Peer Review EvaluationTool (PRET). It was evaluated as a system which supports self-regulated learning toa medium-to-high degree (4 out of 5), offering its best on the social level (5 out of 5),showing its weakest aspect on the cognitive level (3 out of 5), and leaving thecognitive and emotional levels heavily dependent on the use made by the agentsinvolved in the learning process (teacher, tutors and students). The explicitness ofSRL support was just average (3 out of 5).

TELE Report

31

These results were rather expected, since the software underlying the environment isessentially a collaboration platform. Moreover, the configuration prepared aimed toprovide a balance between individual and co-operative work, helping the learner toexploit the interactions with tutors and peers by sharing reflections, comparing resultsand receiving support, yet leaving space to personal work, reflection and initiative.

The main points of strength were considered to be: 1) the easy and intuitive interface,apt to shape a creative learning environment; 2) the teaching strategy applied, apt tofavour the increase of self-confidence by the students.

The main weaknesses were considered to be: 1) the lack of a quick overview ofcourse content; 2) the prevalence of text-based interaction and limited exploitation ofmulti-media potential; 3) the limited document-sharing tools and functions offered byFirst Class.

The points of strength result from suitably exploiting the potentialities of the softwareplatform; the points of weakness depend both on structural limitations of the platformused, and on the fact that the development of SRL abilities was not among theexplicit aims of the course.

The following suggestions emerged to improve the support of this TELE to SRL:

• Providing methodological help to which the students may refer, in order to makemore explicit the support to SRL;

• Providing automatic functions to stimulate the students to undertake appropriateactions (e.g. remind them to intervene in a discussion/forum, point out theexistence of new messages or new material, etc). This service should not beimposed, but offered as a possibility to be set by the user.

• Giving the opportunity to have a space to record the learner’s activities, thepartner’s review and its evolution.

• Adding a personal planning space where it could be possible to record theevolution of the learner’s experience, so as to promote self and peer evaluation.


Comments on Part C of PRET or on PRET in generalCNR Istituto per le Tecnologie DittaticheThe global evaluation reported below was made based on our general “feeling” about the consideredTELE; it turns out to be in line with the analytical answers given in part B.The considered TELE appears to be particularly fostering SRL as concerns the social level, and showsthe cognitive level as its weakest part. These results were very much to be expected, due to the natureof the considered environment.

Factors that contribute to the strength of the TELECNR Istituto per le Tecnologie DittaticheBeing a collaboration environment, this TELE allows the learner to exploit the interactions with tutorand peers, hence receiving support, occasions to compare and reflect, etc. At the same time, it allowsthe learner to work autonomously, self-regulating his/her working time and pace. This balancebetween individual and cooperative work seems positive as concerns SRL.

Universidade Católica PortuguesaThere is an intuitive interface, with limited facilities, but enough to create a creative learningenvironmentThe proactive tutorial strategy allows a collaborative learning environment and self-confidence instudents.

TELE Report

32

students.

Universitat de BarcelonaThe collaboration environment allows the learner to work together with peers and tutors. It is a friendlyinterface and allows traditional records of activities. The TELE also allows the tutor to promote SRL.

Factors that represent weaknesses of the TELECNR Istituto per le Tecnologie DittaticheExploiting the potentialities of the considered CMC environment depends on many factors:

the didactical project and instructional design; the context of use (including both teachers and peers): the type and amount of scaffolding given by the tutors.

Universidade Católica PortuguesaThe interface facilities do not allow a quick overview of the course content (it’s on the library – notusual in current LMS)Does not have a tool to build automatic questionnaires.There is no facility to keep the groups working alone (without allowing others to see their work)There is no visible space where students can share files, in an open way.

Universitat de BarcelonaIt is difficult to find a global vision of the course in the TELE. (Only an easy vision of the activities bythe conferences).The TELE leaves an important role in promoting SRL to the tutor.Only traditional way of recording activities.

Suggestions for improving the TELECNR Istituto per le Tecnologie DittaticheThe software platform used could improve its support to SRL by:Providing a methodological help function to which the students may refer, in order to make the supportfor SRL more explicit;Providing an automatic function to stimulate the students to undertake action they should be doing(e.g. send a mail to invite intervention into a forum, send a note to point out the existence of newmessages or new material, etc). This service should not be imposed, but offered as a possibility to beset among the preferences.

Universitat de BarcelonaThe TELE will improve by offering a space to record the learner activities, the partner’s review and itsevolution.Adding a personal planning space where the evolution of the learner’s experience can be added - willbe useful to promote self and peer evaluation.

TELE Report

33

4. Content Systems Without Tutor Support Group 3

4.1 Sunpower – CD-ROM to train business EnglishUniversität zu Köln

Communication Strategies in English for Business Purposes

4.1.1 Program development

Sunpower is a multimedia learning program available on four CD-ROMs. It wasdeveloped at the University of Applied Sciences Cologne (FH Köln), in collaborationwith language and media consultants in the UK and the Institute for IntegratedCampus and Distance Learning (Institut der Verbundstudien der FachhochschulenNordrhein-Westfalens) at Hagen (Germany).

4.1.2 Main ideas

Recent research results on learning prove that the acquisition of knowledge differsamong individual learners, largely depending on their backgrounds, learningexperiences and existing knowledge. It is recognised that each learner constructshis/her knowledge him/herself. Learning is considered an active process where (likein real life) learners select the information they need from a rich authenticenvironment and incorporate it into their mental structure. The more the learningenvironment resembles the situation to which transfer is required the easier is thetransfer of the new acquired knowledge, if the opportunities presented are relevantand appropriate to their level. Autonomous learners who want to monitor their ownlearning process should be aware of this. The aforementioned findings also apply tosecond-language acquisition and have been implemented in the Sunpower program.

The Sunpower program focuses on communication strategies in English for businesspurposes. It was developed on the basis of the concept of situated learning and moregenerally on constructivist ideas: authenticity, complexity, holistic approach, user’sresponsibility for and awareness of their own learning process, and learnerautonomy.

Any second language learning program has, of course, to be selective and can onlyoffer a reduced image or simulation of real life.

Authentic materials contain hesitation devices, overlapping dialogues, pauses, non-sequiturs, slips of the tongue and so on, which correspond fully to real-life conditions.Although the material presented by the program exhibits these qualities, the programcan be adjusted for the purpose of interaction and exercise.

4.1.2 Target group

The main target group are adults, mostly students of business administration andrelated subjects. The students have either an entry qualification for a Universitycareer, have been trained in professional context or have had other kinds ofoccupation. This means that they want to use their limited time most efficiently. At theend of their language training they have to pass a major examination, where they

TELE Report

34

have to prove their ability to implement their language competence within aprofessional field.

The second target group are professionals, already working with English speakingpartners or planning to do so in the near future. The third target group are distancelearners or students in open learning institutions.

4.1.3 Objectives of the program

The Sunpower program aims at enabling the adult intermediate to advanced levellearner;- to recognise and use communication strategies in a professional field- to identify cross-cultural features in a specific professional field and avoid cross-

cultural misunderstandings- to improve language skills (listening and speaking) within a specific professional

field.

4.1.4 Principles underlying the instructional design of the program

The Sunpower program;- intends to offer a rich learning environment that is as close to real life as possible- allows for learner autonomy – providing free and transparent access to all

features- enables the learners to be aware of their own learning strategies and encourages

them to take responsibility for their own learning process.

4.1.5 Navigation

The program provides the students with a choice of different paths he can take. Italso offers graded help facilities (transcription, vocabulary, grammatical comments,support language) in order to meet the heterogeneity of the target group. It is up tothe user how much help he wants to call on.

The ORIENTATION menu offers the user an overview of different paths. He can startwith a short self-evaluation and then continue with one of the four EPISODES thatmake up the core of the program. In these episodes, the student is presented with anumber of communication strategies as they are employed in business negotiations.The contextual framework is provided by the Intersolar group, a group based in theU.K. that focuses on solar energy.

EXPLORE allows the learner to select;• Texts of video sequences• Language lab facilities• Translation (into German)• Hyperlink explanation of vocabulary, grammar and cross-cultural issues.

After clicking on INTERACT, he can go to;• Video sequences• Interrupt a dialogue and decide how to continue this dialogue• Explanation of the communication strategies used.

TELE Report

35

STRATEGY will allow the user to focus on;• Explanation of communication strategies• Examples• Activities: exercises on strategies.

Clicking on the WORKSHOP icon provides the user with a number of facilities thatmay help him to monitor his/her learning progress , in particular the optionsWORKPLAN, NOTEPAD and SELF-ASSESSMENT and TESTS.

The Sunpower program takes into account the various types of learners and theirdifferent needs by offering a great variety of audio, audio-visual and textual stimuli.Besides, it allows the use of the learner’s individual learning strategy: they maychoose the way which is appropriate to them. They may, for example, prefer learningby watching and listening or learning by studying examples and annotations andlooking up linguistic specificities first.

4.1.6 Availability

http://www.sunpower.fh-koeln.de/BEENGL.HTM

The sunpower program (4 CD-ROMs) including a workbook is available from HueberVerlag in Germany (http://www.hueber.de). Price: 99.95 €


Comments on Part C of PRET or on PRET in generalUniversität zu KölnSunpower is basically a program for self-study, i.e. it does not provide for communication with peersand tutors. It therefore rates very low on the social dimension of the PRET which in turn leads tooverall evaluation results that do not do justice to the program as a TELE for self-study.

Nottingham Trent UniversityIn the totals above, Sunpower scores well on the cognitive aspects, but less well on the social andemotional aspects. However, the TELE is intended to be used entirely independently with no socialelement, and so it should not be presumed that a low score on this dimension means a poor TELE forSRL. Also, if the aspects which target the cognitive factors of the learning process are successful,presumably this successful learning would facilitate a positive emotional reaction to using and learningwith the TELE.

Factors that contribute to the strength of the TELEUniversität zu KölnAuthentic situations and dialoguesThe TELE offers a number of strategies for self-regulating learning

Nottingham Trent UniversityAbility to work at own pace, and at times convenient to the learnerAbility to re-do assessmentsThat learning progress is guided by the structure of the workbook, but learners can choose a differentorder if they wish to

Université de Technologie de CompiègneMotivational aspects

ß Arouses the personal interestß Numerous tests

TELE Report

36

Self-Regulated Learningß High autonomy in using learning strategiesß Self-evaluationß Resources for individual organization of learning procedure

Well-developed audio-visualsß Participation of user’s voiceß Listening to the written textß Watching the intonation of speakers

Close to real-life situations

Factors that represent weaknesses of the TELEUniversität zu KölnNavigating takes some time to get accustomed toGrammar exercises are inappropriately basic given the high level of competence in English the usermust have if he wants to work with the program successfullySome exercises do not react appropriately to false answers

Nottingham Trent UniversityTechnical requirements to use the TELENo contact with tutor or peers to discuss/compare progress and performance – help is restricted tothat programmed into the CD

Université de Technologie de CompiègnePoor interface

ß Lack of homogeneity and stabilityß Low ergonomics

Implicit navigationß Hard to orientate at the beginning

Poor feed-backß Doesn’t indicate the wrong answersß Lack of communication among peers

Suggestions for improving the TELEUniversität zu KölnImprove overview structureCheck thoroughly exercises and testsAllow for communication with peers and tutor

Nottingham Trent University

Université de Technologie de Compiègne¸ Feedback¸ Communication among peers¸ Interface and navigation¸ Voice recognition

Contents translation adjusted to the user’s language

TELE Report

37

4.2 Databases – A model for online studiesUniversité de Technologie de Compiègne

4.2.1 The course Databases (NF17)

The objective of the course is to lead the students to control the conception ofrelational databases and relational objects. This course mainly focuses on learningconceptual and logical modelling and obtaining technological competence in classicalDBMS (like Access, MySQL or Oracle) by applying database languages (like SQL,PHP or JAVA).

The course is divided into two main parts: lectures, supervised works and practicalworks conducted in the same time period. During the course, the teacher explainsthe new concepts to be assimilated (which may be either not very clear, or are toonumerous). During the lab works and the supervised works, the students assimilatethe concepts taught at courses, but there is always a person in charge of thesupervised work who can help with any questions regarding the course. Then at theend of the six-month period a project in groups of four must be conducted. Thestudent is not obliged to choose the technical tools. The students submit theirhomework via VLP. Moreover, the experts also suggest to the students that theypresent real projects on the creation of database.

The documents of learning are formatted by the Scénari platform. Indeed, that allowsthe student to act, because it is s/he who has to advance in his/her learning andmanage his/her acquired knowledge. Thus, the students are confronted with the factthat a new learning approach is necessary. The work in groups is significant in thismodule because it is the centre of this learning program. The student is betterprepared to co-operate. S/he controls his/her own project and his/her knowledge ofvarious technological database tools (SQL, MySQL, Oracle, Access, etc…).

At the end of each chapter, there is a series of exercises. In addition constantly, it ispossible to affix a flag at the end of the chapter in order to know what was taken ornot.

4.2.2 The Forum

The forum is an asynchronous and collaborative communication tool. It makes itpossible for a group to exchange and discuss around a topic or question. There areseveral types of forum. The principles which we chose are as follows:- the forums are created around main categories: Project, Course or TP.- each one can then begin a new debate or raise a question in the forum of its choiceby the creation of a new discussion- each actor can feed the debate or answer the put question. The interventions andthe answers all appear on the same level in the discussion in chronological order.

The Forum heading comprises a sub-menu made up of the list of the forumsavailable.When a forum is selected, the list of the discussions within it appears in theright part.A discussion can be introduced by a question or a topic.

Once the discussion has been created, all the actors can feed into it by a successionof answers which will appear in the discussion in chronological order of remark.

TELE Report

38

4.2.3 Document Space

Document space is a storage space for documents produced or used within theframework of the course. When a document appears in this space, it means that it isregistered on the server of the formation. The people registered with the courseconcerned will thus be able to reach these documents from any station connected tothe network.

When a person downloads a file in document space, this means that it records it onthe hard disk of his or her working station. In order that the modifications arereflected on the file of space documents, it is necessary to update this last byreplacing it by the file modified on your hard disk.

Space documents contain two distinct parts accessible by a sub-menu:

4.2.4 Common Space

The common space is visible by all. Only the teachers can add, modify or removedocuments to and from it. Students can download on their hard disk the files presenton this space. It is a place for provision of documents concerning the whole of thegroup.

4.2.5 Space for returned homework

There is a space for "Returned" documents. This space is a place for exchangebetween the student and the person in charge of the practical work. Students can putthere the items to be returned or any other document which they wish to make visiblewith the tutor.

The tutor has access to the whole of the space and can also add to it and modify filesin each of these spaces. This makes it possible for students to return their projects.

4.2.6 Availability

http://www4.utc.fr/~nf17/


Comments on Part C of PRET or on PRET in generalUniversität zu KölnIn the totals, the TELE Databases (NF 17) scores very highly on all aspects.

Even though the high support of the social aspect is misleading since it consist of (0; 0; 5), it isnonetheless given because the learner can communicate with her/his tutor during the lecture.

Therefore “NF 17” is a TELE that strongly supports SRL.

TELE Report

39

Factors that contribute to the strength of the TELEUniversité de Technologie de CompiègneWell organized course and tutorial material.Self-evaluation.High interaction with the environment: tutors, peers…

Universität zu KölnThe TELE is integrated into a context of face-to-face learning, thus the learners use it as asupplementing tool to deepen their knowledge acquired during the lectures and the practical work.The learners have the opportunity to learn at their own pace, and at the times convenient to them.They have the opportunity to re-do the assessments.A strength of the TELE is that it recommends numerous links to the learner which provide her/himwith further information and which offer the student other learning strategies and possibilities todeepen and test her/his knowledge.It is free and uses standard web technology.

Factors that represent weaknesses of the TELEUniversité de Technologie de CompiègneNo automatic adaptation to the level of the learner.No personalization of the interface.

Universität zu KölnThe TELE gives the learner no possibility to contact her/his tutor or other peers. However as the TELEis a complementing part of a lecture, the social aspect is given nonetheless.

Suggestions for improving the TELE

TELE Report

40

4.3 Programming Tutorial – Additional online exercisesUniversité de Technologie de Compiègne

4.3.1 The course Algorithmics and programming (NF01)

The objective is to help the student better understand the programming as well asalgorithms. The student learns how to model a problem of programming and how tofind a solution in the form of an algorithm, by applying the Pascal program.

4.3.2 The module

The module is divided into three parts: lectures, supervised works and practicalworks. During the lectures, in the first half hour of the course the teacher will resumethe chapter. In the second half hour the students address questions on the presentchapter. During the rest of the time the chapter to be prepared for the next week willbe presented (15 min), and in the remaining quarter of an hour the students will haveto take a quiz on the present chapter. During the supervised works (TD), the studentsare divided into groups of four. At the beginning of six-month period they receive allthe subjects to be applied during the supervised works throughout the entire year.Each person in charge of the supervised work does not correct the exercises, butcalls a student to the blackboard after giving him/her time to think over them. Itobliges them to prepare the course before coming to supervised works. The labworks put into practice all the theoretical knowledge of the module. They last twohours, take place every fifteen days, and are conducted at computers. The first labworks are not graded, and the last three are evaluated with a value of 5% for each.Each lab work deals with the main notions of algorithms, the other labs refer to theremaining elements.

The evaluation is continuous and final. There is a mid-term exam and a final exam tobe passed. The last three lab works complement the final grade of the student withthe continuous control.

4.3.3 The website

The website includes a number of tools; among these are simulation, exercises,quizzes, electronic notes, and links to other notes.

All the tools permit the students to verify their assignments. The website is a real toollike the VLP in distance modules because it offers the students a real possibility ofevaluating their own knowledge. There is also the little quiz, which allows students toevaluate their level. The website is for the first cycle students to help them tounderstand the course as well as possible. On the website they will find the course,various corrections of the former examinations, simulations for better apprehendingthe data-processing concepts, for exercises as well where there may be some doubt.The website is divided into four parts:

4.3.3.1 Course

Quick access to the course is achieved by the menu on the left. Afterwards, theinterface will enable us to reach the contents of a course by two different routes. On

TELE Report

41

the right-hand side, it is always possible to consult simulations or exercisesconnected to a chapter.

4.3.3.2 Exercises and quizzes

The quizzes and exercises allow the students to evaluate themselves and to learnfrom the errors they may make. Thus, they can deal with any gaps in their ability.They will then be able to work more efficiently.

The exercises and the quizzes are gathered together by chapter. After each one, thestudent can apply the concepts directly that he has just acquired, while being testedduring the relevant quiz. At the end, an indicative mark is given so the student canknow if his/her understanding was accurate.

4.3.3.3 Simulations

Various simulations are placed at the disposal of the student.What can happen ineffect is that the training by using examples can be theoretical. With simulations, thestudent can visualize the problem better. As with the exercises and the quizzes,simulations are easily accessible and are gathered together by name.

4.3.3.4 Game

The game is the "final surprise" for the learners. It is an effective means for the userto realise if s/he has mastered the entire contents of the course. It is role play wherethe user will be presented with various options when being asked a question. It is allintegrated into a captivating story. The player has a certain quantity of energy whichs/he will see progressively decreasing depending on his/her choices. he can recoverenergy while answering the questions which relate to the whole of the knowledgewithin the framework of course NF01.

4.3.4 Availability

http://www4.utc.fr/~nf01


Comments on Part C of PRET or on PRET in generalUniversité de Technologie de CompiègneNo peer to peer interactionNo social aspectEmotional aspects taken into account by the use of a gameGood cognitive aspectsGood human-computer interfacePoor educational model (not a strong theory, but more heuristics)

Factors that contribute to the strength of the TELEUniversité de Technologie de CompiègneStructurationOrganisational aspects : SRL to prepare the quizEasy access to course grainsMany exercises and assessment tools

TELE Report

42

Good man-machine interface

Universität zu KölnThe TELE is integrated into a context of face-to-face and distance learning, thus the learners use it asa supplementing tool to deepen their knowledge acquired during the lectures and the practical work.The learners are allowed to learn at their own pace, and at a time convenient to themThey are given the opportunity to re-do the assessments.The TELE is highly motivating because of its use of many colours, simulations, animations, games,stories and riddles that are to be solved with the knowledge acquired before.It is free and uses standard web technology.

Factors that represent weaknesses of the TELEUniversité de Technologie de CompiègneNo social aspectNo forum

Universität zu KölnThere is no possibility to work together with a peer on the TELE, nevertheless, the peers canexchange their thoughts in the classes.The learning strategies explicitly presented are restricted to the examples given in the section “course”

Suggestions for improving the TELEUniversité de Technologie de CompiègneAdd communication toolsAdd a forumAdd exercisesCreate collaborative work and collaborative gamesUsing ontologies - domain : documents, persons…- application : programming and algorithms concepts and relations

TELE Report

43

4.4 SWIM – Online tutorial for information seekingAalborg University

4.4.1 Streaming Web-based Information Modules (SWIM)

Why SWIM? SWIM is a TELE intending to help students reflect on their information-seeking behaviour. This helps them to become self-regulated information seekerswithin problem-based learning, that is the pedagogical model of Aalborg University.SWIM is Aalborg University library’s streaming web-based information module. Thisis a service available on the library’s website, taught as part of the introduction to thelibrary services offered to students and new users in general.

To be more specific SWIM is a game simulation, intending to give “the students/userof the program a higher degree of information competence that can be utilizedindependently of specific information resources, time, and place. In addition, thisincreased level of competence can be used to support [their] life-long learningprocess… SWIM is intended to communicate an appropriate strategy for retrievinginformation that is generally applicable. Also, to communicate to its users informationretrieval techniques that are easy to understand and can be utilized when searchingfor information in connection with studies, projects, essays, assignments, etc.”(SWIM pedagogical manual, p. 3).

4.4.2 Features of SWIM

SWIM has three primary components:

• The role-playing simulation of project work that exposes the student to generalinformation-seeking processes and problems.

• Information modules that expose the student to specific information on thedifferent stages and problems of information-seeking.

• A pedagogical manual that has suggestions regarding use of SWIM indifferent contexts.

The information modules and the pedagogical manual are primarily text-based anduse relatively traditional materials. In this overview of the features of SWIM, we thuswant to focus on the features of the role-playing game.

The game is designed to reach its goal on the basis of two premises: It should buildon a theoretical understanding of information-seeking, and relate to group organizedproblem-based learning that is the pedagogical model of Aalborg University. This haslead to a game in which the player is a member of a virtual project group with threevirtual members. These members of course represent different approaches toinformation seeking.

During the game video sequences present the player with discussions regardinginformation seeking between the other three members of the group. Each of the threemembers take positions in the discussion according to the information-seekingpersonality they are given in the game. All discussions end with the virtual groupturning to the player for a decision on which information-seeking approach to choose.

TELE Report

44

The game is composed of six situations. In the first five scenes the player has todecide on an information-seeking strategy. In the last scene the group receivesfeedback and is graded at the project exam. The feedback tells the player about prosand cons of the different information-seeking choices throughout the game. Thisrepresents a working and learning process that new students at Aalborg Universityare becoming familiar with.

4.4.3 Benefits of using SWIM

Besides the obvious benefits of improving the information-seeking skills of studentsthe design of SWIM is a benefit in the way that it accelerates student learning. It doesso by involving students in a situation that is as close to real life as possible, and byinviting the student to reflect on the simulation of information-seeking practice.Compared to the old practice in which a librarian gave the students a lecture followedby information-seeking exercises, this is an improvement.

SWIM also offers a benefit by enabling distributed and flexible use. SWIM can beused in a traditional computer lab with a librarian available, at home by the studentsitting alone at his/her computer, or in a project room by a smaller group of studentspreparing for a project.

4.4.4 Implementation

4.4.4.1 Technical requirements

SWIM is a web-application that can be distributed from any web-server. The web-browser used to access SWIM must be Flash enabled because SWIM is developedin the Macromedia Flash MX environment.

SWIM is available in two video qualities: one suitable for at least 512 Kbps internetconnections (typical ADSL or cable TV) and one suitable for connections of at least 1Mbps internet connections.

4.4.4.2 Price and availability

SWIM is developed by Aalborg University Library (http://www.aub.aau.dk) andf u n d e d b y t h e D a n i s h E l e c t r o n i c R e s e a r c h L i b r a r y(http://www.deff.dk/index_eng.html) - a governmental institution. It is thus freelyavailable to Danish institutions. Other institutions can negotiate a fee with AalborgUniversity Library.

The program is available in English onhttp://www.aub.auc.dk/swim/swim_uk/adsl/splashintro.html.



It is fun, it is inviting

TELE Report

45


Does not discuss the underlying model of good behavior

Suggestions for improving the TELEThe history of the choices should be made available to the user

TELE Report

46

5. Concluding remarks

5.1 Complexity of the task

In the TELEPEERS project, we studied a very wide range of Technology EnhancedLearning Environments. The different TELE in this project range from CD-ROM andFlash tutorials, through weblogs and databases courses, to complete educationalprograms lasting several years. This enriches the project but also makes it hard tocompare the analyses.

Also, the concept of Technology Enhanced Learning Environments is a rather wideone. A CD-ROM with a programme to learn a foreign language which is used byindividuals in their spare time is a TELE. If the same CD-ROM is used in a course atschool, university or an other educational institution, then the whole course is theTELE. A TELE should therefore not be reduced to its technological component,rather, it is the complete learning environment.

Hence, the project team was led to analyse the role of technology in a broad contextof both learning strategies and educational resources. While technologies under usewere easy to categorise, educational approaches displayed a wide variance in natureand scope.

To illustrate the variety of situations under review it is necessary to bear in mind thefollowing four dimensions of TELEs:

a) While some partners licensed currently available commercial software(especially LMS) to which they added the value of original educational inputsothers developed their own technologies from scratch that served theirlearning purposes.

b) TELEs under experimentation could either apply to the entire programme ofstudies (i.e. a full graduate course) or else be used to the teaching of a singlesubject of a wider course.

c) Projects under review stretch across very different phases of deployment:some are simple pilots, others have matured to scalable applications, and afew are undergoing major re-orientation.

d) In some instances the partner is the core implementor of the TELE while inother cases the partner is investigating an educational development that wasdesigned and executed under the responsibility of another department oragency of the University.

It was therefore necessary to arrive at a common language and pull togethertransversal concepts.

Diversity also existed among the reviewers. Although each partner is an expert in thefield of Technology Enhanced Learning Environments, there were clear differences inperspective when it came to evaluating specific TELEs. One of our most strikingobservations were the differences in evaluations produced by system developers andend users. End users tended to be satisfied when the TELE they used met theirneeds, while system developers were much more critical in their evaluations. A clearexample of this are the different evaluations of the sunpower program.

TELE Report

47

It should be pointed out that the objective of this study was not to conductevaluations of the TELEs in themselves. Rather, we were interested to what extentthe TELE we evaluated showed a potential for supporting self-regulated learning(SRL). Very generally speaking, we came to the conclusion that the TELEs weevaluated all seemed to possess this potential to some degree although there werealso differences between the different TELEs.

5.2 Considering learners, teachers and other protagonists

It is, however, true that the extent to which the potential for supporting SRL is actuallymade use of also depends on the particular students and on the particular learningenvironment in which the TELE is embedded.

We are presently developing a tool that can be used by students to evaluate to whichextent they feel a particular TELE does indeed support their self-regulated learning.We already have, however, some findings on this question from other sources.

Observing students working with some of the TELEs that were being studied, theDutch partner (Vrije Universiteit Amsterdam – VUA) found that

• a TELE scores higher on SRL aspects if these are built explicitly into thecourse design, if students are made aware that SRL is regarded as aprofessional competence etc. The educational context and activities are in thatrespect more important than the technical features of the TELE to realise aSRL learning environment.

• if SRL is an explicit part of a TELE, however, then students are more aware ofthe technical features that are needed to facilitate SRL.

• sometimes, technical features present for e.g. collaboration are not used tofacilitate the (e.g. social) aspects of SRL (this relates back to course aims anddesign).

• a TELE may facilitate one or more phases of the cycle of SRL, dependingagain on the course aims and educational context. E.g. some TELEs may bemore focused on the planning of learning activities, others on the monitoring ofacquired skills and competencies while still other TELEs may facilitate thewhole cycle. This applies to social, cognitive, emotional and motivationalaspects of SRL:.

• some TELEs are focused on the social aspects of SRL, e.g a CMC, otherfocus on the cognitive aspects (e.g. a website). The emotional andmotivational aspects are less clearly present in all TELEs, more in the coursedesign, guidance and assessment methods, etc. and implicitly in the individuallearning experience of the student. It might be worthwhile to explore theconcept of metamotivation in this contect (possibly measuring this via studentquestionnaires/narrative approach).

• Ownership of a TELE in the context of SRL is also an interesting issue toreflect upon. Many TELEs are tutor-driven and regulated. In the case ofweblogs and to a lesser extent portfolio's, ownership is very much in the handsof the student who decides on structure, type of usage and user access, whilstthe context is one of reflection on the learning process.

Since the objective of TELEPEERS was not to conduct evaluations of the TELEs assuch but of its potential to support self-regulated learning, the Universidade Católica

TELE Report

48

Portuguesa (UCP) felt the need to further research the self-regulation aspect byconducting a motivational questionnaire to students. The main objective of thisquestionnaire was to pinpoint what factors contributed to self-regulation and whatwas the motivational level of students throughout the course.

From the questionnaire the Portuguese partner was able to draw the followingconclusions:

• Technology is not an important motivation factor: UCP used two very differentLMSs and the motivational level did not change significantly because of theLMS. This is because much of the characteristics that promote self-regulationare not embedded in the technology. That was in fact one of the problems ofusing the PRET to evaluate our TELE. Much of the work being done aroundself-motivation is invisible to someone who is not an active part of thecommunity;

• The motivational level seems to be a function of time and trust in the personalcompetences. This is especially relevant in UCP’s master which is quiteextensive and demanding on students commitment: After a high initialmotivation level the hard work of the first term made it decrease. With timeand trust in the learning processes the motivation gradually rises.

• Students classified the several activities as following: 65% (level 4&5) forums;66% (level 4&5) chats; 68% (level 4&5) on class sessions; 89% (level 5&6,with 46% in 6) multimedia resources. The pedagogic resources seems to playa decisive role in ODL but the rating for forums and chats raised some doubts,especially given the investment made on this functionality and the fact that thepedagogical model used it as one of its main pillars. For the vast majority ofthe students (84,6%) this was their first experience in distance learning andthat help to be able to look at these results from a different perspectives. FromUCP’s point of view, students were not familiarized with the distance learningmodel that was implemented and as consequence did not value the use offorums or chats. UCP hopes that by the end of the course this evaluation haschanged and that the value of forums and chats has been made clear tostudents. The questionnaire will be applied again to validate this.

• Globally students consider that all functionalities are important for learning:Forums – 60% (level 4 & 5); Chats – 68% (“); E-mails – 80% (“); Faceto face sessions – 68% (“). The high rating for e-mail will also have to bestudied in depth. And as for chats its was known that students were not usingthe LMS chat but MSN which all of them had and which was a much morefamiliar tool. It was decided to conduct in depth interviews using a forumformat to clarify these questions. But the interviews were not particularlyconclusive. UCT is still trying to further analyze the results. This particularaspect has even led the evaluators to suggest as improvement of the TELE tofind strategies to track other communication tools and collaborative learningused by students such as Messenger (MSN), e-mails, etc. This could makethe contribution of the TELE, especially on the social learning aspect, morevisible.

To comply with the overall goal of the TELEPEERS project: to identify and evaluateTechnology Enhanced Learning Environments (TELEs) that support self-regulatedlearning, it was important to take the pedagogical context as well as the practice ofteachers and students into consideration. The Danish partner selected theapplication SWIM for evaluation because it is designed to fit with the pedagogical

TELE Report

49

model underlying teaching at Aalborg University (AU). This has turned out to be anadvantage for the project as a whole, since most teaching cultures are orientedtowards individual learning. This way SWIM has become a challenging critical caseby having team-teaching as its underlying paradigm, and by being used in a teachingculture, where peer learning plays a formatting role in the students habits of studying;furthermore the use of SWIM crosses organizational and cultural boundaries bybeing taught by the library staff, for purposes of supporting the curricula of universitystudy programs. SWIM is intended to support the goal of turning students into self-regulated learners, but in practice it is a teaching material, the use of which if framedby the very practice of teaching and learning, which should be reflected in theevaluation.

The studies at Aalborg University are based on problem based learning andorganized in semesters, Sept-Jan, Feb-June. Each semester consists of 15 ECTScoursework and 15 ECTS project work. The students form project groups of 4-5,around a problem chosen after 3 weeks of course introduction, and meet regularlywith a supervisor until they turn in the project report (end of December, end of May).Oral exams take place June and January, and since the grading is based on the oraldefense of the written report, the project groups, after turning in the project report,prepare intensively for the orals The students are of course required by theirsupervisors to conduct literature surveys and information seeking from their firstsemester onwards. They have scheduled introduction to the library services as partof curriculum on 1st, 3rd and 5th semester.

The library staff uses SWIM as part of the scheduled introduction on semester 3rd.Each semester the students have the project writing and the project exam as theirprimary concern and driver of study activity. They run full circle from problemformulation, re-formulation, re-reformulation until the deadline cuts off the process.As assumed by the designers behind SWIM, the project work falls into three ratherdistinct phases in time and in type of activities: the exploratory phase, the inquiringphase and the concluding phase. Information search is put on the agenda at leasttwice: in the transition from exploration to inquiry and in the transition from inquiry toconclusion. The librarians serve their customers in three ways: a. ad hoc support, b.offering planned sessions where students sign up for an hour of consultation, send ininformation problems from the project work, the librarian prepares advice, they meet,and the librarian helps out in the concrete through making the student conductadvanced search in databases etc., c. 3-hour classes of teaching for a wholesemesters contingent of students, incl. SWIM.

The librarians see two missions for their work. The first is the classical ‘the right bookto the right reader at the right time’. The second, more postmodern, is to provide thecustomer with sufficient skill to be self-regulated information seeker. Wheninterviewed the librarians talk mostly about the second, but an analysis of thediscourse underlying what they say reveals a strong devotion to the former, modern,mission – which is easily understood when observing librarian-student encounters:the librarians get little recognition for training students to become self-regulatedinformation seekers, since their effort can be misunderstood as rejection, while theyget – a lot – of instant gratification by helping people fulfilling their information needs.

As far as the teachers are concerned, the demands of the curriculum are fulfilled bymaking sure the students are sent to the library teaching sessions as planned and

TELE Report

50

the supervisors are concerned mostly that the report meets the standard with respectto literature survey and list of references, otherwise they trust the library to teach thestudents how to meet these demands. When students come and ask for help to findinformation, supervisors tend to give out references and sometimes even lend outbooks, since that is the easiest way for them to hand over to the students theinformation they find of most relevance. And again the supervisor gets more instantgratification this way than be sending the students back to the search procedures ondatabases.

The design team behind SWIM has been a professional director of simulation gamesas a consultant, and a librarian with system development as his special area ofcompetence, and a team of professional developers, using professional actors andcomposer etc. The budget has not allowed user testing, so the librarians knowledgehas been filled in there.

With respect to the reception amongst students, Aalborg has conducted number of indepth interviews with off campus students about their information seeking behavior. Itturns out that they pay little attention to SWIM, even though SWIM is prominentlypresent on the library’s website and mentioned in the study introduction. Theconclusion of a survey of on campus students’ satisfaction, being introduced toSWIM as part of the library’s teaching, shows however a positive attitude whendirectly asked about the impression of SWIM. This must, however, be held up againsta quite strong belief in the students ability to search information on their own. 87% ofthe students in the third semester always almost always seek information without anyhelp from teachers or other advisors. In the same semester 97% say that they areable to plan information seeking on their own. This is a development compared to thefirst semester where 66% always almost always seek information without any helpfrom teachers or other advisors and 87% say they are always or almost always ableto plan information seeking on their own. This indicates that the motivational qualityof the game structure in SWIM is worth valuing, since it seems quite difficult toconvince the students that they really need to learn about information seekingbehavior.

5.3 Evaluating the Peer Review Evaluation Tool (PRET)

Developing the Peer Review Evaluation Tool (PRET) and identifying and evaluatingTechnology Enhanced Learning Environments with regard to their potential tosupport self-regulated learning were two activities in the project that were undertakenat almost the same time. Both activities influenced each other. Working on the PRET,we had to develop an evaluation framework that was general enough to encompassthe different TELEs that we had in mind, and looking at the TELEs we became awareof the complexity of each TELE whose characteristics we wanted to capture. In thisprocess, we continually tried to improve the fit between the TELEs that we wereevaluating and the PRET that we were developing as an evaluation tool.

As a result of the evaluation activities, the partners basically agreed that the PRETwas a useful instrument for evaluating the potential of TELEs to support self-regulated learning. They are also aware that at the moment, there is not other toolsthat can be used for this kind of evaluation. Notwithstanding this basically positiveevaluation, critique and suggestions for further improvement were also voiced.

TELE Report

51

Aalborg University, for instance, used the PRET for the evaluation of their SWIMTELE and found that the PRET exhibits some shortcomings with respect to taking thespecifics of a pedagogical context and culture of learning into account. Hence, fromtheir point of view, the PRET should be revised and improved to take intoconsideration these aspects, in order to fit evaluation also of TELEs that aim tosupport team based learning across organizational boundaries.

Aalborg and Bergen partners conducted expert walk throughs of SWIM and Bergensblog-software using the PRET. The main conclusion with respect to using PRET asan instrument for evaluation was that for expert walk throughs – which is thetechnical term for a heuristic evaluation based in evaluators’ competence in the fieldof the product – there is no need for the numerical summarizing, but the scalingsupports communication well. Subscribing to a situated, practice oriented approachto the study of learning, these evaluators have reservations about the relevance ofseparating out planning and execution from evaluation since it is experienced as onegoal directed action. Such separation fits laboratory testing, but is less illuminatingwhen data are collected through studies of practice, or through questionnaires, andthe students will have a hard time distinguishing in the same way. The separationbetween motivational, emotional and social aspects is equally difficult to handleoutside the laboratory setting. With respect to the issue of transferability theseevaluators find that examples with click boxes attached would be better. Havingpractice and situation as point of departure, these evaluators suggest the PRET to beexpanded by also addressing collective routes to self regulation such as the role ofmore capable peers.

Explicit suggestions for improving the PRET also came from our French partner, theUniversité de Technolgie de Compiègne (UTC) who had conducted a peer review ofthe SUNPOWER TELE.

From their point of view, the time constraints imposed on evaluating the TELEs madeit difficult to do an in-depth evaluation of SUNPOWER and to cover all aspects ofassessment. Moreover, the questions of PRET did not provide enough criteria forcomprising all the theoretical aspects of the questionnaire. The program was notevaluated by students, which respectively did not evoke the user’s view. Theevaluation of the contents of its message (strategies, conversations, dialogues, tests)was not full, as there were no specialists in English or business strategies involved togive a rather pedagogical opinion of the program’s efficacy and usefulness. Thus,UCT intends to evaluate again this TELE, taking more time and asking studentsopinions. It should be noted as well that a more complete and correct evaluation willfollow, as two more evaluators will be involved who are specialists in Englishlanguage and who will give a different view on the quality of the program.

As strong points of the program, UCT lists the motivational aspects, the self-regulated learning, and the well-developed audio-visuals. An attractive point ofSUNPOWER is represented by the conversations, which give the impression theyare real and are directly taken from real enterprises. However, some factors canbring weaknesses to the system and can have a strong influence during theevaluation of the TELE: they are the ‘Human Computer Interface’ and the‘navigation’. The navigation is implicit at the beginning as it is hard to orientate

TELE Report

52

among the choices and points to start with. It also provides a poor interface of theprogram, because it lacks homogeneity and stability in the icons and actions.

UCT therefore suggests to take into account some of these factors, by adding newquestions in the PRET, such as :

• Does the application answer quickly to the user actions?• Are images and sounds loaded quickly in the web site?• Are the pages optimized for the loading and the presentation?• Did it happen to you not to know what to do, or how to do, for continuing to

navigate the site?• Are the icons, the images or the used symbols sufficiently explicit?• Can the user configure interfaces according to his work practices?• Does the site provide the user with the means to know where he is and what is

left for him to do?• Is it possible to clearly identify the functioning and the possibilities of the site

(icons, navigation menus, etc.)?

5.4 Outlook

To study a wide range of TELEs in great a variety of situations is certainly a task ofhigh complexity. It was therefore not to be expected that in the TELPEERS project,we should come up with simple answers and clear-cut results. Rather, the results ofour evaluations form a mosaic, a multi-facetted picture.

It there is anything like a simple conclusion, then it is expressed in our conviction thatthe originality of the TELEPEERS project does not lie in the different TELEs that weidentified and evaluated, but in the Peer Review Evaluation Tool which wedeveloped. The PRET represents a tool useful for the evaluation and characterizationof TELEs with respect to its potential to support self-regulated learning. It is basicallythis unique approach that we hope will be transferred.

TELE Report

53

5. List of contributors

Universität zu Köln (Germany):Karl Steffens, Kathrin Petzold

Vrije Universiteit Amsterdam (The Netherlands):Jos Beishuizen, Patris van Boxel

Universidade Católica Portuguesa (Portugal):Roberto Carneiro, Carla Ganito, José Lagarto

Aalborg University (Denmark): Ellen Christiansen, Tom Nyvang

CNR Istituto per le Tecnologie Didattiche (Italy):Giuliana Dettori, Tania Giannetti, Donatella Persico

Nottingham Trent University (United Kingdom):Jean Underwood, Allison Ault, Phil Banyard

Universitat de Barcelona (Spain) :Antonio Bartolomé, Martín Aiello, Celia Willem

Université de Technologie de Compiegne (France) :Philippe Trigano, Marie Héléne Abel

Universitetet i Bergen (Norway):Rune Baggetun

TELE Report

54

Annex 1: Peer Review Evalutation Tool (PRET)

TELE Report

55

PRET

Peer Review Evaluation Toolfor Technology Enhanced Learning Environments (TELEs) that Support Self-

Regulated Learning

Version 6.0

October 25, 2004

Universität zu Köln / Germany,Vrije Universiteit Amsterdam / The Netherlands,Universidade Católica Portuguesa / Portugal,Aalborg University / Denmark ,Istituto per le Tecnologie Didattiche / Italy ,Nottingham Trent University / U.K.Universitat de Barcelona / Spain ,Université de Technologie de Compiegne / France,University of Bergen / Norway.

This tool was developed within the project “Self-regulated Learning in TechnologyEnhanced Learning Environments at University Level: a Peer Review(TELEPEERS)”. The project is being carried out with the support of the EuropeanCommission (Grant agreement 2003-4710-/001-001 EDU-ELEARN). The content ofthis project does not necessarily reflect the position of the European Commission,nor does it involve any responsibility on the part of the European Commission.

TELE Report

56

Part A: Technical description of TELE

Objective ofTELE

Educationalmodel

Mode of delivery

Platform No platform usedCommercial platform usedProprietary platform used

Parties involved

Places oflearningContext

Level ofinteractivity

No interactivityInteractivity with TELEInteractivity with tutorInteractivity with peers

Technicalrequirements

Operating system (server/client)Hard disk space requiredDisplay resolution, colour depthHard ware components(working memory, processor,sound card, graphics card)External devicesConnectivity (e.g. type ofbrowser, band width)

Importance oftechnology forTELE

Low importance 0 1 2 3 4 5 High importance

Accompanyingdocumentation

User’s handbook (printed or in electronic form) for teachersUser’s handbook (printed or in electronic form) for studentsTutorialGuidelinesHelp function

Transferability(Portability)

CostsLanguageLegal aspects

Technical aspects (seetechnical requirements)Limitations in using TELE

TELE Report

57

Part B: Detailed evaluation of support for self-regulated learning

Evaluation of support given to self-regulated learning

Please try to evaluate to which extent the TELE you are working with supportsself-regulated learning by facilitating (1) planning, (2) executing and monitoring and(3) evaluating learning activities.

Please feel free to comment a statement if you think that it needs to beimproved.

(1) Planning

Cognitive aspects

The TELE helps the learner to structure the learning content.Not supported 0 1 2 3 4 5 well supported

The TELE has an easy and intuitive interface.Not supported 0 1 2 3 4 5 well supported

The TELE records a history of learner activities.Not supported 0 1 2 3 4 5 well supported

The TELE allows the student to plan her/his learning with the help of activityplans, personal development plans, progress reports etc.Not supported 0 1 2 3 4 5 well supported

The TELE provides the student with the opportunity to choose betweendifferent course modules.Not supported 0 1 2 3 4 5 well supported

The TELE provides the student with the opportunity to choose betweendifferent learning paths.Not supported 0 1 2 3 4 5 well supported

The TELE provides the student with the opportunity to choose betweendifferent modes of delivery.Not supported 0 1 2 3 4 5 well supported

Motivational aspects

The TELE is likely to arouse the learner’s interest.Not supported 0 1 2 3 4 5 well supported

The TELE allows each student to personalize the interface used in theenvironment.

TELE Report

58

Not supported 0 1 2 3 4 5 well supported

The TELE allows the student to set personal learning goals.Not supported 0 1 2 3 4 5 well supported

The TELE helps the learner plan her/his activities by pointing out to her/himwhich external resources (websites, help options) are available.Not supported 0 1 2 3 4 5 well supported

The TELE reminds the learner of her/his own knowledge and skills relevant tothe task at hand.Not supported 0 1 2 3 4 5 well supported

The TELE sensitises the learner with respect to how the problem might besolved.Not supported 0 1 2 3 4 5 well supported

There are explicit mechanisms in the TELE to encourage the learner to tacklethe task.Not supported 0 1 2 3 4 5 well supported

There are implicit mechanisms in the TELE to encourage the learner to tacklethe task.Not supported 0 1 2 3 4 5 well supported

Emotional aspects

The TELE helps the learner to cope with the challenges of the task.Not supported 0 1 2 3 4 5 well supported

The TELE may be adapted to reach a congruence between the learner’s levelof competence and the level of difficulty of the task.Not supported 0 1 2 3 4 5 well supported

The TELE is organised in such a way that the learner enjoys working in it.Not supported 0 1 2 3 4 5 well supported

Social aspects

The TELE offers the possibility to set up both public and privatecommunication.Not supported 0 1 2 3 4 5 well supported

The TELE provides the learner with the opportunity to negotiate with her/histutor/instructor how to organise her/his work.Not supported 0 1 2 3 4 5 well supported

The TELE allows the learner to work together / communicate with her/hispeers.Not supported 0 1 2 3 4 5 well supported

TELE Report

59

(2) Executing and monitoring

Cognitive aspects

The TELE allows the user to make decisions on how to proceed (e.g. skip asection, choose among different options).Not supported 0 1 2 3 4 5 well supported

The TELE allows the learner to reflect on her/his own problem solvingactivities.Not supported 0 1 2 3 4 5 well supported

The TELE provides the user with the possibility to find out to whatextent she/he is achieving her/his learning goals.Not supported 0 1 2 3 4 5 well supported

The TELE allows the learner to switch to another learning strategy ifnecessary.Not supported 0 1 2 3 4 5 well supported


The TELE provides the learner with strategies to maintain her/hismotivation.Not supported 0 1 2 3 4 5 well supported

The TELE provides help facilities that aim at strengthening the learner’sperseverance in case of failure.Not supported 0 1 2 3 4 5 well supported

Emotional aspects

The TELE provides the user with formative feedback that facilitates themaintenance of a positive working attitude.Not supported 0 1 2 3 4 5 well supported

The TELE provides the user with formative feedback that intervenes at criticalpoints in the learning cycle in order to restore a positive working attitude.Not supported 0 1 2 3 4 5 well supported

Social aspects

The TELE allows the user to contact and receive help from her/histutor/instructor.

TELE Report

60


The TELE provides the user with the opportunity to communicate with her/hispeers in order to exchange ideas or to ask for help.Not supported 0 1 2 3 4 5 well supported

The TELE provides the user with possibilities to collaborate with her/his peers.Not supported 0 1 2 3 4 5 well supported

(3) Evaluation

Cognitive aspects

The TELE helps the user to reflect on her/his learning progress.Not supported 0 1 2 3 4 5 well supported

The TELE encourages the learner to compare her/his present state with thestate she/he wanted to be in.Not supported 0 1 2 3 4 5 well supported

The TELE provides the learner with the means to assess her/his ownachievements.Not supported 0 1 2 3 4 5 well supported

The TELE allows the student to select the achievements to be assessed.Not supported 0 1 2 3 4 5 well supported

The TELE allows the student to select the competencies to be assessed.Not supported 0 1 2 3 4 5 well supported


The TELE provides the learner with feed-back that leads to appropriate self-efficacy beliefs.Not supported 0 1 2 3 4 5 well supported

Emotional aspects

The TELE provides the learner with appropriate feedback on her/his effortsand achievements.Not supported 0 1 2 3 4 5 well supported

Social aspects

The TELE provides the learner with the opportunity to compare her/his resultswith that of a tutor/instructorNot supported 0 1 2 3 4 5 well supported

TELE Report

61

The TELE allows the learner to discuss his results with her/his tutor/instructorNot supported 0 1 2 3 4 5 well supported

The TELE provides the learner with the opportunity to compare her/his resultswith those of her/his peers.Not supported 0 1 2 3 4 5 well supported

The TELE allows the learner to discuss his results with her/his peers.Not supported 0 1 2 3 4 5 well supported

Part C: Global evaluation of support for self-regulated learning

Part C is an attempt to evaluate the TELE you are working with on a more generallevel. You can either do so by using the rating scales offered in this part or bycalculating a combined value (either unweighted or weighted means) of theresponses given to the corresponding statements in Part B.

If you would like to comment on the procedure suggested in Part C or on PRET ingeneral, please do so.

Comments on Part C of PRET or on PRET in general

TELE Report

62

Part C: Global evaluation of support for self-regulated learning

Evaluation ofsupport of self-regulated learning

(1) Planning

Cognitive aspectsNot supported 0 1 2 3 4 5 well supported

Motivational aspectsNot supported 0 1 2 3 4 5 well supported

Emotional aspectsNot supported 0 1 2 3 4 5 well supported

Social aspectsNot supported 0 1 2 3 4 5 well supported

Planning global evaluationNot supported 0 1 2 3 4 5 well supported

(2) Execution and monitoring





Execution and monitoring global evaluationNot supported 0 1 2 3 4 5 well supported

(3) Evaluation





Evaluation global evaluationNot supported 0 1 2 3 4 5 well supported

TELE Report

63

Cognitive aspects Not supported 0 1 2 3 4 5 well supportedMotivational aspects Not supported 0 1 2 3 4 5 well supported

Emotional aspects Not supported 0 1 2 3 4 5 well supported

Social aspects Not supported 0 1 2 3 4 5 well supported

Overall evaluation ofsupport for SRL Low support 0 1 2 3 4 5 High supportEvaluation ofexplicitness Support implicit 0 1 2 3 4 5 Support explicit



Suggestions for improving the TELE

TELE Report

64

Annex 2: Summary of the quantitative PRET results

TELE Report

65

DigitalPortfolio

Self-evaluation by Vrije Universiteit Amsterdam


(1) Planning












(3) Evaluation






TELE Report

66

Digital Portfolio Peer review by Nottingham Trent University


(1) PlanningCognitive aspects




Emotional aspects


Social aspects


Planning global evaluation


(2) Execution and monitoringCognitive aspects




Emotional aspects


Social aspects


Execution and monitoring global evaluation


(3) EvaluationCognitive aspects




Emotional aspects


Social aspects


Evaluation global evaluation


TELE Report

67

DiViDU Self-evaluation by Vrije Universiteit Amsterdam


(1) Planning












(3) Evaluation






TELE Report

68

DiViDU Peer review by Universität zu Köln


(1) Planning












(3) Evaluation






TELE Report

69

ILIAS Self-evaluation by Universität zu Köln


(1) Planning












(3) Evaluation






TELE Report

70

WeblogsSelf- evaluation by Bergen = yellowPeer review by Amsterdam = violet and Aalborg = green

Evaluation ofsupport of self-regulatedlearning

(1) Planning


Motivational aspectsNot supported 0 1 2 3 4 4 5 well supported


Social aspectsNot supported 0 1 2 3 4 4 5 well supported





Emotional aspects Not supported 0 1 2 3 4 5 well supported

Social aspectsNot supported 0 1 2 3 4 4 5 well supported


(3) Evaluation

Cognitive aspectsNot supported 0 1 2 3 3 4 5 well supported

Motivational aspectsNot supported 0 1 2 3 3 4 5 well supported

Emotional aspectsNot supported 0 1 2 3 3 4 5 well supported



TELE Report

71

AdvancedTeacherEducation inInformatics

Self-evaluation by Universidade Católica Portuguesa


(1) Planning – Comments






(2) Execution and monitoring - Comments






(3) Evaluation - Comments






TELE Report

72


Peer review by Universitat de Barcelona



Cognitive aspectsNot supported 0 1 2 3 (3,2) 4 5 well supported



Social aspectsNot supported 0 1 2 3 4 (4,66) 5 well supported

Planning global evaluationNot supported 0 1 2 3 (3,10) 4 5 well supported






Execution and monitoring global evaluationNot supported 0 1 2 3 (3,33) 4 5 well supported

(3) Evaluation -

Cognitive aspectsNot supported 0 1 2 (2,3) 3 4 5 well supported




Evaluation global evaluationNot supported 0 1 2 (3,33) 3 4 5 well supported

TELE Report

73


Peer review by CNR Istituto per le Tecnologie Didattiche


(1) Planning












(3) Evaluation






TELE Report

74

Cognitivepsychologycourse

Self-evaluation by Nottingham Trent University


(1) PlanningCognitive aspects




Emotional aspects


Social aspects


Planning global evaluation


(2) Execution and monitoringCognitive aspects




Emotional aspects


Social aspects


Execution and monitoring global evaluation


(3) EvaluationCognitive aspects




Emotional aspects


Social aspects


Evaluation global evaluation


TELE Report

75


Peer review by Vrije Unviersiteit Amsterdam


(1) Planning












(3) Evaluation






TELE Report

76


Peer review by CNR Istituto per le Technologie Dittatiche


(1) Planning












(3) Evaluation






TELE Report

77

Digital VideoCourse Peer review by Universidade Católica Portuguesa




















TELE Report

78

Teacher TrainingCourse on ET

Self-evaluation by CNR Istituto per le Tecnoligie Dittatiche




















TELE Report

79


Peer review done by Universidade Católica Portuguesa




















TELE Report

80


Peer review by Universitat de Barcelona




Motivational aspectsNot supported 0 1 2 3 (3,25) 4 5 well supported



Planning global evaluationNot supported 0 1 2 3 (3,36) 4 5 well supported


Cognitive aspectsNot supported 0 1 2 3 (3,75) 4 5 well supported




Execution and monitoring global evaluationNot supported 0 1 2 3 (3,45) 4 5 well supported


Cognitive aspectsNot supported 0 1 2 (2,2) 3 4 5 well supported




Evaluation global evaluationNot supported 0 1 2 (2,8) 3 4 5 well supported

TELE Report

81

Sunpower Self-evaluation by Universität zu Köln


(1) Planning












(3) Evaluation






TELE Report

82

Sunpower Peer review by Nottingham Trent University


(1) PlanningCognitive aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (3.6)(all statements)Not supported 0 1 2 3 4 5 well supported (2.6)

Motivational aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (3.7)(all statements)Not supported 0 1 2 3 4 5 well supported (2.8)

Emotional aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (4)(all statements)Not supported 0 1 2 3 4 5 well supported (1.3)

Social aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (0)(all statements)Not supported 0 1 2 3 4 5 well supported (0)

Planning global evaluation (only those scored)Not supported 0 1 2 3 4 5 well supported (3.7)(all statements)Not supported 0 1 2 3 4 5 well supported (2.1)

(2) Execution and monitoringCognitive aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (4.75)(all statements)Not supported 0 1 2 3 4 5 well supported (4.75)

Motivational aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (4)(all statements)Not supported 0 1 2 3 4 5 well supported (4)

Emotional aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (4)(all statements)Not supported 0 1 2 3 4 5 well supported (2)


Execution and monitoring global evaluation (only those scored)Not supported 0 1 2 3 4 5 well supported (4.4)(all statements)Not supported 0 1 2 3 4 5 well supported (2.8)

TELE Report

83


(3) Evaluation

Cognitive aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (5)(all statements)Not supported 0 1 2 3 4 5 well supported (4)

Motivational aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (5)(all statements)Not supported 0 1 2 3 4 5 well supported (5)

Emotional aspects (only those scored)Not supported 0 1 2 3 4 5 well supported (4)(all statements)Not supported 0 1 2 3 4 5 well supported (4)


Evaluation global evaluation (only those scored)Not supported 0 1 2 3 4 5 well supported (4.8)(all statements)Not supported 0 1 2 3 4 5 well supported (2.6)

TELE Report

84

Sunpower Peer review by Université de Technologie de Compiègne


(1) Planning












(3) Evaluation






TELE Report

85

Databases Self-evaluation by Université de Technologie de Compiègne


(1) Planning

Cognitive aspectsNot supported 0 1 2 3 4 5 well supported 4

Motivational aspectsNot supported 0 1 2 3 4 5 well supported 3

Emotional aspectsNot supported 0 1 2 3 4 5 well supported 3

Social aspectsNot supported 0 1 2 3 4 5 well supported 4

Planning global evaluationNot supported 0 1 2 3 4 5 well supported 3,5






Execution and monitoring global evaluationNot supported 0 1 2 3 4 5 well supported 4

(3) Evaluation





Evaluation global evaluationNot supported 0 1 2 3 4 5 well supported 2,5

TELE Report

86

Databases Peer review by Universität zu Köln


(1) Planning

Cognitive aspectsNot supported 0 1 2 3 4 5 well supported = 4.75

Motivational aspectsNot supported 0 1 2 3 4 5 well supported = 4.1

Emotional aspectsNot supported 0 1 2 3 4 5 well supported = 4.3

Social aspectsNot supported 0 1 2 3 4 5 well supported = 0

Planning global evaluationNot supported 0 1 2 3 4 5 well supported = 4.3




Emotional aspectsNot supported 0 1 2 3 4 5 well supported = 0


Execution and monitoring global evaluationNot supported 0 1 2 3 4 5 well supported = 4.37

(3) Evaluation

Cognitive aspectsNot supported 0 1 2 3 4 5 well supported = 5

Motivational aspectsNot supported 0 1 2 3 4 5 well supported = 4



Evaluation global evaluationNot supported 0 1 2 3 4 5 well supported= 4.5

TELE Report

87

ProgrammingTutorial

Self-evaluation by Université de Technologie de Compiègne


(1) Planning












(3) Evaluation






TELE Report

88

ProgrammingTutorial

Peer review by Universität zu Köln


(1) Planning



Emotional aspectsNot supported 0 1 2 3 4 5 well supported = 4.66


Planning global evaluationNot supported 0 1 2 3 4 5 well supported = 4.48






Execution and monitoring global evaluationNot supported 0 1 2 3 4 5 well supported = 4.5

(3) Evaluation

Cognitive aspectsNot supported 0 1 2 3 4 5 well supported = 5




Evaluation global evaluationNot supported 0 1 2 3 4 5 well supported = 4.5

TELE Report

89

SWIM Self-evaluation by Aalborg University


(1) Planning

Cognitive aspectsNot supported 0 1 2 X 4 5 well supported

Motivational aspectsNot supported 0 1 2 3 X 5 well supported

Emotional aspectsNot supported 0 1 2 X 4 5 well supported

Social aspectsNot supported 0 1 2 X 4 5 well supported

Planning global evaluationNot supported 0 1 2 X 4 5 well supported



Motivational aspectsNot supported 0 1 2 3 X 5 well supported

Emotional aspectsNot supported 0 1 2 3 X 5 well supported

Social aspectsNot supported 0 1 2 3 X 5 well supported

Execution and monitoring global evaluationNot supported 0 1 2 3 X 5 well supported

(3) Evaluation


Motivational aspectsNot supported 0 1 2 X 4 5 well supported


Social aspectsNot supported 0 1 2 3 X 5 well supported

Evaluation global evaluationNot supported 0 1 2 X 4 5 well supported

tele report version 7 - lmi.ub.es report report on technology enhanced learning environments (teles)...

Documents