DOCUMENT RESUME

ED 386 923 FL 023 104

AUTHOR Little, David; And OthersTITLE Report on the ESF-Funded Project to Consolidate and

Develop Foreign Language Modules for Students ofOther Disciplines Implemented October 1,1993-September 30, 1994.

INSTITUTION Dublin Univ. Trinity Coll. (Ireland). Centre forLanguage and Communication Studies.

PUB DATE Oct 94NOTE 30p.

PUB TYPE Reports Descriptive (140

EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS *College Second Language Programs; College Students;

Distance Education; Foreign Countries; French;German; Higher Education; Language Proficiency;*Program Descriptions; Program Evaluation; SecondLanguage Instruction; *Second Language Learning;Student Evaluation

IDENTIFIERS *Ireland

ABSTRACTThis report presents the development and results of a

I-year foreign language module program of 15-weeks duration foruniversity students in the arts, commerce, science, and engineering.The project's central concern was to develop and test a general modelfor the delivery of foreign language teaching to students of otherdisciplines. The program included audio-video learning units,satellite television facilities, and computers. Modules were designedto engage students in immediate, active language use and plactice ofthe target language (German or French) to provide a solid foundationin basic speaking, listening, reading, and writing skills; classesfocused on broadly defined domains of communicative activity.Assessment took place in weeks 5 and 6 and involved 113 students.Results indicate that students completing the modules attained asatisfactory level of language competence as measured by deploymentof communicative skills and general underlying proficiency. Externalevaluators rated the project as successful as well. Special fundingpermitted the modules to be continued for a second year. Appendixesinclude a list of related publications, an outline of the Frenchlanguage module for students of health sciences, a summary ofexpenditures (in British pounds), band descriptors and rating sheet,and a correlational analysis of test components. (NAV)

Reproductions supplied by EDRS are the best that can be madefrom the original document.

*********************************************************"..A.AAA**A

*

*

Report on the ESF-Funded Project to Consolidateand Develop Foreign Language Modules for Studentsof other Disciplines Implemented by the Centre forLanguage and Communication Studies1 October 1993-30 September 1994

PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

,

A \-,_

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

U S DEPARTMENT OE EDUCATION

EDOCATIONAt RESOURCES INFORMATIONCENTER (ER(C)

In,, document has been reproduced asreceived IrOM porsoo or wpm/Minn

ortabne, a

C.1 IdImm change!. have Iner-, macietinpOW repnoduttion quahly

Point% el woe. (1, n))nn.ons stated In thin,doCkrnent do nol no,es;anty lopre:,ontoff OEPI posit.r re Orrnir

University of Dublin Trinity College

Report on the [SF-funded project toconsolidate and develop foreign language modules for

students of other disciplinesimplemented by the

Centre for Language and Communication Studies1 October 1993-30 September 1994

Contents

1 introductionDavid Little,Director, CLCS

2 ImplementationEma UshiodaProject co-ordinator

3 External evaluationEdith Esch and Anny KingLanguage CentreUniversity of Cambridge

4 ConclusionDavid Little

Appendices

1.1 The Centre for Language andCommunication Studies

1.2 Background to CLCS's languagemodules project

1.3 The ;aunch of CLCS's languagemodules project

2.1 Consolidation of existing languagemodules

2.2 Recruitment of students to take thenew modules

2.3 Rates of participation

2.4 Design of the new modules2.5 Design of assessment procedures

2.6 Assessment results

3.1 Introduction3.2 Design of the pilot project

3.3 The pilot project in the context oflanguage learning provision in highereducation in Europe

3.4 Management of the project

3.5 Evaluative approach adopted for the project3.6 Recommendations for the future

Notes

4.1 Response to the external evaluators'recommendations

4.2 Provision of language modules in 1994-54.3 Research dimensions4.4 The future

1 Applied linguistic research in CLCS2 French module for students of

Health Sciences3 Summary of expenditure4 Band descriptors and rating sheet5 Correlational analysis of test components

Si

2

2

3

4

4

7

9

11

13

13

14

15

16

16

17

18

18

19

20

22

24

25

26

28

BEST COPY AVMLABLE

1 Introduction

1.1 The Centre for Languageand CommunicationStudies

The Centre for Language and CommunicationStudies (CLCS) was established in 1978 as aspecial development associated with thecommissioning of Trinity College's new Artsand Social Sciences Building.

From the beginning CLCS has had a threefoldfunction: (i) to provide technical support forlanguage teaching and learning; (ii) to engage in

theoretical and applied linguistic research; and(iii) to develop an appropriate teaching role.

For a number of years CLCS has providedcourses in theoretical and applied linguistics atundergraduate and postgraduate levels. InTrinity term 1990 the University Council madeCLCS responsible for co-ordinating theprovision of language teaching for students nottaking a foreign language as part of their degreeprogramme. The project that is the focus of thisreport represents CLCS's first involvement inlanguage teaching on a large scale.

Over the past fifteen years applied linguisticresearch in CLCS has focussed on manydifferent aspects of language learning andlanguage teaching, including: self-access andself-instruction; learner autonomy; interactivevideo for language learning; the definition of

language teaching syllabuses; the developmentof language learning materials; pedagogicalgrammar; the development of the secondlanguage mental lexicon in learners at unb.ersitylevel; the agr factor in language learning; thtrole of the mother tongue in language learning(Appendix 1, pp.22f. below, lists a selection of

relevant publications).

At every stage CLCS's language modulesproject has been informed by one Of anotherstrand of relevant applied linguistic research,

2

David Little

and as it has proceeded it has generatedresearch dimensions of its own. It is worthemphasizing that this is by no means a commonoccurrence. Even in universities languageteaching of the kind described in this report isall too often treated as a service provision whichneither needs nor deserves the benefits oftheoretical and empirical research.

1.2 Background to CLCS'slanguage modules project

In Trinity term 1993 the Higher EducationAuthority announced an EST-funded pilotproject for the restructuring of primary degreesand invited proposals from the universities. Oneof the three measures to be implemented wasthe introduction of foreign language modulesfor students in the Faculties of Arts, Commerce,Science and Engineering.

In its submission to the HEA Trinity Collegeproposed (i) to consolidate language modulesalready offeied by enhancing the facilitiesavailable to support them, and (ii) to establishnew language modules.

As regards consolidation, by the academic year1992-3 the following courses stood to benefitfrom the proposed project:

in the Faculty of Business, Economic andSocial Studies, partly integrated two-vearcourses in French (non-beginners) andGerman (non-beginners), taken by studentsin their junior and senior freshman (first andsecond) years;in the Faculty of Engineering and SystemsSciences, partly integrated two-year coursesin French (non-beginners) and German(beginners and non-beginners), taken bystudents in their senior freshman and juniorsophister (second and third) years;in the Faculty of Science, a non-integratedtwo-year course in German (beginners and

non-beginners), taken by students in theirsenior freshman and junior sophister years.

As regards new modules, the project's centralconcern Nvas to develop and test a general modelfor the delivery of foreign language teaching tostudents of other disciplines. In other words, itwas to be a major concern of the project to laydown firm foundations for the futuredevelopment of the undergraduate curriculumin this area. The proposal submitted to the HEAincluded the following new modules:

French for non-beginners in Science;French for non-beginners in Arts;German tor beginners in Arts;German for non-beginners in Arts.

A self-access programme in German for Sciencestudents was among the language modulesalready provided in Trinity College (see above),and for this reason the College's proposal didnot include German for Science students amongthe new modules. However, scarcely anyScience students enrolled in the existingprogramme at the beginning of the academicyear 1993-4, so the following new modules wereadded to the project:

German for beginners in Science;German for non-beginners in Science.

The fact that only French and German wereprovided for is a reflection of the dominance ofthese two languages in the school curriculumand in relation to ERASMUS student exchangeprogrammes. The latter dominance emergedclearly in a survey of foreign language needsthat was un&rtaken in Michaelmas term 1990by the Director of CLCS in consultation withfaculty deans. The situation had not changedwhen the exercise was repeated in Hilary term1993.

3

Students in the Faculty of Health Sciences werenot included in the population targeted by theHEA's pilot project. However, in Michaelmasterm 1993 the Faculty asked CLCS to provide aFrench module for non-beginners in Hilary andTrinity terms 1994 in order to supportERASMUS exchanges with institutions inFrance. The Faculty financed the module, whichwas organized within the general frameworkestablished for the ESF-funded modules. Forpurposes of comparison, a brief report on ratesof participation and assessment results isprovided in Appendix 2, p.24 below.

1.3 The launch of CLCS'slanguage modules project

The proposal that Trinity College submitted tothe HEA was based on a budget for 1993-4 of£168,000 and the assumption that additionalaccommodation could be made available tohouse new technical installations. In the event,the. HEA awarded the project a grant of £100,000and no additional accommodation could bemade available.

The project was launched with the approval ofthe University Council early in Michaelmas term1993. Ema Ushioda was appointed project co-ordinator, Marc Gallagher vas appointedpedagogical adviser, and Dr Edith Esch andAnny King of the Language Centre, Universityof Cambridge, were appointed externalevaluatom

The funding period of the project was from 1October 1993 to 30 September 1994. Appendix 3,p.25 below, provides a summary of expenditure.

5

2 Implementation

2.1 Consolidation of existinglanguage modules

Before the project began CLCS offered thefollowing technical support for languagelearning:

an audio-active-comparative languagelaboratory of 40 booths, used almostexclusively for self-access learning;an audio-active-comparative languagelaboratory of 20 booths, used almostexclusively for class teaching;a network of 12 computers, used almostexclusively for class teaching;3 individual satellite television viewingfacilities offering broadcasts in French,German and Russian.

Trinity College's proposal to the HEA includedthe provision of 20 individual video playbackand satellite television viewing facilities, to beinstalled in a room of their own. But since noadditional accommodation was available, it wasnecessary to expand and enhance technicalsupport for language learning within existingaccommodation. Obviously, it was important tofind space for new facilities and functionswithout disrupting well-established patterns ofclass teaching and self-access learning. At thesame time, the decision was taken to expand notonly individual video playback and satellitetelevision viewing but also computer facilities,since the project offered an opportunity to openthe network to self-access learning.

The following measures were taken prior to thelaunch of the new modules in Hilary term 1994:

The 12 booths arranged in pairs along theright-hand wall of the large languagelaboratory were removed. Five of them wereadded to the main block of 28 booths on theleft-hand side of the laboratory, giving a totalof 33 booths for self-access learning; livemore were added to the small languagelaboratory, increasing its capacity to 25; and

4

Ema Ushioda

the remaining two booths were put intostorage.15 individual video playback and satellitetelevision viewing facilities were installedalong the right-hand tyall of the largelanguage laboratory. In addition,adjustments were made to the distributionsystem for satellite television signals to allowa maximum of ten signals to be delivered tothe large language laboratory via a singlecable. This not only simplified theinstallation of the new facilities but openedup the possibility of making broadcasts inlanguages other than French, German andRussian available at relatively low cost indue course.The existing Nimbus computer network wasreduced from 12 to 10 machines and asecond network of 10 PCs was installed inthe same room. Purpose-built furniture wasbought for the two network:;.Since any expansion of self-access activityraises new security problems, surveillancecameras were installed in the two languagelaboratories and the computer room, and amonitor in the main CLES office.Finally, the main CLCS office was completelyreorganized to accommodate the project co-ordinator, the pedagogical adviser and theadditional language learning resourccsgenerated by the project.

2.2 Recruitment of students totake the new modules

The new modules were scheduled to begin inthe first week of Hilary lecture term 1994, inorder to allow sufficient time for planning,publicity and recruitment during Michaelmaslecture term.

Tlw modules could not be accommodatedwithin the normal lecture timetable, so it wasagreed with the respective faculty deans thatthey would run on Saturday mornings from 10

aan. to 12 noon for students in Science, and onWednesday evenings from 7 to 9 p.m. forstudents in Arts.

It was also agreed that the modules would beoffered to all junior freshmen in Science, and toall junior and senior freshmen in Arts. Animportant consideration was the fact that theopportunity to undertake study abroad isgenerally offered to students in their second orthird year. In the final stages of recruitment,however, places remaining were offered to therest of the undergraduate population in Scienceand Arts i.e., senior freshmen and sophister(third and fourth year) students in Science, andsophister students in Arts.

For students in Arts, it was agreed that priorityshould be given to those not studying a foreignlanguage as a degree subject, though within theTwo-subject Moderatorship there seems nogood reason in principle why a studentcombining German with history should beprevented from reviving her French with a viewto studying history at a French-speakinguniversity.

The modules were publicized in two ways. Forjunior freshmen in Science, a general meetingwas held in Week 5 of Michaelmas term (on 12November 1993), and was attended by anestimated 150 out of a possible total of 227students. For junior and senior freshmen in Arts,a two-day information and recruitment session(18 and 19 November) was held. This waspublicized by means of posters which heads ofdepartment were asked to display ondepartmental noticeboards.

Applications were processed on a "first comefirst served" basis, the minimum requirementfor those applying for non-beginners' modulesbeing a Leaving Certificate (or equivalent)qualification in their chosen language.

The notional maximum number of students tobe admitted to each module %vas 50, whichyielded a maxinuim of 300 students across thesix modulesapproximately a quarter of thetotal freshman population in Science and Arts.In practice, admissions were handled flexibly tobalance the numbers between over-subscribedand under-subscribed modules. There was aparticularly high demand for the l'renchmodules (see Table 2.1, p.6 below).

5

Applications were also accepted from 14students who did not belong to that part of thepopulation targeted by the project, but who hadexpressed a strong interest in the modules andhad applied in the early stages of recruitment.These included students from the Faculty ofEngineering and Systems Sciences and theFaculty of Business, Economic and SocialStudies, as well as a small number ofpostgraduate students. For the purposes of theproject statistics, however, these students weretreated as supernumeraries and not included inthe monitoring of participation figures.

2.3 Rates of participation

The new modules began in the first week ofHilary lecture term 1994, and ran for a total of15 weeks to the end of Trinity lecture term. Inorder to monitor rates of participation, regularattendance records were kept for the weeklycontact sessions. In the early weeks of HilaryTerm, thc rate of withdrawal was to some extentmatched by late admissions and participation bya handful of unregistered students in the contactsessions. As places became available, thesestudents were officially admitted to thernodules.

In Table 2.1 and Figure 2.1, p.6 below, the rate ofweekly participation is expressed as apercentage of the total number of studentsenrolled for each module. This includesconfirmed and unconfirmed withdrawals, aswell as week-by-week increases through lateadmission of applicants during Hilary Term.These increases explain why the totalrecruitment figure for all six modules exceedsthe initial maximum of 300 students referred toin 2.2. It should be added that minoradjustments in the numbers enrolled are alsodue to transferrals by a few students from onemodule to another (e.g., from German fcrbeginners to German for non-beginners or fromSaturday mornings to Wednesday evenings).

Although the participation levels showed asteady decline from the initial high rates ofattendance, the general picture which emergesseems on the %vhole very positive when put intoperspective. It was expected that the decline

Module Initialtotal

recruitment

Numberattending

first session

Finaltotal

recruitment

Numbercompleting

module

French non-beginnersin Arts (Wednesday)

70 Arts2 Science

58 (81%) 76 22 (29%)

French non-beginnersin Science (Saturday)

78 Science1 Arts

70 (89%) 85 32 (38%)

German non-beginnersin Arts (Wednesday)

32 Arts 30 (94%) 38 11 (29%)

German non-beginnersin Science (Saturday)

44 Science2 Arts

41 (89%) 47 29 (62%)

German beginnersin Arts (Wednesday)

45 Arts1 Science

32 (70%) 47 13 (28%)

German beginnersin Science (Saturday)

26 Science 22 (85%) 27 6 (22%)

Table 2 1

Rates of participation in French and German modules

90%

80%

70%

60%

50%Attendance

40%

30%

20%

10%0%

Total Student Participation

Wednesdaymodules (Arts)

1/ Saturday modules(Science)

HT HT HT HT HT1 2 3 4 5

HT HT HT HT6 7 8 9

Weeks (HT:Hilary Term, TT:Trinity Term)

Figure 2.1

would be steeper in the early weeks of themodules, and this proved to he the (7.1,,e. AsFigure 2.1 shows, the rate of attendai ice tcodedto level off about the middle of Hilary term. ,Afurther drop in participation was expected inTrinity term, with the onset of examination

6

pressures from the students' main subjects ofstudy. While participation rates were lower inTrinity than in Hilary term, attendance duringTrinity term nevertheless remained steady.Indeed, it showed a slight upward trendtowards the end of the modules. In all, 113

students underwent assessment in Weeks 5 and6 of Trinity term.

As Figure 2.1 clearly shows, a higher rate ofparticipation was maintained for the Saturdaymorning than for the Wednesday eveningmodules, even though attendance on Saturdayswas sometimc.s affected by the scheduling offield-trips. Two possible reasons for thisdifference in participation levels may beinferred. Feedback from students who withdrewfrom the modules indicated that the pressure ofa full timetable was the most common cause,cited by over 70% of those who providedexplanations for dropping out. With the normalcollege lecture timetable stretching from 9 a.m.to 6 p.m. and sometimes 7 p.m. for students inArts, it seems probable that the strain of afurther two-hour midweek evening sessionproved too great for many students whoembarked on the Wednesday evening modules.In contrast, physical or mental fatigue was lesslikely to be a contributing negative factor onSaturday mornings, once students hadcommitted themselves to devoting this period oftheir week to language study.

It should also be noted that students in Science(Saturday morning modules) were given theincentive that extra percentage points would beadded to their end-of-year examination results ifthey did well in the language modules.

The differing rates of participation onWednesday evenings and Saturday mornirigssuggest that two important issues would need tobe addressed if the language modules were tobe fully integrated into the primary degreestructure: (i) a reduction in students' workloadto compensate for the extra demands oflanguage learning; and (ii) integration of thegrade achieved into students' end-of-yearexamination result.

2.4 Design of the new modules

Each module involved students in two contacthours per week. In addition they wereencouraged to devote a further hour each weekto self-access learning.

Learners' needsBecause of time constraints itwas not possible to undertake a needs analysissurvey among the target student population.

Instead, students' needs were defined in termsof the general rationale underlying the proposedrestructuring of primary degrees. In the shOrtterm this entailed the development of basiccommunication ski,ls for the purposes of study,travel or work experience abroad duringstudents' undergraduate years. In the longerterm it entailed the development of practicallanguage skills to enhance students' academicqualifications, vocational prospects andpotential for future mobility. Added value andfuture mobility were highlighted in the publicityfor the modules.

Course designIn accordance with the short-term needs specified above, the modules weredesigned to engage students in immediate activeuse and practice of their target language so as toprovide them with a solid foundation in basicspeaking, listening, reading and writing skills.One of the reasons why so many languageteaching programmes fail to make a long-termimpact on their learners is that they do nothingto make them users of their target language.Perhaps the easiest way to ensure that languagelearners become language users is to employ thetarget language as the medium of classroommanagement and instruction. This procedurehas the added advantage that it increases theamount of target language "input" that learnersreceive and interactively stimulates the"digestion" of that input.

Classes focussed no broadly defined domains ofcommunicative activity so as to provide ameaningful context within which to develop,recycle and extend the students' range offunctions, notions, le-ds and grammar, andpromote productive and receptive skills.

For the beginners' German modules, thesedomains of communicative activitycorresponded to the practical demands ofsurvival in a German-speaking environment,such as basic greetings and social interactions,travel, obtaining food and drink, and findingaccommodation, and gradually progressedtowards dealing with bureaucracy and taking aninterest in cultural issues and current affairs.

For the non-beginners' modules, the domains ofcommunicative activity moved through threephases: (i) consolidation of basic communicativeskills for everyday survival and social needs; (ii)refinement of expressive and interactive skillsfor effective involvement in social and studyenvironments abroad; (iii) development ofspoken and written discursive ykills and

extended listening skills for the purposes ofstudy abroad, using academically appropriatesubject matter.

Contact sessions were initially structured so asto provide students with thematic expositionand organizational guidelines from thelanguage teacher (30 minutes), followed byrotations between, on the one hand, small-groupwork with native speaker student assistants (30minutes,, and on the other, teacher-supervisedself-access work using the language laboratory,computer laboratory, satellite television andvideo playback facilities (30 minutes). Sessionsended with a final integration and consolidationof the tasks and material covered with theteacher (30 minutes).

As things turned out, by the later weeks ofHilary term both teachers and students felt thata longer opening session could be more usefullyexploited for task-based student activitiesderiving from the initial exposition. In addition,a number of more academically-focussed shorttalks were arranged for the final half-hourperiod during these weeks. These talks onscience and arts topics were given by French-and German-speaking assistants as appropriate.Again it was felt that such talks could be moreeffectively deployed as expository material for

listening input, leading to productive tasksintegrating speaking and writing skills (e.g.,asking questions, taking notes, summarizing).For these reasons, a modification was made tothe two-hour structure of the contact session inTrinity term: a full hour was devoted to initialexposition leading to various task-basedactivities; in the second hour, students dividcdinto small groups for work with native-speakerstudent assistants and self-access work.

Meeting learners' long-term needsIf a

language teaching programme is to be of long-term benefit to students it must help them toimprove the efficiency of their learning. In otherwords, it must seek to foster learner autonomy,encouraging students to accept responsibility fortheir learning and to develop an ever morefinely tuned awareness of language andlanguage learning. With a view in particular tostudents long-term needs viz., theenhancement of vocational prospects and futuremobility it was desirable to encourage thedevelopment of a capacity for continuedlearning on an independent basis through theremaining undergraduate years. It was hopedthat this would be achieved through theintegration of self-access learning in themodules.

60%

50%oco

40%

.47: 30%

0%

Students using self-access facilities outside contacthours (expressed as percentage of weekly module

attendance total)

HTwk HTwk HTwk HTwk

3 5 7

HTwk

9

Trwk2

Weeks (HT:Hilary Term, TT:Trinity Term)

TTwk Thsk6

Figure 2.2

Self-access language learning facilities enablelearners to make use of resources (e.g., printedmaterial, audio and video tapes) and facilities(e.g., a language laboratory) on an individualbasis and at times that fit in with their othercommitments and activities. Ideally, self-accesssystems are supported by an advisory servicethat helps learners to determine their generallearning strategy, select learning materialsappropriate to their needs and interests, exploreand refine their preferred methods of learning,and develop a capacity for self-assessment.Within the particular time constraints of thetwo-term modules in their pilot year, it was notpossible to give self-access learning the highprofile originally intended, although the overallrates of usage of CLCS's self-access facilitieswere encouraging (see Figure 2.2, p.8 above).

2.5 Design of assessmentprocedures

For the purposes of formal student assessment,two types of test were used(i) a pencil-and-paper test to measure students' generalunderlying control of the target languagesystem; and (ii) a communicative test to assesstheir ability to deploy their language skillsinteractively.

The pencil-and-paper testFor the pencil-and-paper test two instruments were chosen: ashort dictation and a battery of four C-tests. Asubstantial body of empirical research hasshown that both instruments are highly reliableindicators of global language proficiency.

The dictation was read by a native speaker andprerecorded for transmission via individualheadphones in the language laboratory. Itconsisted of a 100-word text, read through atnormal speed, then read phrase by phrase witheach phrase repeated once, and finally readthrough once again at normal speed. Thedictation was scored at i wo levels: one mark foreach correctly transcribed word; half a mark foreach phonologically approximate word.

The C-test is a modified form of the doze test inwhich parts ot words rather than \vhole wordsare deleted. The beginning of the text is leftintact to provide a contextualizingfollowing which every second word has itssecond half deleted. A battery of f our texts ww,

chosen for each module, each text containing 25deletions. The C-tests were scored for correctresponses only.

In terms of cost-effectiveness, dictation and theC-test have the advantages (i) that they are easyto administer and to score, and (ii) that the sameestablished battery of unseen texts can be usedfrom year to year.

All five texts used in the pencil-and-paper testwere authentic in origin, although for testpurposes it was necessary to edit them slightly.The texts lay within the thematic range ofmaterial treated in the module in question, andwere selected on the basis of an intuitivejudgement that they were likely to present themore able students taking the module with noserious comprehension problems.

The communicative testThe communicativetest was conceived as a short group presentation(5-6 minutes) in the appropriate language on achosen topic, for performance in front of anaudience made up of fellow students and fourindependent assessors. This form of assessmentoffered three advantages: economy; focus ondeployment of communicative skills; andintegration with the design and delivery of themodules.

The group format of the test provides a methodof assessing the oral skills of a large number ofstudents which is more efficient and cost-effective than individual interviews. The formatof the presentation provides a natural vehiclefor the deployment of acquired language skillsin a meaningful and interactive context.Individual questioning of students following thepresentation ensures that the assessmentincludes a judgement of spontaneouscommunicative ability.

The performance context has high face validity,since students could conceivably be called uponto perform similar types of tasks in their secondlanguage in a study or work environmentabroad. Also, because preparation of the grouppresentations is integrated into the overalldesign and delivery of the modules, there is adirect correspondence between learningobjectives and assessment of studentperformance. During the weeks leading up totlw presentations, the group sessions withnative-speaker student assistants were de ,tedto their preparation. During this period also,attention was given to the rhetorical skills that

students need in order to make an effectiveformal presentation.

The group assessment rating-scales weredesigned to incorporate a grade for eachstudent's performance and an overall grade forcollaborative group performance. Individualstudent performances were scored on a 5-bandscale for each of three elements: phonology,fluency and grammatical control.

Group performances were given a single scoreon a 10-band scale for the quality with whichcontent, vocabulary, language use, presentationand rhetorical skills were integrated in thecollaborative execution of the task. A system ofpenalties was devised for group performanceswhich failed to adhere to two principalcriteriaviz., that the presentation must beinteractive, and that students must not readfrom scripts. Groups failing to fulfil one of thetwo criteria were scored on bands 1-8 only,while groups failing to fulfil both criteria werescored on bands 1-6 only.

The scales employed in the assessment of bothindividual and group performances weredefined by appropriate verbal band descriptors.In aci, ;tion, the projected maximum global levelof attainment for students vas defined (seeAppendix 4, pp.26f. below, for non-beginnersrating-scales with full band descriptors and asample rating-sheet).

It should be noted that the communicative testand the rating-scales were piloted at the end ofHilary term. The piloting entailed a statisticalanalysis of inter-rater reliability and detailedconsultation with the external assessors (see also3.5 below).

Validity and reliabilityFor this pilot phase ofthe modules, it was thought particularlyimportant to assess the validity and reliability ofthe combined testing instrument. A statisticalanalysis was therefore carried out of (i) thedegree of correlation between student resultsacross the separate test components and (ii) thedegree of inter-rater reliability in thecommunicative test conducted in Trinity term.

As regards the modules for non-beginners, theanalysis of student results revealed moderatepositive correlations between the communica-tive test and the pencil-and-paper test,suggesting that the assessment instrumentschosen provided comparable measures of

language proficiency, but that they weretapping different aspects of this proficiency. Thestatistical analysis thus supported the dualobjectives of the two types of test, which were toassess ability to deploy communicativelanguage skills on the one hand, and underlyingknowledge of the target language system on theother.

The pattern of correlations between thecommunicative and pencil-and-paper testcomponents vas much weaker in the analysis ofstudent results in the German beginners'modules. This was interpreted as a function ofthe type of language teaching to which thestudents had been primarily exposed. A strongemphasis on the development of basic oral andaural communicative skills meant that measuresof communicative production were much lesslikely to be strongly correlated with moresystem-referenced measures of target languageknowledge.

10

As expected, for all modules the analysisrevealed a strong positive correlation betweenthe parallel components of the pencil-and-papertest.

The analysis of inter-rater reliability in thecommunicative test revealed a very high level ofagreement across the four independent raters inboth French and German modules. Reliabilitycoefficients of 0.89 or greater were obtained inthe analysis of total test score§ across raters,thereby sustaining confidence in the design ofthe testing instrument and rating-scales, andconfirming the implementation of this mode ofassessment in future modules, in conjunctionwith the pencil-and-paper test. A summary ofthe statistical analysis is given in Appendix 5,p.28 below.

Calculating gradesThe overall grade for eachstudent was calculated on the basis ofperformance in both the pencil-and-paper testand the communicative test. In view of the moredirect correspondence between the grouppresentation and the content and learningobjectives of the modules, the communicativeand pencil-and-paper tests were weighted in theratio of 3 to 2. Within the pekcil-and-paper testthe weighting of the C-test battery and dictationwas in the ratio of 2 tO 1, to reflect theproportions of time allowed for the completionof these components (-10 minutt,. ,ind 20 minutesrespectively).

1 9

Fail Fail III 11.2 11.1 1

20% 29% 30% - 39% 40r/ 419% 50% - 59% Ma 69% 70q 80%

Band level 1 1.5 / 2.5 3 3.5 4 4.5 5

Rmv score 30 40 50 60 70 80 90 100

Table 2.2

Transposition of raw scores to classes of pass and fail (Arts students)

The use of a 5-band scale for the communicativetest scores was designed to facilitate thetransposition of students' total raw scores (i.e.,combined communicative and pencil-and-papertest scores) to the four classes of pass and twoclasses of fail employed in the College'sexamination system. Table 2.2 shows therelation between the marks and classes used inArts examinations and raw scores distributedacross our five bands 11.n individual student'sraw score was transposed to the Arts system ofmarks and classes by a simple formula: multiplyby 0.6 and add 20.

In the Faculty of Science examinations aremarked using the full percentage scale.Accordingly, for students in Science, the rangeof percentage grades attainable in the first classwas expanded from 70-80% to 70-100%. Thelower end of the scale would have beenexpanded downwards had any Science studentsfailed the end-of-module examination.

Full integration of the modules within theprimary degree curriculum was not attemptedduring this pilot year. However, the UniversityCouncil determined that the grades achieved bystudents in their language module "should bepresented as part of their annual examination

result". In addition to the overall grade,students were given a more detailed breakdownof their attainment levels across the variousskills. This information should help thosestudents who intend to engage in furtherlanguage Laming.

2.6 Assessment results

As noted above, assessment took place in Weeks5 and 6 of Trinity Term, and 113 studentsparticipated in the assessment process. Theresults are summarized in Table 2.3 below,which indicates the range of student perfor-mance in each module, and Figure 2.3, p.12below, which conflates the results in a singlehistogram.

The results indicate that all students completingthe modules attained a satisfactory level oflanguage competence as measured in terms of(i) their deployment of communicative skills and(ii) their general underlying proficiency. Takencollectively, 62 (55%) of the 113 studentscompleting the modules achieved a 11.2 grade,while 49 (43%) achieved a 11.1 grade.

Module Student nos Average mark Highest mark Lowest markFrench for non-beginners in Arts 20 60% 67% 51%

French for non-beginners in Science 34 57% 65% 51%

German for non-beginners in Arts 12 63% (,9%

German for non-beginners in Science 28 60% 70';', 49%

German for beginners in Arts 13 65% 66% 51%

German for beginners in Science 6 57% 63% 50%

Table 2.3 Assessment results tArts and Science)

113 3

Assessment Results (Arts and Science)

41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75

Figure 2.3

12

14

3 Externalevaluation

3.1 Introduction

From the beginning of the project, it x%'as agreedthat as external evaluators we should fulfil aconsultative role during its developmentalstages, as well as offer a qualitative commentaryon the project overall in terms of its design,management, outcomes and implications for thefuture. We were fully aware that the project'scentral concern was to establish a viableframework for the provision of foreign languageteaching for non-language students, withinparticular budgetary and time constraints.

Our evaluation is based on observations madeduring our visits to Trinity College, ourconsultations with the project team, and theregular reports and student assessment resultssubmitted to us. The evaluative report has fivesections: design of the project; the project'ssignificance in the wider context of highereducation in Europe; project management; theevaluative approach adopted; andiecommendations for the future.

3.2 Design of the pilot project

Has the programme achieved its primary aim ofdeveloping and testing a model for the deliveryof foreign language teaching to students of otherdisciplines? In our view, the answer to thisquestion is an emphatic "ves". We would like todraw attention in particular to the followingaspects of the project design.

Language learning needs analysis andsyllabus design--An effective and highlysuccessful module design was implemented,shaped by flexibility and a firmly learner-centred approach within the framework of aspecified syllabus content. One of the problems

Edith Esch andAnny King

of designing a foreign language syllabus forstudents in non-language degree programmes isthe difficulty of catering for varying levels ofproficiency, as Iva as wide-ranging needs andinterests. Students may quickly lose motivationif they do not perceive the language programmecontent to be appropriate or relevant to theirneeds. In these language modules, this problemwas resolved through a flexible learner-centredapproach to the selection of topics and thenegotiation of learning activities and materials.The great range of interests displayed bystudent groups in their assessment presentationswas strong evidence that such flexibility wasimportant, since it would have been impossibleto predict and designate appropriate topics andactivities in advance.

The approach was clearly successful, both inmaintaining high numbers of students across themodules despite the study pressures of theirother subjects, and in producing excellentresults among a multi-level mixed-abilitypopulation of non-specialist students.Outstanding success was evident in particular inthe non-beginner modules, where the range ofproficiency levels on entry differentiatedbetween students more than in the beginnermodules.

A very useful tool for the future would be adetailed account of the actual procedureswhereby students' needs and the syllabus werebrought together, topics and materials wereselected, and learning activities were negotiated.It is clear that the constraints of time andstudent numbers during this pilot phase madesuch detailed documentation impossible.Nevertheless, given the success of the projectthis year, an information resource of this kindwould be most valuable.

The modular conceptOne of thefundamental concepts of the Council ofEurope's modern languages projects is thedesign of learning units or "blocks" which arerelevant to the communicative needs of

1 311I a

particular categories of learners. This conceptwas developed in the 1970s by Trim et al.' It isappropriate to refer to it here because the CLCSlanguage modules demonstrated thepedagogical, methodological and institutionaladvantages that it offers.

Pedagogical advantagesFrom the point ofview of the students, the modules offer learningtargets which are meaningful, realistic andattainable within a relatively short period oftime. l3y its very nature the module structure isfar more likely to increase access to foreignlanguages than other programme designs. Theenthusiastic response to the modules at thebeginning of the year testifies to this.

Methodological advantages I_mprovingaccess to foreign languages "generally" is notsufficient. If it were, spending money onequipment would be enough. In our view Iiproject showed particular strength in that itgave students access to a language learningmethod and to principles for managing theirown learning. This was partly the result of thehigh degree of in-built flexibility for privatestudy and the learner support provided. Aboveall, however, access to effective learningmethods xvas gained through the task-basedgroup work activities. These encouragedcreativity and collaborative learning, producedsome remarkable results, and integrated theprocesses of evaluation and peer-assessment.=This achievement is reflected in the success withwhich students worked together at theassessment stage. Its implications are far-reaching, since the acquired self-managementskills and cooperative learning techniques aretransferable to other domains, and willhopefully be deployed throughout students'lives.

Institutional advantagesThe maininstitutional advantage of the modular structureof the pilot language programme is itsflexibility. The modules are modifiable andtransportable. For example, they could readilybe adapted to a semester rather than trimesterstructure if necessary, and they could lendthemselves to short intensive programmescompressed over a few days or weeks, althoughsuch schemes tend to have fewer long-termbenefits than more extended courses. Themodules could also be adapted lw otherinstitutions, or tailored to languages other thanFrench or German.

A second institutional advantage lies in thelikely integration of module development withthe predicted rapid expansion of technicalsupport in the future. For example, one can seethe potential benefits of creating a readilyaccessible learner support system through theuse of electronic mail, and of developing aflexible network of communication links of thiskind with French-and German-speakingstudents in other institutions in Europe.Technical expansion through European high-speed information networks would also makeavailable a range of CALL (computer-assistedlanguage learning) and multimedia packages,such as those produced in the United Kingdomin the framework of the Teaching and LearningTechnology Programme. Such materials couldbe used as learning/teaching resources andintegrated into the modules, thus making itpossible to offer language modules to largernumbers of students in Ireland. It is conceivablethat such developments in the use ofinformation technology will lead to its partialdeployment in assessment procedures also.3

14

A third major advantage is that the languagemodules could lend themselves to the Council ofEurope system of "credits"i.e., qualificationswhich can be combined in a flexible fashion. TheNational Vocational Qualifications (NVQs) arerecognized by employers in the UnitedKingdom, and may soon become validthroughout Europe under the Council of Europesystem of credits. Validating the languagemodules in terms of NVQs would require time,research and effort, but would offer significantpotential benefits and attractions to students,who would be able to "sell" their linguistic skillsmore easily on the national and Europeanmarket alike. Such a project for the future seemshighly desirable.

3.3 The pilot project in thecontext of languagelearning in highereducation in Europe

Non-compulsory character of themodulesFrom our own expel ience at theUniversity of Cambridge, we approve of thedecision to offer the language modules on avoluntary enrolment basis. At Cambridge, non-compulsory French and German programmes

16

were introduced in the new four-yearEngineering Tripos. The programmes includedclasses comparable in many ways to thelanguage modules at TCD, and were offeredduring the first two years of the course. Wefound that student take-up of the languageprogrammes was very high, and that althoughclass attendance did show some decline throughthe year, about 60% of students in the first twoyears were involved.

However, one of the worrying problems insustaining a non-compulsory scheme of thiskind at third level is that it may not be fundedproperly. Lack of adequate funding has obviousrepercussions on provision for staff recruitmentand training, with the ultimate highly negativescenario in which foreign language programmesfor non-language degree students are poorlytaught and reduced to second-rate status inhigher education.

1he project was particularly successful ineffectively deploying the guidance of twoexperienced teachers, the skills and talents of asmall team of native speaker language tutors,and the enthusiasm and motivation of a largenumber of French and German studentassistants. It would be regrettable if 1 ck offunding meant that this combined teamapproach could not be sustained in futuremodule development. It is not possible, forexample, that native-speaker student assistantson their own, even with appropriate short-termtraining, could replace experienced andqualified teachers. This aspect needs to becarefully monitored and the issue of fundingaddressed.

Returning to the question of voluntary studentenrolment, it seems that making foreignlanguage learning compulsory may becounterproductive, in the short term at least.Potential employers are likely to respond morefavourably to the evidence of real enthusiasm,effort and dedication shown by those who haveparticipated voluntarily in language learningprogrammes, rather than to evidence ofopportunism and the pre-packaged "flavour ofthe day" degree. On this front, Irish studentsseem to have an advantage over their UnitedKingdom counterparts because the bilingualcontext in which they have been brought upappears to have instilled positive attitudes toforeign language learning. Although theymostly share the same first language, theattitudinal pattern seems to be different amongEnglish students, according to the Young

Europeans in 7987 survey conducted by theEuropean Commission.4

Integration into specialists' degreesThelanguage modules could be tailored to providean access route for students aiming to specializein European course options leading to Euro-careers. In the Cambridge course mentionedabove, a European option which forms part ofthe new four-year engineering degree isavailable from the third year onwards. Studentstaking this option have to work on engineeringprojects through the medium of French and/orC :man and are assessed in these languages inAeir final examination. Making foreign

language skills an entry requirement forstudents taking such courses would deny a largenumber of students access to these particularcourses. Moreover, although all educatedindividt.ils ideally need to be able to survive inat least two other European languages, it is truethat only a proportion of students vill actuallygo for Euro-careersall the more r2ason whyaccess to courses with a European option shouldnot be restricted at college entry level. Theprovision of language modules, on the otherhand, gives students Nyho have specialmotivation or talent for languages the chance toprepare themselves to take a special option ofthis kind in their third or final year.

1 5

Choice of French and GermanThe decisionto offer modules in these particular languagesmakes sense not only for historical and culturalreasons, but more importantly for linguisticreasons. Knowledge of French gives access toRomance languages such as Spanish,Portuguese, Italian and Catalan, and even tosome extent Modern Greek. Knowledge ofGerman of course gives access to Dutch,Luxemburgish and to a number of dialectsspoken in Switzerland, Belgium and France. Inthis way, both languages offer direct accessroutes to a majority of the Indo-Europeanlanguages spoken in the European Union. Forstudents who already have a Gaelic language,knowledge of French/German thus gives anadded bonus in the context of the EuropeanUnion.

3.4 Management of the project

The management of the project was \ Try goodindeed. A timetable was established from the

_17

beginning, and the overall design of the projectwas very clearly stated so that there were noambiguities. As evaluators, we felt that we weregiven all the information we required, thateverything was explicit and carefully planned,and that what we were asked to do met with ourexpectations of our role. Two points need aspecial mention for excellence.

Firstly, the log-book system used to keep usinformed of week-by-week project developmentwas implemented in a very effective way. Itsnarrative aspect provided easy and multiple"entry points" which made it possible for us asevaluators to move into the project quickly andeffectively.

Secondly, the group assessment rating-gridswere particularly well thought-out in terms ofpresentation and layout, as well as content. Theywere easy to understand, clear and explicit..Thismade it possible for any person not involved inthe project to understand the structure and maincategories for the implementation of theassessment instrument. We feel that CLCSshould provide a "project management tool-kit"(on paper and diskette) to support and guideother institutions of higher education involvedin similar projects.

3.5 Evaluative approachadopted for the project

We believe that the approach to projectevaluation used here is to be recommended as amodel of the genre. We certainly hope that itwill be built upon and generalized in theprofession.

We find that the process of formative as well assummative evaluation is an excellent approach.It saves an enormous amount of time and turnsa so-called objective "judgement" into acooperative search between respectedcolleagues for the best solution in a givencontext. We do not wish to imply that ourevaluation was somehow "cosmetic", however.Evaluation developed rather as a constructiveprocess whereby any problems or criticisms weraised could be discussed and dealt withimmediately in a positive cooperative fashion.

This constructive process was best seen in theway in which commentary and feedback

following the piloting of the group taskassessment in Hilary term led to fine-tuning ofthe rating-grids and clarification of task criteriafor the Trinity term assessment. In this way webelieve we were able to ensure that ourevaluation had maximum positive impact onproject development while minimizing the face-threatening potential that our evaluative rolemight impose.

16

In the context of language programmedevelopment of this kind, we are convinced ofthe mutual benefits of such evaluativeprocedures both for the institution undergoingevaloltion and also for the evaluators, wholearn from the process as well. We are currentlyleading a Teaching and Learning TechnologyProgramme (TLTP) for the development ofmaterials for language learning for Engineersand Scientists.5 We were therefore able toapprise the CLCS team of our own findings,while they could alert us to particular types ofcommon problems which we had not foreseen.In other words, project evaluation was definitelya valuable exercise for both institutions andcountries.

3.6 Recommendations for thefuture

1 The project should lead to a consolidatedprogramme of language modules.

2 A second-stage pilot should be conducted toinclude:(a) a detailed study of the module content

and of the range of procedures used tomatch needs and syllabus.

(b) a training module for all staff involved.(c) a study of the way the modules could be

related to the NVQ scheme.

3 The consolidated programme should begeneralized to include other disciplinesand/or languages if required.

4 CLCS should plan to make information onthe modules and their assessment availableto other universities by means of advancednetworking, such as Superjanet, betweenuniversities. Problems of infrastructure arecrucial at this juncture if duplication is to beavoided.

iS

5 CLCS should produce a project managementtool-kit for the management of similarlanguage programmes in other third-levelinstitutions. This should include in particularan assessment procedure pack with all therating-grids and explanations of what theydo, how to use them, etc.

6 A promotional video should be produced ofthe group tasks performed by the students inHealth Sciences, which could be included inthe pack mentioned under 5, and used totrain tutors and assistants, etc. Other video-recordings should be made to train staff inthe use of the assessment rating-grids.

Notes

1 Systems Development in Adult LansungcLearning, Council for Cultural Cooperation,Council of Europe, Strasbourg, 1973.

2 Anny King's comment on the TCD medicalstudents' performance was: "the result ofnon-specialist language teaching at its best".

Students were using their specialistknowledge while being entertaining andimaginative in the second language.Moreover, there was evidence of theeducational value of the exercise. The teameffort had made students think abouteffective communication in a cooperativefashion: they had been using each others'special talents to best effect for the group.

3 Cf. publications of the Teaching andLearning Technology Programme (TLTP)and ALTER (Assessment of Learningthrough Technology for Efficiency and-Zigour) published by the UK Universities':ltaff Development Unit (USDU) and theUniversities of Kent and Leeds.

4 Young Europeans in 1987, Commission of theEuropean Communities, Brussels, 1988.

5 TLTP Project CKS33, a Cambridge-ledconsortium for the production of multimediapackages in French and German forEngineers and Scientists. The CambridgeLanguage Centre is in partnership with theUniversities of Kent and Southampton forthe production and with the Universities ofSurrey and Durham for the evaluation.

4 Conclusion

4.1 Response to the externalevaluators'recommendations

The CLCS project team accepts therecommendations made by the externalassessors in 3.6 above.

The measures reported in section 4.2 belowconstitute acceptance of recommendation 1;while the three parts of recommendation 2 havebeen built into the organizational framework for1994-5.

In the course of 1994-5 we plan to conduct asurvey to establish what language learningneeds exist in Trinity College beyond thosecovered by the modules we have developed todate (recommendation 3). Previous surveyshave indicated that the demand for languagesother than French and German is likely to berelatively low (see 1.2 above). If this finding isconfirmed, it will have obvious implications forthe ways in which these needs should be met.

We agree that it would be appropriate to makeinformation on the modules and theirassessment available to other 'Universities(recommendation 4). As a first step in thisdirection, the present report will be distributedas widely as possible. Depending on theouLome of the Information Systems PolicyDevelopment exercise that Trinity College hasbeen conducting during the academic year1993-4, it may be possible in the fairly nearfuture to make information about the modulesavailable via advanced networking, as theexternal assessors recommend.

In the course of 1994-5 we plan to produce aproject management tool-kit (recommendation5) and to make video recordings tor use in thetraining of teachers, assistants and assessors(recommendation 6).

1 8

David Little

4.2 Provision of languagemodules in 1994-5

FundingIn its original invitation to theuniversities to submit proposals for funding, theHigher Education Authority emphasized thepilot nature of the 1993-4 projects. Nevertheless,at various stages in the academic year 1993-4the College was given to understand thatfunding would again be made available tosupport foreign language modules in 1994-5. Ittherefore came as an unwelcome shock whenthe HEA informed the College in July 1994 thatno further funding would be available.

The success of the pilot language modules hadbeen such that it was unthinkable to the CLCSproject team that they should be abandonedwithout further ado. In any case, the need toprovide foreign language learning opportunitiesfor students is not diminished by fundingdifficulties.

Fortunately it has proved possible to offer (i) themodules that were piloted in 1993-4 to a newgeneration of students in 1994-5, but over threeterms rather than two, and (ii) second-yearmodules to all students who successfullycompleted the pilot modules. CLCS has beenable to draw on its own research and non-payfunding to provide the organizational basis forthe modules, while the faculties involved haveagreed to meet the cost of hourly-paid teaching.Material support costs (learning materials,printing, stationery, etc.) will be met byrequiring each student who enrols for alanguage module in 1994-5 to pay a levy of L20.

It must be emphasized that these fundingarrangements are an ad hoc response to anemergency, and that they entail the use ofresources that will not be available in futureyears. They are thus no more than a verytemporary solution to a problem that theCollege must solve in the course of 1994-5 if it

wishes to provide foreign language modules inthe long term.

Organizational arrangementsAninformation sheet advertising the 1994-5modules has been sent to all incoming juniorfreshmen in Arts and similar publicity materialswill be disseminated to incoming juniorfreshmen in other faculties during Freshers'Week (3-7 October 1994). Applications will beprocessed in the first two weeks of Michaelmasterm, and the modules will begin in Week 3. Asin 1993-4, preference will be given toapplications from junior freshmen in Scienceand to applications from junior and seniorfreshmen in Arts. But as in 1993-4 we expect tobe able to accommodate a small number ofundergraduate students in other years as well asa few postgraduate students.

Student inductionThe first two weeks of the1994-5 modules will be giVen over to studentinduction. Students will be provided withinformation about the processes involved inforeign language learning and introduced to theself-access resources and facilities. Workshopactivities will be used to explore appropriateapproaches to learning, including the keeping ofa regular log-book as a record of what work hasbeen done, a compilation of what has to belearnt, and an assessment of how well learningis progressing.

Induction of teachers and ERASMUSstudent assistantsEarly in Michaelmas termapplications will be sought from ERASMUSstudents who wish to act as language assistants.All applicants will be screened for theirsuitabilityin particular they will be asked tomake a brief presentation in French or Germanas appropriate.

There will be formal meetings of teachers andERASMUS students at the beginning and end ofeach term, and regular briefing and feedbacksessions during term.

Learning materials and resources for self-accessThe core learning materials for bothnew and continuing students have beenprepared, including a student handbook foreach term. The handbook contains coursematerials, advice on language learning ingeneral and the use of self-access resources inparticular, and templates and guidelines forstudents' log-books.

In the course of the year particular attention willbe given to the development of resources tosupport the self-access component of themodules. Work in this area will benefit fromexploratory visits made in 1993-4 to Paris andFrankfurt (Marc Gallagher), and to WidaSoftware, London (David Little, Erna Ushioda,Marc Gallagher). Provision will be made for theuse of CD-ROM materials in CLCS's computerroom.

In March 1994 David Little and Ema Ushiodavisited the Media Center for Language andCulture, University of Amsterdam, which hasbegun to develop a computerized catalogue ofself-access language learning materials. Workwas subsequently begun on the development ofa computerized catalogue for CLCS's materials,including a guide to their use. This project willcontinue in 1994-5.

4.3 Research dimensions

Section 1.1 of this report noted that the designand implementation of the pilot modules wasable to draw on a strong research base in CLCS.This meant that the pilot modules rapidlygenerated a research dimension of their own,which will be continued in 1994-5 in five areas:assessment, interactive video for self-accesslanguage learning, learner autonomy, learnercounselling, and the use of information systemsand electronic communication in foreignlanguage learning.

AssessmentAt the Third CERCLESConference, University of Hull, 23-25September 1994, D'avid Little and Ema Ushiodagave a paper reporting on the development andvalidation of the assessment instruments used atthe end of the pilot modules, with particularreference to the group presentations. In 1994-5these assessment instruments will be furtherdeveloped to include a test of students' writingcompetence at the end of the second-yearmodules.

Interactive video for self-accessFor anumber of years CLCS has been workingexperimentally with interactive \'id,'o forlanguage learning in self-access. At EUROCALL1994 in Karlsruhe, 15-17 September, DavidLittle gave an invited keynote paper thatelaborated the theoretical foundations of this

Nvork, explored some data elicited fromstudents, and set out a development plan for thefuture. In 1994-5 CLCS intends to refine itsinteractive video system further, with a view togiving it a central role in the self-accesscomponent of the language modules from 1995-6 onwards.

Learner autonomyThe fuller use of self-access resources will be influenced by DavidLittle's continuing work on learner autonomy,which in 1993-4 included a study commissionedby the Council of Europe's modern languagesproject, two invited articles for internationaljournals, and an invited keynote paper at aninternational conference on learner autonomy inHong Kong. Pursuing a related line of research,Marc Gallagher gave a paper on learnerautonomy at the Third CERCLES Conference,University of Hull, 23-25 September.

Learner counsellingA new researchdimension is planned for 1994-5, focussing onthe language learning process and learnerautonomy. The intention is to develop asuccession of instruments to elicit data fromstudents at various stages in the academic year,and to analyse the data thus elicited in such away as to provide individual students withguidance on the direction of their furtherlanguage learning. The instruments will includea questionnaire on students' languagebackground, a language aptitude test, measuresof student attitudes and motivation, a C-testbattery, and learner log-books.

This dimension of the project is closely relatedto the consultancy work David I.ittle and EmaUshioda have undertaken for the Centre forModern Languages at the Open University,Milton Keynes.

Information systems and electroniccommunicationIn the course of 1994-5 theproject team Nvill explore the feasibility ofestablishing an international research project tolook into Nva y s of exploiting electroniccommunication and electronic databases tosupport language learning. In particular, wehope that it may be possible to establish atriangular link between ourselves anduniversities in France and Germany, so thatIrish learners of French and German can be putin touch with French- and German-speakingpeers.

4.4 The future

At a formal institutional level Trinity Collegehas been seriously imerested in providinglanguage learning opportunities for thegenerality of students at least since Trinity term1990, when (as noted in the introduction to thisreportsee 1.1 above) the University Councilmade CLCS responsible for the co-ordination ofefforts in this area.

20

The principal obstacle to the implementation ofthe proposals that were brought forward inMichaelmas term 1990 was lack of funding. Thisobstacle was temporarily overcome in 1993 bythe sudden availability of special funding forthe pilot project which is the subject of thisreport. As we move none too securely into thesecond year of the modules, two large issueshave to be confronted: integration and long-term funding.

IntegrationIntegration can be approached intwo ways. On the one hand foreign languagemodules can become integral parts of thecurriculum in the sense that they are eithercompulsory for all students or else options thatbecome compulsory for students who choosethem. The obvious advantage of this approach isthat there is no significant drop-out, whichmeans that the organization of teaching can befinely calculated to be as cost-effective aspossible. The disadvantage is that foreignlanguage learning becomes one more subject ofstudy, so that the provision of foreign languagemodules cannot reasonably be described asgiving added value.

On the other hand, foreign language modulescan be integrated in the weaker sense that theyare designed to address students' needs inrelation to their main academic discipline andrelated career options. The disadvantage of thisapproach is that there will always be asignificant rate of attrition early in the academicyear. The advantagewhich should not belightly dismissed (see the report of the externalevaluators, 3.3 above)is that students whofollow an optional programme of languagelearning to its conclusion mostly do so becausetheir learning is successful. This helps to explainthe fact that all students who completed the1993-4 pilot modules passed the end-of-yearassessment; it also helps to explain why classeswere so strongly cohesive and enjoyable.

Long-term fundingIn Hilary term 1994 the.question of the integration of foreign languagemodules was discussed by several TrinityCollege committees responsible for courses andtheir development. Each committee quicklycame to the same conclusion: that the discussionwas meaningless in the absence of guaranteedfunding to support foreign language modules inthe future.

As this report shows, a great deal of progresshas been made in a very short space of time.What is more, that progress has been not simplyin the provision of "service" language teaching,but in the beginnings of a solution to theproblem of foreign language modules for allthat started from a strong research base and hasitself generated further research. This is by no

means a widespread phenomenon, but it issurely one with which any university should bepleased.

As noted in 4.2 above, the foreign languagemodules that were piloted in 1993-4 will beoffered again in 1994-5, and second-yearmodules will be offered to students whosuccessfully completed the pilot modules. Thishas been possible thanks to special financialcircumstances that have been created in part bythe very fact of the special funding madeavailable in 1993-4. However, it will not bepossible to continue the modules beyond 1994-5without the provision of at least some long-termfunding security. How that is to be achievedmust now be addressed as a matter of urgency.

Appendix 1

Applied linguistic research in CLCS:selected list of publications

Self-access and self-instruction

Devitt, S. M., D. G. Little, S. P. O Conchtuir andD. M. singleton, 1982-3: "Learning Irishwith Anois is Ar13". CLCS Occasional PaperNo.6.

Little, D. (c_d.), 1989: Se!f-Access Systems forLanguage Learning. With contributions fromEdith Esch, Marie-José Gremmo, DavidLittle, Harvey Moulden, Philip Riley andDavid Singleton. Dublin: Authentik.

Little, D. G., and A. J. Grant, 1986: "LearningGerman without a teacher: report on a self-instructional programme for undergraduatestudents of Engineering Science at TrinityCollege, Dublin, 1982-4", CLCS OccasionalPaper No.14.

Learner autonomy

Little, D., 1991: Leanwr Autonomy. 7: Dtfinitions,Issues and Problems. Dublin: Authentik.

Little, D., forthcoming: "Learner autonomy: atheoretical construct and its practicalapplication", Die Neueren Sprachen.

Interactive video in languagelearning

Little, D., 1988: "Interactive video: teaching toolor learning resource?", Journal of AppliedLinguistics 4 (Greet Applied LinguisticsAssociation), pp.61-74.

22

Little, D. G., and E. J. Davis, 1986: "Interactivevideo for language learning: the Autotutor",System 14, pp.29-34.

Little, D., 1994: "Interactive videocassette forself-access: a preliminary report on theimplementation of Autotutor II", Computersin Education, 165-170.

Definition of language teachingsyllabuses

Little, D., H. 0 Murch6, and D. Singleton, 1985:A Functional-Notional Syllabus for AdultLeanwrs of Irish. Dublin: Trinity College,Centre for Language and CommunicatioilStudies.

Little, D., H. O Murch6, and D. Singleton, 1985:Towards a Connnumicatiiv Curriculion for Irish.Dublin: Trinity Colley,o, Centre forLanguage and Communication Studies, inassociation with Bord na Gaeilge.

Singleton, D. M., and D. G. Little, 1985: "Foreignlanguages at second level: defining asyllabus, with particular reference to theneeds of the senior cycle". CLCS OccasionalPaper No.12.

Development of languagelearning materials

Little, D., S. Devitt and D. Singleton, 1989:Learning Foreign Languages from AutIwnticTexts: Tlwory and Practice. Dublin: Authentik,in association with the Centre forInfon, -,tion on I Anguage Teaching,London.

2 4

Pedagogical grammar

Little, D., 1994: "Words and their properties:arguments for a lexical approach topedagogical grammar." In T. Odlin (ed.),Perspectives on Pedagogical Grammar, pp.99122. Cambridge: Cambridge UniversityPress.

Little, D., and D. Singleton, 1991: "Authentictexts: pedagogic grammar and languageawareness in foreign language learning". InLanguage Awareness in the Classroom, editedby C. James and P. Garrett, pp.123-32.London: Longman.

Second language mental lexicon

Singleton, D., and D. Little, 1991: "The secondlanguage lexicon: some evidence fromuniversity-level learners of French andGerman", Second Language Research 7, pp.61-81.

The age factor in languagelearning

Singleton, D. M., 1981: "Age as a factor insecond language acquisition". CLCSOccasional Paper No.3.

Singleton, D. M., 1989: Language Acquisition: theAge Factor. Clevedon & Philadelphia:Multilingual Matters.

Role of mother tongue andlanguage transfer in languagelearning

Scarpa, F., 1990: "Contrastive analysis andsecond language learners' errors: ananalysis of C-test data elicited frombeginners in Italian". CLCS OccasionalPaper No.27.

Singleton, D. M., 1981: "Language transfer: areview of some recent research". CLCSOccasional Paper N6.1.

Singleton, D. M., 1987: "Mother and othertongue influence on learner French: a casestudy", Studies in Second LanguageAcquisition 9.3.

Singleton, D. M., 1987: "The fall and rise oflanguage transfer". In The AdvancedLanguage Learner, edited by J. Coleman andR. Towell. London: CILT.

Singleton, D. M., 1990: "The cross-linguisticfactor in second language learning: a reporton some small-scale studies recentlyconducted at the CLCS", CLCS OccasionalPaper No.24.

Singleton, D. M., and D. G. Little, 1984: "A firstencounter with Dutch: perceived languagedistance and language transfer as factors incomprehension". In Language AcrossCultures, edited by L. Mac Mathtina and D.Singleton. Dublin: Irish Association forApplied Linguistics.

Appendix 2

French module for students of Health Sciences

Background

The French for non-beginners module forstudents in Health Sciences was launched inHilary term a week later than the othermodules, in order to allow sufficient time for theprocessing of applications. Two-hour contactsessions were arranged for Saturday mornings,starting at 9 a:m. The early start meant that boththis and the French module for students inScience (starting at 10 a.m.) could be managedby a single team of teachers and languageassistants. In terms of general design, content,organizational structure and assessmentprocedures, the module for students in HealthSciences followed a pattern identical to thatalready described.

Rates of participation

Initial Numbertotal attending

recruitment first session39 18 (47%)

Finaltotal

recruitment39

Numbercompleting

module9 (23%)

Assessment results

24

Numberassessed

9

Averagegrade67%

Highestgrade82%

Lowestgrade6tri

The results indicate a particularly high level ofattainment across the students completing themodule, with an average grade in the upperrange of tiv2II.1 class, and a first class gradeachieved by 4 of the 9 students. The differencein the level of performance between the studentsin Health Sciences and students in Arts orScience was especially salient in the quality ofthe group presentations. This difference is aclear reflection of the high standards ofacademic achievement required of thoseentering the Faculty of Health Sciences, many ofwhom have superlative school-leaving results inforeign languages. The success demonstrated bythose completing the French for non-beginnersmodule in 1994 points to the desirability ofmaking further provision towards languagelearning opportunities for students in HealthSciences, ideally within the same fundingscheme as that governing the modules forstudents in Arts and Science.

2 6

Appendix 3

Summary of expenditure

Staff

Project co-ordinator and pedagogical adviser 27,100

Hourly paid teachers (3 French, 3 German) andERASMUS language assistants (13 French, 17 German) 10,204

37,304

Equipment

Computer network: 1 server, 10 computers, network cards,cabling, printer, furniture, 3 CD-ROM drives 24,549

Satellite television distribution systern, 15 receivers,15 video cassette recorders, cabling, furniture 15,092

CCTV surveillance system 1,520

Reconfiguration of language laboratories 1,360

Computer and printer for project administration 1,912

44,433

Language learning materials 5,636

External evaluation

Fee 2,078

Administration cosis (Cambridge) 2,613

4,691

Travel and subsistence

External evaluation 1,585

Exploratory visits to Amsterdam, Paris and Frankfurt 1,742

EUROCALL 1994 1,580

Third CERCLES Conferenco 1,046

5,953

Stationery, printing, photocopying, office consumables 1,983

TOTAL £100,000

25

2 7

Appendix 4 Band descriptors and rating sheet

Assessment Guidelines (Non-Beginners)Expected maximum global level of attainment

Can speak at length with confidence and reasonable fluency on familiar or general topics, adaptingappropriately to formal/informal contexts. Lacks facility in handling abstract or unfamiliar topics but cancope reasonably well. Difficulty with complex sentence and discourse structures.

Individual Student Assessment

CATEGORY Level 1 Level 2 Level 3 Level 4 Level 5

Phonology: almostunintelligible

strong Llaccent

fair L2accent

good L2accent andintonation

very good L2accent andintonation

Fluency: very slowand hesitant

frequentstops andstarts

stilted andsomewhathesitant butmanages tokeep going

flowsreasonablywell despiteoccasionalhesitations

fluent, goodexpression

very fluentandexpressive,normalspeechtempo

Grammatical control: very littlecontrol -grammaticalerrorsimpede themessage

errorsfrequent,difficultywithcomplexstructures,but messageintelligible

good controlof basicstructures,some errorsin morecomplexstructuresanddiscoursepatterns

onlyoccasionalerrors,mostly incontrol ofcomplexstructuresanddiscoursepatterns

very fewnoticeableerrors,competentcontrol ofcomplexstructuresanddiscoursepatterns

Group Task Assessment

How effectively the task is executed in terms ofcontent/substanceappropriate languagevocabulary rangeorganization

rhetorical skillspresentation skillscontinuity and flow

To be marked out of 10, the presentation must fulfil TWO conditions:(a) Students should not read from scripts (prompts, visuals, props, overhead transparencies, etc., may of coursebe used);(b) The presentation must be interactive (i.e., there should be regular interaction and communication betweenthe students in the ;rou ), and not list a sirn Ic se uencin; of individual talks).

1 2wook

3 4satisfactory

5 6good

7 8very good

9 10outstarolin

If the presentation fails to fulfil ONE of these conditions, it can onir be marked out of 8:1 2

weak

3 4satisfactory

5 6good

7 8stile yowl

If the presentation fails to fulfil BOTH of these conditions, it can only be marked out of 6:1 2

,,,..il3 4

siitisint tiny

5 6giiiiit

26

26

Sample rating sheet

MODULE DAY GROUP ASSESSOR

To ic:

Individual student assessmentName

1

Phonology2 3 4 5

Fluenc Grammatical control1 2 3 4 5 1 2 3 4 5

Final revision

Final revision

Final revision

Final revision

Final revision

Group Task AssessmentTo be marked out of 10, the presentation must fulfil TWO conditions:

(a) Students should not read from scripts(b) The presentation must be interactive

1 2weak

3 4satisfactory

5 6good

7 8very good

9 1 0outstanding

If the presentation fails to fulfil ONE of these cond.tions, it can only be marked out of 8:1 2

%%cak

3 4satisfactory

5 6ood

7 8%cry nod

out of 6: JBOTH of these conditions, it can only bc markedIf the presentation fails to fulfil1

,

2wv.ik

3 4caticla( tory

5 6oncl

.

27

Appendix 5 - Correlational analysis of test components

French Non-Be inners German Non-Be inners German Be innersGroup

PresentationDictation Group

PresentationDictation Group

PresentationDictation

Dictation 0.425139 Dictation 0.375993 Dictation 0.20419

C-Test 0.420545 0.68396 C-Test 0.572294 0.533901 C-Test 0.117565 0.805523

Writtentest total

0.458111 Writtentest total

0.563065 Writtentest total

0.160559

Communicative test: inter-rater reliabilityi;or the estimation of inter-rater reliability, the Spearman-Brown Prophecy Formula was used, based onth number of independent raters (4) and fir. average correlation among raters. (For full details of theprocedure, see Henning, G., 1987: A Guide (o Language Testing, Cambridge, Mass.: Newbury House,pp.82-83)

French for Non-Be inners Modules: Correlations between raters' marksphonology rater 1 rater 2 rater 3 fluency rater 1 rater 2 rater 3

rater 2 0.478566 rater 2 0.451849rater 3 0.474273 0.537)53 rater 3 0.487429 0.770671

rater 4 0.632066 0.699134 0.494578 rater 4 0.591522 0.591546 0.618614

grammar rater 1 rater 2 rater 3 group task rater 1 rater 2 rater 3

rater 2 (1.577794 rater 2 0.486457rater 3 0.539127 (1.542007 rater 3 0.754427 0.598905rater 4 0.534124 0.552907 0.502368 rater 4 0.639769 0.38685 0.430813

total score rater 1 rater 2 rater 3 phonology 0.83rater 2 (1.557626 Inter-rater fluency 0.85rater 3 0.67692 0.75280) reliability grammar 0.83rater 4 0.753184 0.704756 0.673945 coefficients group task 0.83

total score 0.90

German for Non-Be inners Modules: Correlations between raters' marksphonology rater 1 rater 2 rater 3 fluency rater 1 rater 2 rater 3

rater 2 0.606134 rater 2 0.454016rater 3 0.593243 0.567313 rater 3 0.535745 0.549366rater 4 0.631713 0.448527 0.591623 rater 4 0.336497 0.186625 9 621083

grammar rater 1 rater 2 rater 3 group task rater I rater 2 rater 3

rater 2 0.651488 rater 2 0.509203rater 3 0.615661 0.532433 rater 3 0.361372 0.833946rater 4 0.507682 0.383(194 0.580402 rater 4 0.807637 0.422477 0.231556

total score rater 1 rater 2 rater 3 phonology 0.84rater 2 0.77365 Inter-rater fluency 0.76rater .i 0.682344 0.754512 reliability grammar 0.83rater 4 0.721185 (1.495167 0.528646 coefficients group task 0.82

total score 0.89

German for Be inners Modules: Correlations between raters' marksphonology rater 1 rater 2 rater 3 fluency rater 1 rater 2 rater 3

rater 2 0.416709 rater 2 0.669049rater 3 0.452671 0.5482 rater 3 (1.813105 0.398949rater 4 0.535179 0.279881 0.537289 rater 4 (1.741952 0.476206 0.67477

grammar rater 1 rater 2 later 3 group task rater 1 rater 2 rater 3

later 2 0.445522 rater 2 0.700847rater 3 0.773572 0.367413 rater 3 0.841489 0.926926rater 4 0.74778 0.102857 (1.661644 rater 4 0.771058 0.971041 (1.88(1921

total score rater 1 rater 2 rater 1 phonology 0.77later 2 (1773)2 Inter-rater fluency 0.87

rater 3 (1.813871 0.87811 reliability gl ammar (1.83

rater 4 0.788331) 0.698459 0 791493 coefficients group task 0.96total score 0.94

2 8