Microsoft Corporation, 1978Microsoft Corporation, 1978

The future is more about innovation than invention. The difference ?

Innovation uses inventions to do something in a new and better way. It's

about the application of inventions to deliver value to clients and

society. We will continue to innovate to progress business and society.Bart Van Den Meersche

Country General Manager IBM-Belgium/LuxembourgNew Year’s Speech, Brussels 20 Jan. 2004

The c-test revisited

A freeware for placement in the new Ba-Ma structureAutomated Tool for Trainers, Students and Test

AdministratorsLut Baten

Lut.baten@ilt.kuleuven.ac.behttp://millennium.arts.kuleuven/ctest

Overview

• 1. Specification– Aims, what, target groups

• 2. Demonstration – Examples

• 3. Validation– Data collection and analysis

• 4. Questions and Problems

In short

• Placement testing – large groups of LSP st.• Validity: controlled input

– lexical difficulty : WordClassifier– different text types

• Facility: Authoring tool• Implementation: ELP, e-learning, autonomy• Recommendations

What?• placement device• °1980s at Duisburg Univ. • 4 to 6 texts : completed in 30’• text = unit; 70 to 110 words

– sentence 1: intact –– then the rule of two : word 2 in sent 2, 2nd half of every

2nd word is deleted– Last sentence: undamaged

• repair 25 incomplete words in whichever order.

1. Specification

• Needs• Requirements• Target groups

Occasion: English optional in Ma

Placement test needed- speed, size, reliability, flexible, free

Customized - Concept validity: content level

- Text selection: text typology

- CEF compatibility:- vocab, comprehension and stylistic level- Calibration of tests

Automated

2. Demonstration

- Preparation of a text- Word Classifier- Lay-out

- Authoring tool - Student version- Reporting- Community application

Procedure

• Insert text

Set the parameters

Prepare for target & group

Packaging

Reporting

Delivery

Specific requirements

• user-friendliness – develop exercises + tests.

• automatic + immediate correction • reporting functions• tighter and more customized control

– e.g making use of the Word Classifier.• freely available on the Web and on cd-rom. • a test battery

– students of Business English • on the KULeuven learning platform

• Question Mark Perception

Example: analysis WoCLLetter refusing a request for credit• Dear Mrs Wardle• We were glad you approached us with a view to placing an order, and to learn of

the good start of your new business. • The question of granting credit for newly established businesses is never an

easy one. Many owners get into difficulties because they overcommitthemselves before they are thoroughly established. Although we believe that your own business promises very well, we feel it would be better for you to make your purchases on a cash basis at present. If this is not possible for the full amount, we suggest that you cut the size of your order, say by half.

• If you are willing to do this we will allow you a special cash discount of 4 % in addition to our usual trade terms. If this suggestion is acceptable to you, the goods could be delivered with you within 3 days.

• We hope that you will look upon this letter as a mark of our genuine wish to enter into business with you on terms that will bring lasting satisfaction to us both.

• When your business is firmly established we will be very happy to welcome you as one of our credit customers.

• Yours sincerely

Colourised – Pruned

range 0 range 1 range 2 range 3 range 4 range 5 outsiders other

• Dear Mrs We were glad you approached us with a view to placing an orderand to learn of the good start of your new business The question ofgranting credit for newly established businesses is never an easy oneMany owners get into difficulties because they overcommit themselvesbefore they are thoroughly established Although we believe that your ownbusiness promises very well we feel it would be better for you to makeyour purchases on a cash basis at present If this is not possible for thefull amount we suggest that you cut the size of your order say by half Ifyou are willing to do this we will allow you a special cash discount of 4 inaddition to our usual trade terms If this suggestion is acceptable to youthe goods could be delivered with you within 3 days We hope that you willlook upon this letter as a mark of our genuine wish to enter into businesswith you on terms that will bring lasting satisfaction to us both When yourbusiness is firmly established we will be very happy to welcome you asone of our credit customers Yours sincerely

WordClassifier

Hedging (pruned text)Word Classifier-Report - Text with (after pruning) 2,221 word ;

= 668 different words• .• Range 0: 139 word(s) - 21 %• a, about, after, again, all, also, although, always, am, an, and, any, anything, are, as, at, back, be, because, been, before, being,

both, but, by, can, cannot, could, did, do, doing, done, down, during, each, every, few, first, for, from, get, had, half, has, have, having, he, here, his, how, however, i, if, in, indeed, into, is, it, keep, let, made, make, many, may, me, mid, might, more, most,mr, mrs, much, my, never, next, no, not, now, of, on, one, only, or, other, our, out, outside, over, put, same, say, several, shall, should, since, so, some, someone, such, take, taken, than, that, the, their, them, themselves, then, there, these, they, third, this, Range 1: 257 word(s) - 38 %

• able, accept, acceptable, acted, active, addition, afraid, agree, allow, april, arise, arts, ask, august, bankers, basis, believe, better, bill, book, bring, business, businesses, buyer, buyers, call, carried, carry, cause, caused, class, clear, come, company, complete, completed, concerned, concerning, condition, consider, consideration, continues, cost, cut, days, deal, december, department, difficult, difficulties, difficulty, drew, e, earlier, easter, easy, effect, end, english, even, examination, examined, expect, explain, explains, explanation, fact, february, feel, figures, final, finally, finding, following, forward, found, full, further, gave, give, given, Range 2: 154 word(s) - 23 %

• account, advance, agent, agents, ago, amount, anxious, apparently, application, approached, arrange, arrival, authorize, avoid, aware, balance, charge, charged, cheque, choice, chosen, claim, claims, collection, commission, commitments, competition, competitors, completion, conditions, consequently, continually, conversation, current, dear, demand, design, detailed, discount,discover, discussed, discussion, drawer, due, enclose, enquiries, enquiry, ensure, enter, entirely, escaped, essential, established, exactly, expenses, extend, failing, failure, fair, financially, firm, firmly, firms, gained, grant, granting, honour, identical, identify, Range 3: 85 word(s) - 13 %

• absence, accuracy, adjust, afterwards, apologies, assessor, assessors, belgium, cash, cd, christmas, circumstance, circumstances, complains, complaint, comprehension, confirm, consulted, contractor, contracts, credit, delay, delivered, deliveries, delivery, deposit, disappointing, disturbing, facilities, faithfully, faulty, faxed, file, flanders, focus, genuine, grateful, inconvenience, indefinitely, inspection, instructions, internal, investigate, japanese, madam, mails, manufactured, manufacturers,

• Range 4: 16 word(s) - 2 %• compelled, compensate, customer, customers, distress, dutch, hedging, overdraft, overdrafts, overlooked, premises, prompt,

rotation, sincerely, solicitors, submit, • Range 5: 2 word(s) = 0 %• defer, outstanding, • Outsiders: 15 word(s) = 2 %• authoring, batch, dishonoured, downloaded, feedback, freeware, incurred, invoice, millennium, overcommit, oversight,

remediation,

Target group for cd-rom

• KULeuven ILT – English + all W. languages

• Teachers and ALO-students: grip on- placement testing - CEF, Word Classifier

• Graduate students- Students: Stocktaking for portfolio and self-assessment

- Skill : accuracy, reading comprehension, register

->http://millennium.arts.kuleuven.ac.be/ctestftp://engels.vvkso-ict.com/engict/wordclassifier2.zip

Reporting

- Cd-rom: Platform independent- Eg via e-mail or via ftp adres for larger groups

- Learning platform- Within QMQ: added application (grade book)

- Follow-up over different years, for - groups of students- texts used

ILT-version

- Closed Test battery- customized (difficulty + content)

- Authoring tool- BB integration for

- tests: community (limited)- Reporting in grade book + per group

- exercises: in BB (free access)

3. Validation

- Validity : - Text selection: competence prediction

- Reliability: Compatibility- c-test performance and other tests

- User Friendliness- For authors- For students

- Test Difficulty- Fatigue and acquaintance

Validity : texts

LanguageWord frequency: Word Classifier

Domain Oriented: Business E.Content and Type

General EconomicsLanguage of Meetings &Negotiations

Memos/Letters/Agenda/ Minutes/ Reports/Newspaper Announcements/

Summaries/abstractsNews flashes and internal communicationHedging

Length

For a valid text selection, we need textsspread over different text types

of a limited length

• with a clear text function, • with specific economic purposes, not anecdotal, • with lexical input scanned byWordClassifier, • and a content selection in view of the target audience

4. Questions and Remarks

• CEF compatibility: clearcut parameters forLevels?

– B2 level? C1 level?• Tool for students: indicator of level? general E vs

business E vs... • Motivation: progress is slow...• Group intuition and individual intuition

– Lexical difficulty, text choice, content• Learning platforms and e-learning: in need of

flexible tools

Recommendations

• Assess lexical difficulty (eg WCl)• Taxonomy of texts

– Length, Content, Register, Type• Test efficiency

– Exercise with feedback, Exam : retake• Integrate tool

– Autonomous, platform, reporting

• For all these years of help, trust, friendship as a good working mate....